Accelerator Update for July 31, 2006 - August 2, 2006
Monday July 31
The day shift began with Operations monitoring store 4862 and a Recycler stash, with the Collider Detector at Fermilab (CDF) and D-Zero (D0) using the colliding beam, with the Antiproton Source (Pbar) off due to problems with A:IB, with the Linac, Booster, Main Injector (MI), and Recycler operating normally, and with NuMI and MiniBooNE taking beam.
At 8:23 AM, a Linac quadrupole power supply (L:QPS105) tripped off twice in eight minutes. It reset both times.
At 9:05 AM, Linac technicians swapped the control cards between QPS103 and QPS105. QPS103 went out of tolerance and had to be adjusted. At 9:11 AM, the cards were swapped back and QPS105 went out of tolerance. A new card was installed in QPS105 at 9:44 AM.
The power supply expert working on the Accumulator power supply A:IB reported that a cable from the transformer was bad. He had A:IB on by 10:29 AM.
Pbar tried to resume stacking at 11:10 AM, but an RF station (ARF1) wasn’t ramping. At 11:24 AM, problems with BPMs arose. At 11:38 AM, Operations reported that Pbar was stacking, but at a poor rate due to a problem with the control program (Overthruster).
Experts began tuning Booster and MI for higher intensities at 12:09 AM.
FESS reported at 1 PM that divers resumed cleaning the A1 pond strainer. They had the job finished thirty minutes later.
At 1:12 PM, all the Booster RF station began sparking due to the tuning and BRF3 & 10 tripped off. The expert put back his most recent changes.
At 2:24 PM, Operations reported that a B0 compressor had tripped off. CHL was aware of the problem. They have 26 compressors running.
At 2:40 PM, Operations reported a problem with a MI-10 CAMAC crate; corrector magnet power supplies were not ramping. This caused problems with the MI orbit and with NuMI beam. Controls experts replaced a card at 2:54 PM.
At 4:55 PM, BRF13 & 14 tripped off due to a vacuum burst. Everything reset.
At 5:21 PM, Operations reported that the Tevatron (TeV) sector B0 electronics room temperature was rising. FESS personnel placed a portable cooler in the room and the temperature began to drop.
Operations began preparing the Recycler at 6:17 PM to stash antiprotons.
At 6:53 PM, CDF reported that they had to turn off their detector due to the heat. (Much later, investigators found that a valve had been opened to divert pond water back to the pond. This valve was closed and cooling improved.)
At 7:02 PM, Operations reported that not all the beam was getting extracted from MI to the Recycler. Operators contacted a MI expert who said there was a large phase offset between Pbar and MI. The stash ended.
A Pbar expert came in at 9:44 PM to prepare for the next stash.
The Recycler began stashing at 11:01 PM. The transfer looked much better. The stash ended and Pbar resumed stacking at 11:18 PM.
Tuesday August 1
The midnight shift began with Operations monitoring store 4862 and the stash, with Pbar stacking, and with NuMI and MiniBooNE taking beam.
At 12:54 AM, many of the Pbar devices stopped reporting. Operations rebooted AP1001 (a network switch) and this brought back A:IB and a couple more devices, but there were still problems and many alarms. Operations contacted a controls expert.
At 2 AM, Operations reported that all of the Main Control Room consoles seemed to be failing.
The controls expert reported at 2:02 AM that he traced the Pbar problems to a switch in AP-10. An operator power cycled the switch. Everything came back except for the Pbar flying wire, which will get fixed during the day shift.
The Recycler began preparing to stash at 3:26 AM. Problems with GPIBs, a Pbar beam permit trip, and with the reverse protons not making it into the Accumulator caused Operations to halt the stash and resume stacking.
Operations attempted at 4:26 AM to prepare the Recycler to stash. The Pbar beam permit tripped again and the reverse protons never made it. Operators contacted an expert.
Operations terminated store 4862 at 5:02 AM.
CDF and D0 were ready for access by 5:19 AM.
At 5:28 AM, a Recycler expert noticed that two power settings were wrong. Operators set them back to a previously used setting and the transfer went from Pbar to MI, but then fell out of MI. The investigation continued.
At 5:28 AM, the Accumulator IKIK injector shutter was off, which halted stacking.
Pbar resumed stacking at 5:41 AM, but poorly.
A Pbar expert suggested at 6:20 AM that Operations run the stacking aggregates again. This worked, but then they restored a file and the extraction and injection shutters alarmed. Operators restored a different file at 6:31 AM and Pbar resumed stacking.
D0 came out of access at 6:54 AM.
Earlier, a TeV expert came in to look at a kicker (F17K2) problem. At 7:02 AM, he reported that the problem was in the tunnel. Operators began preparing the TeV at 7:10 AM for access.
CDF came out of access at 7:59 AM.
A Pbar expert reported at 8:11 AM that he suspected H926 was the cause of the shot problems. Another expert will investigate further.
D0 accessed their collision hall at 8:20 AM.
The TeV experts came out of F sector at 9:44 AM.
MiniBooNE resumed taking beam at 10:11 AM.
D0 came out of access at 10:16 AM.
NuMI resumed taking beam and Pbar resumed stacking at 10:26 AM.
At 10:30 AM, LAM42 was holding off the MI beam permit.
A power supply expert investigating the MI Lambertson problem (I:LAM42) reported at 12:25 PM that he found a bad magnet Klixon. He bypassed the Klixon and turned on the power supply.
NuMI resumed taking beam and Pbar resumed stacking at 12:28 PM.
A power supply expert reenabled the MI I:LAM42 Klixon at 2:01 PM.
TeV experts began a study period shortly after 3:19 PM.
At 5:03 PM, Operations reported a small glitch; all the Booster RF stations tripped off, including Lam 60 & 61, and ARF1. Everything reset.
The Recycler began preparing to stash at 5:12 PM. The stash ended and Pbar resumed stacking at 5:54 PM.
Operation put the TeV into shot setup at 7:27 PM.
Operations established store 4868 at 9:08 PM with an initial luminosity of 120.16E30.
The Recycler began preparing to stash at 9:52 PM. The stash ended and Pbar resumed stacking at 10:24 PM.
Wednesday August 2
The midnight shift began with Operations monitoring store 4868 and the stash, with Pbar stacking, and with NuMI and MiniBooNE taking beam.
Operations reported at 2:54 AM that a stash, which had started earlier, ended and stacking had resumed.
At 3:06 AM, the Klystron Debuncher tripped off and would not reset. Operators called in experts.
Linac experts repaired the Klystron Debuncher and got the H- Source running, though at a limited capacity. Beam resumed at 7:26 AM.
Stacking resumed at 7:30 AM.
The Plans for Wednesday
The plans for today are to stack, stash, and store.
Comments and Suggestions
|last modified 08/02/2006 email Fermilab|