Accelerator Update for November 2, 2007 - November 5, 2007 |
Friday November 2 The day shift began with Operations monitoring the Tevatron (TeV) store 5677 (21.32E30) and a Recycler stash (100.9E10), with the Antiproton Source (Pbar) stacking (103.6E10), with the Main Injector (MI), Recycler, Booster, and Linac operating normally, with the Collider Detector at Fermilab (CDF) and D-Zero (D0) using the store, and with beam to the SciBooNE / MiniBooNE experiments. At 8:30 AM, Operations reported that the output from the I- Source was dropping. The Neutron Therapy Facility (NTF) reported at 10:30 AM that they had finished treating their last patient for the day. Operations reported at 10:46 AM that Linac had been turned off and Pbar put in standby for about 30 minutes while Linac experts began work on a Klystron RF station (KRF2). At 11:49 AM, LRF5 tripped off and would not reset. Operators investigated and managed to get the station on 15 minutes later. Operations terminated store 5677 at 11:50 AM. A TeV RF expert began work on TRF7. He was done 30 minutes later. At 12:22 PM KRF5 tripped off on a coil power supply fault and LCW flow. It reset. TeV experts began a proton tune-up around 12:40 PM. A TeV expert reported at 1:10 PM that proton coalescing looked good. At 1:24 PM, LRF4 & 5 tripped off. They both reset. Operations reported at 2:02 PM that a scheduled Meson area power shutdown began. At 2:38 PM, the MI power supply interlock loop tripped off. An expert changed out a control chassis at MI-30. Beam resumed about 30 minutes later. The expert had found a bad solder joint. At 3:22 PM, CDF experimenters reported a problem with their silicon detector. They said they would not be ready to collect data until 6 PM. Operations reported at 4 PM that the Recycler had stashed antiprotons once during the day shift. Operators reported at 4:15 PM that they had completed a search and secure of NuMI / MINOS. Operations put the TeV through a dry squeeze at 4:28 PM. Safety experts reported at 4:46 PM that they had begun testing the NuMI/MINOS safety system. Operations put the TeV into shot setup at 4:58 PM. NuMI experimenters and safety experts received approval to make a power-on access to NuMI at 5:30 PM to inspect and adjust the horn. CDF experimenters reported at 5:49 PM that their silicon detector would not be ready to run until 8 PM. A safety expert reported at 6:12 PM that they were making the first of two power-on accesses to NuMI. Operations reported at 6:33 PM that after the NuMI power on access ended, Operators would turn on and run all of the NuMI power supplies, except for the Lambertsons, until Monday morning. A Recycler expert began an Electron Cooling study at 7:06 PM. Operations established store 5680 at 7:12 PM with an initial luminosity of 97.23E30. At 9:35 PM, a Linac quadrupole power supply (L:QPS102) came into alarm. Operators readjusted the supply to its normal current. At 11:44 PM, KRF1 tripped off. It reset. At 11:48 PM, a Booster vacuum pump (B:IP11) tripped off. It reset. At 11:56 PM, LRF3 tripped off. It reset. Operations reported at midnight that the Recycler had stashed once during the evening shift. Saturday November 3 The midnight shift began with Operations monitoring store 5680 (41.89E30) and the stash (103.0E10), with Pbar stacking (48.7E10), and with Sci /MiniBooNE taking beam. At 12:24 AM, RF stations LRF3, 4, & 5 tripped off. Everything reset. At 1:22 AM, the 750keV area buncher tripped off. It reset. At 1:44 AM, KRF4 & 5 tripped off due to a LCW flow fault. They reset. At 2:40 AM, KRF5 tripped off on a LCW flow fault. It reset. At 3:15 AM, Operations reported that the Linac QPS102 was drifting in and out of alarm. Operators swapped its readback card with QPS104. At 3:30 AM, BRF16 tripped off. It reset. At 4:35 AM, LRF3 tripped off. It reset. At 4:51 AM, the 750 KeV area buncher tripped off. It reset. At 6:49 AM, a Tevatron Electron Lens (TEL1) tripped off on a vacuum interlock. After talking with an expert, operators turned it back on. At 7:55 AM, Operations reported that emittances in the Pbar Accumulator grew and the stack rate dropped to zero. Operations decreased the stacktail power and everything returned to normal. Operations reported at 7:58 AM that the Recycler had stashed once during the midnight shift. Operations terminated store 5680 at 10:31 AM. Operations put the TeV into shot setup at 10:43 AM. Operations established store 5681 at 12:59 PM with an initial luminosity of 98.57E30. At 1:19 PM, KRF4 tripped off on a coil comparator fault. It reset. At 2:51 PM, the 750 KeV area buncher tripped off. It reset. At 3:06 PM, LRF2, 3, & 4 tripped off. They all reset. At 3:22 PM, BRF6 tripped off. It reset. Later, operators discovered that BRF6 was sparking a lot and causing beam position problems. Operations contacted an expert who said he’d come in to investigate. Operations reported at 5:44 PM that the fire tech had repaired the MC7 & 8 VESDA. At 6:54 PM, LRF2 tripped off. It reset. At 7:10 PM, KRF3 tripped off on a coil comparator fault. It reset. Operators reported at 7:47 PM that they had to fill the LRF3 water skid. Operations reported at 8:01 PM that the Recycler had stashed three times during the last 12 hours. At 8:07 PM, KRF3 tripped off on a coil comparator trip. It reset. At 8:39 PM, Operations reported the first of two Pbar lithium lens trips. Operators contacted an expert both times before resetting the lens. At 10:46 PM, KRF3 tripped off on a coil comparator fault. It reset. At 21:09 PM, the Linac buncher tripped off. It reset. Sunday November 4 The midnight shift began with Operations monitoring the store 5681 and stash, with Pbar stacking, and with Sci /MiniBooNE taking beam. At 1:03 AM, the Pelletron tripped off. It reset. At 1:06 AM, Operations reported that the Sci/MiniBooNE beam had tripped off several times over the last couple of hours due to a low power supply current problem (MBEXPS). This continued to be a problem throughout the rest of the midnight shift. Operations terminated store 5681 at 5:31 AM. Operations put the TeV into shot setup at 5:42 AM. At 7:07 AM, the 750 KeV area buncher tripped off. It reset. Operations established store 5682 at 7:55 AM with an initial luminosity of 89.84E30. Operations reported at 8:31 AM that the Recycler had stashed once during the last 12 hours. At 10:42 AM, Operations reported a problem with the Pbar Debuncher cooling. Operators adjusted it so it would keep operating. An access will be needed to make repairs. At 11:51 AM, the 750 KeV area buncher tripped off for the first of two times in about two hours. It reset. At 2:35 PM, LRF2 tripped off. It reset. At 4:04 PM, LRF3 tripped off. It reset. More water was added to this system. At 6:03 PM, Operations reported problems with the KRF2 cavity LCW loop. Operators investigated the water skid. Operations terminated store 5862 at 7:11 PM. At 7:47 PM, Operations reported that LRF3 had tripped off and that its water system level was low again. Operators discovered that the last two trips coincided with low water levels. Operations contacted an expert. Operations reported at 8:03 PM that the Recycler had stashed three times during the last 12 hours. Operations prepared Linac for an access at 8:17 PM to look for a LCW leak. At 8:49 PM, operators reported a LCW leak coming from the LRF3 penetration.
Operators continued to add water to this system, in order to keep it running, throughout the night to Monday morning. Operations put the TeV through a wet squeeze at 9:09 PM. Operations put the TeV into shot setup at 9:42 PM. At 11:30 PM, Operations reported that the P1 transfer line between MI and the TeV had tripped off for the first of three times in 13 minutes. It was reset after each trip. At 11:52 PM, Operations reported that the store was aborted at sequence 15. There was no quench. Operators contacted TeV experts. Monday November 5 The midnight shift began with Operations monitoring the stash, with Pbar stacking, and with Sci /MiniBooNE taking beam. A TeV expert reported at 12:20 AM that he was removing the helix changes at step 15. At 12:59 AM, Operations reported that a TEL2 corrector power supply (T:L2C5) had tripped off during the TeV abort and would not reset. Operations contacted experts. Operations prepared CDF and D0 for access at 1:25 AM. At 2:39 AM, LRF5 tripped off. It reset. Both D0 and CDF had come out of access by 3:28 AM. Operations put the TeV through a dry squeeze at 4:08 AM, but during the squeeze a quadrupole magnet (T:QB15) tripped off on a quench indication. Operators found a bad power supply; there was no quench. Operators had replaced the T:QB15 power supply by 6 AM. Operations put the TeV through a dry squeeze. A TeV expert began a study at 6:33 AM. Operations turned off all the NuMI power supplies at 6:56 AM. At 7:18 AM, KRF4 tripped off. It reset. At 7:33 AM, KRF7 tripped off. It reset. At 7:55 AM, Operations reported that the power supply they had used to replace T:QB15 was bad and that when it tripped off, all of the supplies in the rack tripped off, but everything reset. After Operations had finished putting the TeV through a dry squeeze there was a sector B1 corrector abort. Operators investigated and found that the power supplies were actually on and ramping and showing an abort status. Operations contacted the power supply group. Operations reported at 8:01 AM that the Recycler had stashed three times during the last 12 hours. The Plans for Monday The plans for today are to allow TeV and MI experts to conduct studies. The Run Coordinator will allow Linac experts to access, pull out the LRF3 penetration shielding, and make preliminary evaluation of the LCW leak. The Run Coordinator hasn’t determined the time yet, but the best guess is that there will be a TeV access starting at 6 AM on Tuesday; it will last 6 to 12 hours. Its purpose is to move the low beta magnets at CDF. Other machines will be allowed to make accesses for maintenance work during this period. More Information Comments and Suggestions |
last modified 11/05/2007 --> email Fermilab |
FRLsDFx9eyfrPXgV