 |
Accelerator Update - July 14-July 16
The integrated luminosity for last week was 8467.4 nb-1.
Monday July 14 A Linac RF station (KRF4) tripped off at 9:50 AM due to an LCW flow switch failure. An operator applied a "mechanical transient" to the switch and the station was back up within fifteen minutes. A technician began work, at 10:51 AM, on the vacuum pressure input to two Booster RF stations (BRF13 & 14). The inputs had been disabled on an earlier shift. A tech, at 11:54 AM, replaced a Booster power supply for a quadrupole magnet (B:Q10). The output of this supply was drifting. Operations resumed sending beam to MiniBooNE at 12:01 PM. The Linac beam in the 400 MeV area tripped off at 2:29 PM. It tripped off four more times in the next half hour. A Main Injector (MI) expert began conducting slipstacking studies at 3:35 PM. Switchyard experts began a study period at 8 PM. At 10:15 PM, a fire alarm was issued for Tevatron (TeV) sector F0. At the same time a quadrupole power supply at TeV sector F0 (Q703) alarmed and Operations lost the ability to monitor many devices in the P2 transfer line. Operators responded to the area along with the Fire Department. At 10:18 PM, half of the TeV RF stations tripped off, which caused the loss of store 2786. Curiously, the same thing happened at nearly the same time last night. Experts were called in to investigate. Operators discovered the Q703 power at F0 supply spewing smoke and water at 10:32 PM. Operations valved off the water and called in techs to check the LCW system and make repairs.
Tuesday July 15 Five MI RF stations tripped off at 12:03 AM. The D-zero (D0) experiment accessed the collision hall at 12:12 AM. The Collider Detector Facility (CDF) requested an access at 12:29 AM. The MI expert completed her studies at 12:44 AM. CDF came out of access at 2:36 AM and D0 came out of access at 4:06 AM. At 4:35 AM, a thunderstorm rolled across Fermilab. A lightning strike caused a power glitch. This was followed by the TeV sector F1 cryo system going into a quench recovery with no apparent quench. Then the MI ramp went down and all the MI VCBs opened. The MI beam permit went away due to an interlock problem at the NUMI stub gate. And then the cryo system readbacks disappeared for the sector C4 system. The C4 problem turned out to be the worst of the problems. At 5:55 AM, the C4 cryo system went down hard. Operators smelled something burned from its controls system. Its wet engine had tripped off. Also, a 480-volt AC contactor tripped and brought down a bulk power supply. Operations called for help. At 6:01 AM, MiniBooNE tripped off due to an airflow problem. Operators reset the system, but it tripped again. After several more trips an interlock tech came in and adjusted a sensor. The experiment was back up and running by 7:30 AM. The techs working on the Q703 finished their repairs around 6:09 AM. Operations turned on the P1 and P2 transfer line power supplies at 6:17 AM. MI vacuum control problems appeared at 6:55 AM. Operations called in an expert who tried to open the MI-20 vacuum-valves manually, but with no luck. Experts continued searching until they found problems with other crates. (They eventually had everything working by 1 PM.) An expert restored power to C4 and the readbacks for the cryo system returned at 8:27 AM. Booster RF stations 13 and 14 tripped due to a vacuum burst at 9:05 AM. ACNET problems began interfering with Main Control Room consoles at 9:30 AM. An expert investigated and had things back near to normal at 9:41 AM. Two Booster RF stations tripped off at 10:17 AM due to vacuum problems. They reset okay. Work on Booster RF stations 13 and 14 finished at 1 PM. Operations resumed sending beam to MiniBooNE and Pbar resumed stacking. An MI expert began conducting slipstacking studies at 1:34 PM. The MiniBooNE airflow beam permit tripped at 2:24 PM. An expert had the system back up within twenty minutes. At 4:24 PM, four MI RF stations tripped off. After multiple resets the stations all came back on. The MI expert completed her slipstacking studies at 5:15 PM. Booster suffered a vacuum burst at 6:15 PM. Everything reset okay. MiniBooNE requested that their beam be turned off at 7:20 PM so they could fix a problem with their detector. They were back up and running about an hour later. Switchyard experts completed their studies at 7:20 PM. At 8:31 PM, Operations noticed a drop in the Pbar LCW pressure. Operators called the Duty Mechanic to investigate the pumps at CUB. Operations noticed the Pbar stack emittances grow at 8:52 PM. Operators notified an expert. The Duty Mechanic reported, at 9 PM, a possible nitrogen leak as the cause of the Pbar LCW problems. MiniBooNE again asked for their beam to be stopped, at 11:38 PM, while they worked on data acquisition problems. The C4 cryo system came on line at 11:40 PM and Operations started the TeV turn-on. The TeV ramp went away due to a sector C4 lead failure at 11:59 PM. This was due to a weak coupler on a card in its controls chassis. It was replaced.
Wednesday July 16 Another Pbar expert came in to troubleshoot the Pbar emittance problem at 12:08 AM. The C4 lead turned off at 12:30 AM and dumped the TeV ramp again. The Cryo Coordinator investigated. The TeV suffered a sector DA and DB quench at 1:15 AM. The quench occurred just as operators were exercising the C4 leads. Operations contacted the Duty Mechanic about the Pbar LCW problems at 1:21 AM. The problem was still being investigated. Cryo experts reported at 1:26 AM that they would come in and work on the C4 readbacks. Operations contacted another expert, at 1:55 AM, to come in and help with the Pbar LCW investigations. All cryo systems were reported as ready at 2:31 AM. Operations again started the TeV turn-on. The expert at CUB found a bad pump for the Pbar LCW system at 3:26 AM. He switched the system to a different pump. At 3:49 AM, operators reported that they were still investigating why a C4 abort permit would not clear. After talking to an expert and trying several remedies, operators managed to reset the abort permit at 5:37 AM. A Recycler expert began a study period at 4 AM. At 4:06 AM, the Linac 400 MeV area tripped off. It tripped off six more times over the next hour. A Booster RF station (BRF18) tripped, at 6:03 AM, and wouldn't reset. Operations contacted an expert. Operations put the TeV through a dry squeeze at 7:23 AM.
Today's Plan
More Information
Comments and Suggestions
|
| last modified 7/17/2003 email Fermilab |
FRLsDFx9eyfrPXgV