 |
Accelerator Update - July 2-July 7
Wednesday July 2 The Main Injector (MI) tripped off at 9:40 AM on a ground fault indication. Operators prepared the area for access. Experts reported finding a bad quadrupole magnet in MI at 11:45 AM. Technicians began the magnet's replacement. Operations made many areas ready for accesses. The Tevatron (TeV) store 2740 aborted at 1:28 PM due to a possible RF glitch. Operators made the TeV ready for access. Operators made the Collider Detector Facility (CDF) and D-zero (D0) ready for access. Cryo techs began performing maintenance on wet engines around the TeV at 4:48 PM.
Thursday July 3 Operations received a fire alarm for the MI12 service building at 1:49 AM. The building was full of smoke, but Fire Department found no problems. A MiniBooNE expert suspected that it was due to a smoking vacuum pump. At 3:07 AM, the techs working on the MI quadrupole replacement reported that they were finished and exiting the tunnel. A Linac RF station (KRF1) tripped off at 4:40 AM on a LCW flow fault. Operators found and tightened a loose connection. Operations established beam to MiniBooNE at 5:15 AM. Operations put the TeV through a dry squeeze at 5:26 AM. Experts then began a beam maintenance study period. Pbar resumed stacking at 5:52 AM. Operators discovered a MI air compressor running continuously at 6:29 AM. They thought the problem might be due to an air leak in the MI60 service building. Operations informed an expert and he had the leak repaired by 7:29 AM. Operations discovered a MI vacuum problem at 7:26 AM. The leak is somewhere in the region of Q637. Operators informed experts. Operators and a Pbar expert investigated a problem at 8:26 AM, with beam not making it to the Pbar debuncher. Later, they discovered a timing problem with a Pbar kicker (IKIK). A tech began tracing cables and the timing problem cleared itself. This will require further investigation at a later date. At 12:57 PM, a Booster power supply, (B:MH2) began acting up. An expert cycled the supply, which seemed to fix it, but twenty minutes later it was still causing problems. This interfered with MiniBooNE beam. At 1:36 PM, Booster RF stations 14 and 15 tripped off due to a vacuum trip. Then, at 2:07 PM, station 13 and 14 tripped of twice due to a vacuum problem. They all reset okay. TeV experts completed their beam maintenance studies and Operations put the TeV into shot setup at 6:56 PM. A high temperature problem at the sector E4 cryo system building delayed shot setup. It was restarted at 8:42 PM. Operations established store 2746 at 10:28 PM with an initial luminosity of 17.92E30. Operators then prepared MI for a Recycler access. The TeV suffered a sector E1 quench and lost the store at 10:37 PM. Operations lost communication with all the TeV cryo systems at 11:30 PM. Operations investigated. This took about three hours to resolve.
Friday July 4 By 12:09 AM, MI was ready to be turned back on. Pbar stacking resumed at 12:38 AM. A Pbar Debuncher RF problem (DRF1-5) appeared at 12:44 AM. The problem was traced to equipment in the tunnel, which will be repaired at some future date. A Linac RF station (KRF4) tripped and would not reset at 1:51 AM. Operators investigated and then called in an expert. He had the station operating by 3:35 AM. Pbar resumed stacking. Operations put the TeV through a dry squeeze at 3:55 AM. Operations loaded a 12x0 study store into the TeV at 6:21 AM. A Booster kicker (B:MKS05) stopped firing at 4:35 AM. Operations retimed the other kickers and resumed sending beam to Pbar. An expert came in later in the morning and repaired this device. Operations put the TeV into a dry squeeze at 11:08 AM. A Booster power supply (MH2) caused a minor interruption of MiniBooNE beam at 11:54 AM. It acted up again at 12:24 PM. At 1 PM, a Linac RF station (KRF4) tripped off at 1 PM and would not reset. Operations called an expert who suggested some adjustments that had the station on twenty-two minutes later. The Booster power supply, MH2, was turned off for this time. MI suffered a MECAR failure at 2:52 PM. It took experts about two hours to track down and replace a bad CAMAC module. Pbar resumed stacking at 4:39 PM. Booster MH2 regulation problems halted MiniBooNE beam at 4:54 PM. At 5:55 PM, MECAR again failed and would not reset. After an hour plus of investigation, which included contacting an expert, operators found another CAMAC module problem. Operations lost communication with the TeV cryo systems at 6:24 PM. Operators were able to reset the QPM at 7 PM. A Booster power supply (ORBUMP) tripped off at 6:36 PM for the first of four times over a half-hour period. The problem was due to a bad door switch. Pbar resumed stacking at 7:02 PM. Operations reestablished beam to MiniBooNE at 7:12 PM. At 10:05 PM, just after the last load of antiprotons, the TeV suffered an E4 quench. A device (LBSEQ), which was set at the wrong value, caused the quench. Pbar resumed stacking.
Saturday July 5 Two Booster RF stations (BRF11 and 12) tripped off at 12:43 AM due to vacuum problems. Operations put the TeV through a dry squeeze at 1:52 AM. Booster suffered a vacuum burst at 4:23 AM. The TeV study beam aborted at 6:26 AM due to a ground fault indication at sector C3. Operations called in an expert to make repairs. He completed his work at 8:30 AM. A Linac RF station (KRF5) tripped off at 9:08 AM and would not reset. It has a bad coil. Operations called in a Linac tech to make repairs. At 9:10 AM, during the TeV turn on, feeder 23 tripped off. A high voltage expert came in and investigated all day, nine hours. This trip caused lots of problems with many devices. Eventually, a sector D3 VCB was found to be damaged. It would take two days to repair this so a different set of TeV power supplies was switched into action. A ComEd power glitch hit the lab at 9:52 AM. A number of devices tripped off, but all came back okay. The TeV was already off so it didn't have any impact on beam. However, this added to the feeder problem investigation. The MI cavity LCW system suffered a high conductivity alarm at 1:23 PM. An expert confirmed the problem. A tech was called in and he had the problem fixed by 3:17 PM. All the MI RF stations tripped off at 6:20 PM. Operators reset the stations. Pbar resumed stacking at 6:36 PM. Operations hipotted the TeV and found a ground fault at sector B2. Experts were called in.
Sunday July 6 At 12:35 AM, the TeV experts took a B2 power supply out of the circuit and the hipot looked good. Operations put the TeV through a dry squeeze at 2:10 AM. At 2:13 AM, Operations received a fire alarm for sector A3 service building. Operators discover the building full of smoke. Operations dumped the ramp and opened the VCBs. A TeV power supply had overheated. The TeV power supply techs had been here all day and evening dealing with problems. They needed to go home and rest before they could start any repairs. A thunderstorm struck the lab at 2:55 AM. Lightening caused many glitches over the next hour. All the TeV RF stations tripped off at 3:09 AM. Operations had them back on ten minutes later. Pbar resumed stacking at 3:30 AM. Operations reported that the I- Source had been sparking all night. This problem seemed to be humidity related, but operators found no actual cause. The TeV cryo system controls went away again at 7:20 AM. A reboot restored communications. Pbar emittances blew up at 7:30 AM. Pbar began loosing beam from the stack due to this at 7:57 AM. Operations called in a Pbar expert. Operators made the TeV ready for an access at 9 AM. The search for a possible LCW flow restrictions. The I- Source tripped off at 9:19 AM. Operations could not reset the power supply. A Preaccelerator expert came in and had us switched over to the H- Source by 10:37 AM. Pbar resumed stacking. He continued to tune on the source until noon. Operations resumed sending beam to MiniBooNE at 11:04 AM. Recycler experts began a study period at 12:47 PM. Linac suffered ten beam related trips between 1 PM and 1:30 PM. Operators investigated the problem. Pbar LCW temperatures alarmed high 2:19 PM. Operations contacted the Duty Mechanic. He investigated CUB, but the system was cooling at its fullest capacity. TeV vacuum pumps at sector F4 tripped off 3:04 PM. Operations called in vacuum techs to make repairs. Operations turned off MI for the TeV access at 3:50 PM. The access began at 4:40 PM. The techs completed their work and were out of the tunnel by 5:33 PM. During the MI turn on at 5:57 PM, a quadrupole power supply (Q703) would not follow its setting. This held off the beam permit for the P1 transfer line. An expert came in and replaced a driver card. This took about two hours of work. Operations reestablished beam to MiniBooNE at 6:02 PM. Operations sent some beam into the TeV at 8:35 PM. The H- Source tripped off at 11 PM and would not reset. Operations called in two experts to make repairs.
Monday July 7 Pbar began suffering from emittance problems at 1:30 AM. Booster RF stations 1 and 2 tripped off due to a vacuum burst at 2:50 AM. They both reset okay. At 3 AM, the Pbar emittance problems were still too high. Operations called in an expert who had the emittances dropping by 3:30 AM. A TeV expert repaired equipment between A2 and A3 at 4:25 AM and that resolved the TeV regulation problems. Operations put the TeV through a dry squeeze at 4:29 AM. The PreAcc experts changed out two insulators in the H- Source. Operations sent beam to MiniBooNE at 4:40 AM and resumed Pbar stacking at 4:43 AM. MI coalescing looked very bad at 5:54 AM. Operators called in an expert who began tuning at 7:31 AM. Operations reported at 7:43 AM that the H- Source output intensity was stable but reduced.
Today's Plan
More Information
Comments and Suggestions
|
| last modified 7/7/2003 email Fermilab |
FRLsDFx9eyfrPXgV