|
Friday January 21
The day shift began with the Antiproton Source (Pbar) stacking, with the Tevatron (TeV) in shot setup, and with MiniBooNE taking beam, but with the Meson experiments MCenter and MTest off due to shot setup.
At 10:14 AM, while getting ready to ramp the TeV, Operation received an alarm for the TeV sector D1 wet engine. The cryo system technicians at CHL said the engine was going bad.
At 10:21 AM, the TeV suffered a quench at sector D1.
Pbar resumed stacking 10:28 AM.
Operators made the Collider Detector at Fermilab (CDF) and D-Zero (D0) ready for access at 11:10 AM.
Operations established beam to MCenter at 11:24 AM.
At 11:57 AM, cryo system techs reported that D1 wet engine required a complete overhaul, which would be about a six-hour job.
Operators began making the TeV ready for access at 12:50 PM.
Operations reported seeing the first neutrinos at the MINOS near detector.
Techs came out of the TeV access at 4:34 PM.
NuMI experimenters finished using beam at 9:01 PM.
At 10:06 PM, Operations reported that the Pbar emittances were very high. At 10:18 PM, Operations halted stacking due to fix the emittance problem. Stacking resumed at 10:31 PM.
Saturday January 22
The midnight shift began with Pbar stacking, with the TeV waiting for sector D1 cryo system to recover, and with MiniBooNE, MCenter, and MTest taking beam.
At 12:47 AM, the TeV suffered a quench at sector D1. Operators discovered that even though they had received a cryo system permit, the cold compressor was off.
At 1:08 AM, Operations received an alarm for the D1 cryo system building. Operators arrived and found a helium leak. They crashed the building, which shut everything down, and called in experts.
The Cryo Coordinator reported that the picture above shows a high pressure relief venting. The real problem was due to an upstream bayonet can.
Operations put the Recycler in shot setup at 7:18 AM. By 8:29 AM, the Recycler had stashed seven shots of antiprotons. Pbar resumed stacking.
Operations reestablished beam to MTest and MCenter at 9:23 AM.
Operations began sending beam to NuMI for studies at 10:22 AM.
A cryo system tech reported that the D1 system was ready to go at 6:20 PM.
Operations put the TeV through a dry squeeze at 7:50 PM.
NuMI experimenters asked to have their beam stopped at 8:47 PM.
Operations put the TeV into shot setup at 8:57 PM.
At 9:09 PM, a TeV quadrupole power supply (C:B0Q2) tripped off on a ground fault indication. Operators couldn't reset the supply; they called an expert who asked to have power supply techs come in and conduct a hipot.
At 9:31 PM, a Linac RF station (KRF1) began tripping off repeatedly. An expert came in to investigate.
At 9:49 PM, Operations reported that stacking had been down for about twenty minutes due to an emittance blowup. About 10mA of the stack was lost.
KRF1 came back on line at 10:18 PM. Pbar resumed stacking.
At 10:48 PM, B0Q2 showed another problem. Operators replaced a firing card, but this didn't help.
Power supply techs discovered the problem with B0Q2 and at 11:50 PM estimated an hour to repair.
Sunday January 23
The midnight shift began with Operations monitoring the stash, with the TeV shot setup on hold due to B0Q2 problems, and with MiniBooNE taking beam.
At 1:24 AM, the TeV quenched at D0.
MCenter requested a negative beam tune at 1:54 AM. Operators changed the polarities on several power supplies. Two and a half hours later, operators returned the beamline to its normal parameters.
At 2:40 AM, Operations reported that the TeV sector E3 VCB would not close. Operators disabled feeder 23 so that techs could investigate and fix the problem.
At 3:54 AM, cryo tech reported that the TeV sector E2 wet engine was failing due to an exhaust valve. Experts were called in to make repairs at 5:41 AM.
Operations reported that techs had cleaned the E3 VCB at 6:46 AM and that the switch was closed. Operators had the TeV racked in by 7:28 AM.
NuMI began taking beam around 9:12 AM.
At 9:19 AM, B0Q3 tripped off. Eventually, it reset okay.
CHL reported at 9:36 AM that the E2 cryo system seemed to be recovered, but they wanted to monitor it for a while.
Operations put the TeV through a dry squeeze at 10:20 AM.
Operations put the TeV into shot setup at 11:06 AM.
At 1:06 PM, the TeV suffered a sector BB quench. Operators believed that a beam problem caused the quench. B0Q2 did trip off, but it was after the quench.
Operations put the TeV through a dry squeeze at 7:51 PM to check out B0Q2.
A TeV expert came in at 8:44 PM to conduct a study.
A Recycler expert came in at 8:51 PM to investigate a cooling problem.
At 9:10 PM, two Booster RF stations (BRF10 & 18) tripped off. Operators eventually got the stations back on.
At 10:29 PM, BRF10 tripped off again and wouldn't reset. Operators bypassed the station.
Operations put the TeV into shot setup at 10:39 PM.
At 11:27, operators discovered a separator at C49 in the wrong polarity. After trying to reset it, operators called in an expert.
Monday January 24
The midnight shift began with an expert working on the C49 separator, with the TeV shot setup on hold, with operators trying to resolve Booster instabilities, and with MiniBooNE taking beam.
The C49 separator was reset at 12:24 AM. Operations resumed shot setup.
Operations established store 3943 at 1:15 AM with an initial luminosity of 61.37E30. Pbar resumed stacking.
At 3:33 AM, the Pbar Debuncher interlock tripped off due to a LCW flow problem on PMAGV. Operators contacted an expert and he got the device back on. Pbar resumed stacking at 4:30 AM.
At 5:37 AM, Operations reported that the TeV sector E2 wet engine was failing. The Cryo Coordinator shut the engine off and put the house into JT mode.
At 5:51 AM, the TeV quenched at E2.
The Plan for Monday
The plan for today is to stack while techs complete the E2 wet engine repairs, which is estimated to occur around 4 PM. As soon as the work is completed, Operations will put the TeV into shot setup. During this work, the Run Coordinator will allow a CDF and AP0 access. Also, after the stack reaches a 100mA, Booster will be allowed to conduct a study.
Accelerator Update Archive
More Information
For Tevatron luminosity charts and the current status of Fermilab's
accelerators and detectors (live!), please go to Fermilab Now
Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.
| 