Accelerator Update for February 11, 2009 - February 13, 2009
Wednesday Feb 11
The day shift began with the accelerator complex in recovery from a power outage caused by a transformer failure at 11 PM on 2/10/09.
Operations reported at 9:34 AM that TCLK was working.
Operations prepared the Booster for an access at 10:20 AM.
FESS personnel reported at 10:26 AM that they were cleaning basket strainers in MI.
TeV experts reported at 10:52 AM the sector B3 electronic dump switch would be repaired and it would remain an electronic dump switch; the sector C2 dump switch would be converted back to a mechanical dump switch; and that they were investigating a problem with the sector F3 electronic dump switch.
Operations logged pictures of the damaged TeV sector B3 dump switch at 11:37 AM.
Operations prepared MI for access at 11:48 AM.
Operations prepared TeV F sector for access at 11:49 AM.
Operations reported at 11:50 AM that a new gas powered generator was installed and running the Cross Gallery and other areas covered by the failed transformer. The spare transformer out in the Proton Experimental Area failed its hipot. Another transformer was found at the Rail Head.
Operations logged the following pictures of the bad transformer being removed at 2:11 PM.
At 2:48 PM, Operations reported sending a pulse of beam to Booster, but it fell out at low field.
The Neutron Therapy Facility (NTF) reported at 3:09 PM that they had completed treating its last patient for the day.
Operations reported at 3:22 PM that the transformer at the Rail Head had passed its hipot. It will be placed on its pad behind the Cross Gallery tonight, but not hooked up due to the rain.
Operations prepared CDF for an access at 4:13 PM.
D0 came out of access at 4:18 PM. They began a cosmic ray study.
Operations reported at 4:40 PM that they had prepared MI and TeV sector F for interlock testing.
CDF came out of access at 4:40 PM. They began a cosmic ray study.
Operations reported at 5:10 PM that Booster was circulating beam.
At 5:46 PM, a Pbar Debuncher Bendbus power supply (D:IB) tripped off on a ground fault indication. Operators investigated, found nothing unusual, and reset the supply. It tripped off on the same indication. It reset.
Operations turned on MI and established a ramp at 6:26 PM.
At 6:40 PM, D:IB tripped off on a ground fault indication. Operations called in a power supply expert.
Operations had beam established to MiniBooNE and NuMI by 7:13 PM.
The power supply experts confirmed at 9:05 PM that D:IB had a cable ground fault located somewhere between the supply and the yard transformer. Operations informed the Pbar experts. They will decide what to do on Thursday morning. Pbar will not be able to stack.
At 11:06 PM, MIRF13 tripped off on a ground fault indication and would not reset. Operators bypassed the station.
Thursday Feb 12
The midnight shift found the accelerator complex still recovering from the power outage, with Recycler experts conducting studies, and with Pbar down due to a D:IB cable ground fault.
Vacuum experts accessed the TeV sectors A-E and the transfer hall at 5:35 AM.
At 6:39 AM, BRF14 tripped off for the first of six times in 30 minutes. It reset.
An RF expert brought MIRF13 back online at 7:51 AM.
A MiniBooNE expert began a study at 8:01 AM.
CDF accidentally dropped their interlocks at 9:06 AM.
At 9:26 AM, a Linac quadrupole power supply (QPS501) tripped off for the first of two times in 8 minutes. It reset. Operations informed Linac experts.
A Recycler expert began a beam measurement study at 10:23 AM.
Operations established full intensity beam to NuMI and MiniBooNE at 10:24 AM.
At 10:55 AM, a Recycler kicker (R:KPS2A) tripped off on a fail to fire indication for the first of three times six minutes. Operators reported that the trips were occurring every time protons were injected into the Recycler. It reset.
Cryo system technicians reported at 1:11 PM that they were beginning work on the TeV sector A3 wet engine.
A power supply expert reported at 1:58 PM that the Pbar power supplies work was finished.
Pbar began stacking at 2:19 PM.
Operators searched and secured the CDF collision hall at 3:08 PM.
NTF reported at 6:20 PM that they had completed treating patients for the day.
The Recycler proton study ended at 6:26 PM.
Operations established a QBS ramp on the TeV and completed two dump tests at 7:36 PM.
Operations reported at 7:58 PM that the TeV sector B3 dump switch was back on line and successfully tested.
At 10:23 PM, BRF7 tripped off for the first of two times in one minute. It reset.
Operations reported at 10:26 PM that they had the TeV turned on to 90 GeV and would leave it on all night.
At 10:40 PM, LRF3 tripped off and would not reset. Operations contacted experts.
Friday Feb 13
The midnight shift began with Operations monitoring a Pbar stack (141.7E10), with Linac experts working on the LRF3 driver tube, and with the TeV at 90 GeV.
A Linac expert reported at 12:08 AM that the LRF3 driver tube had a water leak. He was calling in experts to help make repairs.
Operations reported at 4:30 AM that they were beginning to prepare areas for the up coming power outage to switch to the new Cross Gallery transformer.
Operations prepared the TeV, MI, and Booster for access at 6:50 AM.
Controls experts reported at 7:32 AM that they were taking down the controls system in preparation for the power outage.
The Plans for Friday and the Weekend
The plan for today is to switch off power at 8:30 AM to connect the new Cross Gallery transformer. Power should be back by 10:30 AM and controls should be back within an hour or two after that. The plans for the weekend are to check out the TeV and all other machines and then begin stacking, stashing, and storing.
Comments and Suggestions
|last modified 02/16/2009 email Fermilab|
|Security, Privacy, Legal|