Accelerator Update for March 31, 2008 - April 2, 2008

Monday March 31

The day shift began with the Tevatron (TeV) in a dry squeeze, with Operations monitoring the Recycler stash (9.3E10), with the Antiproton Source stacking (38.3E10), with the Main Injector (MI) vacuum still recovering, Linac, and Booster operating normally, with the Collider Detector at Fermilab (CDF) and DZero (D0) waiting for the colliding beam, with NuMI having problems with QD111, and with the SciBooNE / MiniBooNE (BNB) experiments taking beam when available.

An RF expert reported at 8:24 AM that a MI RF station (MIRF1) was back in operation. He had replaced a grid power supply.

At 8:46 AM, a TeV feeddown sextupole power supply (C:S4C2B) had tripped off on a fuse fault. Operations contacted an expert who found that a LCW leak from another power supply higher in the rack had caused the trip. He fixed both supplies and repaired the leak.

At 9:21 AM, Operations reported a MI High Level RF Poor Vacuum alarm. It reset.

The Neutron Therapy Facility (NTF) reported around 9:44 AM that they had completed treatment on their last patient for the day.

Operations reported at 9:59 AM that TeV experts were going to reboot the TeV Low Level RF and then conduct a study.

Around 10:13 AM, the Booster West Anode Power Supply (WAPS) tripped off with no indication of a problem. At the same time, a Pbar Accumulator RF station (ARF1) tripped off. Everything reset. Operators checked with the High Voltage Coordinator and discovered that there had been a ComEd power glitch.

Operations resumed ramping the NuMI quadrupole E:QD111 at 10:54 AM to try and dry it out.

A NuMI expert asked Operations to turn off the NuMI horn at 11:28 AM.

Operations put the TeV through a dry squeeze at 11:32 AM.

At 11:49 AM, Operations reported that the Linac Debuncher water skid was vibrating excessively and causing control problems with the SRM mounted on the water skid. Linac experts tried to turn on an alternate pump, but it wouldn’t work.

Linac experts reported at 11:50 AM that they were tuning a Linac RF station (LRF1).

At 12:20 PM, NuMI experts turned off their troublesome quadrupole magnet with an internal LCW leak (E:QD111) to take measurements.

At 12:38 PM, a Booster kicker (MP02) tripped off on a ground fault indication for the first of two times in 16 minutes. It reset.

Operations reported at 12:46 PM that cryo system experts were beginning maintenance on the TeV sector C4 wet engine. They completed their work an hour later.

At 1:03 PM, Operations reported receiving timeout errors from the TeV “States” front-end (FE) while trying to inject protons into the TeV. An operator contacted a controls expert who rebooted the FE.

At 1:27 PM, Operations, with help from NuMI experts, began sending small amounts of low intensity beam to NuMI. Beam positions upstream of QD111 were good, but downstream they were bad and were causing beam permit trips.

At 2:15 PM, BRF1 tripped off. It reset.

Operations reported at 2:54 PM that there would be no more attempts to send beam to NuMI.

TeV experts began a P1 transfer line matching study around 3 PM.

A MI expert began a coalescing study at 3:19 PM.

Operations reported at 4:03 PM that the Recycler had stashed antiprotons three times during the day shift.

At 4:15 PM, BRF19 tripped off. It reset.

At 4:22 PM, Operations reported finding another LCW leak in the TeV sector C2 rack. TeV experts halted their studies to fix the leak.

Power supply experts reported at 4:52 PM that they had repaired the TeV C2 rack LCW leak.

At 5:55 PM, LRF5 tripped off. It reset.

At 6:06 PM, a Linac Klystron RF station (KRF2) tripped off. It reset.

At 6:10 PM, BRF16 tripped off. It reset.

At 6:34 PM, a Pbar Debuncher beam valve (D:BY728) closed. It opened.

At 6:40 PM, LRF4 tripped off. It reset.

At 6:47 PM, three Pbar Debuncher beam valves (BV309, 728, and 210D) closed. These valves were spread out around the ring. Operations investigated and then opened the valves.

At 9:53 PM, a Booster vacuum pump (B:IPBD2) tripped off. It reset.

At 10:46 PM, KRF7 tripped off for the first of two times in four minutes. It reset.

At 11:36 PM, the MI south anode power supply tripped off on an AC input current monitor. It reset.

Operations reported at midnight that the Recycler had stashed five times during the evening shift.

Tuesday April 1

The midnight shift began with TeV experts conducting studies, with Operations monitoring the stash (237.7E10), with Pbar stacking (33.1E10), with NuMI off due to a QD111 internal LCW leak, and with BNB taking beam.

At 12:34 AM, KRF7 tripped off. It reset.

The TeV studies ended at 1:21 AM. Operations put the TeV through a dry squeeze.

At 2:09 AM, BRF2 tripped off. It reset.

At 5:06 AM, KRF3 tripped off. It reset.

Operations put the TeV into shot setup at 5:32 AM.

At 6:13 AM, Operations reported that the MI beam closure program was not working, but since the protons looked good they continued with the injection into the TeV.

At 6:18 AM, LRF5 tripped off. It reset.

At 6:31 AM, a MI kicker (KPS5S) tripped off on a current limit fault. This did not inhibit the proton injections into the TeV. Operators reset the kicker after the injections were completed.

At 6:33 AM, a Booster vacuum pump (IPRF17) tripped off and caused BRF7 and 8 to trip off. Everything reset okay.

At 6:41 AM, KRF5 tripped off. It reset.

Operations halted stacking at 7:06 AM and began preparing MI for an access to inspect the NuMI magnet QD111. A Pbar expert began a study.

Operations established store 6020 at 7:19 AM with an initial luminosity of 261.73E30.

A Linac expert began a LRF5 study at 7:36 AM.

An RF expert took down the Booster West gallery RF at 7:53 AM so he could look at BRF16.

Operations reported at 8 AM that the Recycler had stashed four times during the midnight shift.

Linac experts began work on LRF1 at 9:06 AM.

A Preaccelerator expert began work on the H- Source 750keV area at 9:13 AM.

The Run Coordinator reported at 10:17 AM that a multi-shift access would begin on Wednesday morning to replace the QD111 magnet.

A Pbar expert reported at 11:29 AM that he had completed his AP-10 kicker study.

Linac experts reported at 11:31 AM that they had completed their Klystron Debuncher water skid pump replacement.

Linac experts reported at 11:50 AM that they had completed replacing the power amplifier tube on LRF1.

MI came out of access at 12:24 PM.

At 12:40 PM, Operations reported that the Linac Debuncher LCW pump had tripped off several times. Experts investigated.

At 1:45 PM, MIRF1 tripped off. It reset.

Linac experts reported at 1:57 PM that they had completed their LRF5 studies.

At 2:12 PM, BRF9 tripped off for the first of four times in about 12 minutes. It reset.

At 3:13 PM, KRF4 tripped off. It reset.

At 4:38 PM, BRF1 and BRF18 tripped off. They reset.

At 5:37 PM, BRF4 tripped off. It reset.

At 5:56 PM, KRF6 tripped off. It reset.

At 6:03 PM, MIRF1 tripped off and would not reset. Operators investigated and suspected that the power supply was bad. Experts will investigate further on Wednesday.

At 6:06 PM, KRF7 tripped off for the first of eight times in about four hours. It reset.

At 6:28 PM, BRF3 tripped off. It reset.

At 6:44 PM, BRF1 tripped off. It reset.

At 7:32 PM, Operations reported that D:BV728 closed. The vacuum looked good. Operators opened the valve.

At 9:39 PM, KRF3 tripped off. It reset.

At 10:11 PM, BRF1 tripped off. It reset.

At 11:24 PM, cryo system experts at CHL reported that the TeV sector A2 wet engine was having some problems. A local inspection revealed a lot of play in one valve and the flywheel was wobbling.

Operations reported at midnight that the Recycler had stashed five times during the evening shift.

Wednesday April 2

The midnight shift began with Operations monitoring store 6020 (59.78E30) and the stash (190.5E10), with Pbar stacking (38.2E10), with NuMI off due to a bad magnet, and with BNB taking beam.

At 12:17 AM, BRF9 tripped off. It reset.

Cryo system experts reported at 12:22 AM that they would monitor the A2 wet engine and if it looked like it was about to fail they would put the house in JT valve mode and call in experts to make repairs.

At 12:42 AM, the Booster RF East Anode Power Supply (EAPS) tripped off. It reset.

At 12:43 AM, KRF7 tripped off. It reset.

At 12:50 AM, LRF2 tripped off. It reset.

At 12:52 AM, BRF9 tripped off. It reset.

At 1:09 AM, BRF1 tripped off. It reset.

At 2:54 AM, KRF5 tripped off. It reset.

At 2:56 AM, D:BV728 closed for the first of three times in 25 minutes. A Pbar Debuncher septum (D:ISEPV) tripped off on two of the valve closings. Everything reset.

Operations terminated store 6020 at 3:49 AM.

Operations put the TeV into shot setup at 4:05 AM.

At 4:28 AM, BRF9 tripped off. It reset.

Cryo experts reported at 4:32 AM that they had tightened the valves and the A2 engine was running okay. The flywheel wobble will get repaired at some later date.

Pbar experts began a study at 4:44 AM.

Operators prepared the Recycler and the Pelletron for access at 5:46 AM.

Operators prepared the Booster for access at 5:46 AM.

Operators prepared the Linac for access at 5:49 AM.

Operations established store 6021 at 6:01 AM with an initial luminosity of 206.28E30.

Operators prepared MI for access at 6:22 AM.

Operations reported that the NuMI magnet replacement work had begun at 7:59 AM.

Operations reported at 8 AM that the Recycler had stashed three times during the midnight shift.

The Plans for Wednesday

The driving force behind today’s access is replacing the NuMI magnet, which is estimated as a 20-30 hour repair. Other major jobs being done will include replacing the MI flying wire, changing the Pbar lithium lens, and Pelletron work. Many other smaller jobs will take place.

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.