Accelerator Update for February 6, 2008 - February 8, 2008

Wednesday February 6

The day shift began with Operations monitoring the Tevatron (TeV) store 5887 (61.94E30) and the Recycler stash (229.3E10), with the Antiproton Source stacking (23.1E10), with the Main Injector (MI), Linac, and Booster operating normally, with the Collider Detector at Fermilab (CDF) and D-Zero (D0) using the colliding beams, and with the NuMI and SciBooNE / MiniBooNE (BNB) experiments taking beam.

At 8:32 AM, a Linac Klystron RF station (KRF3) tripped off. It reset.

Operations established beam to Switchyard (SY) at 8:45 AM so the experts could set up beam to the MTest area.

At 8:54 AM, Operations halted the ramp and turned off MI and NuMI, put Pbar into standby, and turned off power supplies for P1, 2, and 3 lines, and for AP1 line. The High Voltage Coordinator reported that a 345kV power line was whipping around in the wind. He wanted to verify that all the lines were okay.

The HV Coordinator reported that everything was okay. Operations began turning MI back on at 10:42 AM.

Operations established beam to MI, Pbar, and NuMI by 10:55 AM.

TeV experts began an end-of-store study around 10:57 AM.

A NuMI expert reported at 10:59 AM that they were still having dehumidifier problems and he wanted an access to make repairs.

After gathering approvals for the NuMI access, operators had the area ready for access by 11:29 AM.

At 12:04 PM, the BNB horn tripped off. It reset.

At 12:05 PM, MIRF1 tripped off. It reset.

At 12:27 PM, BRF9 tripped off. It reset.

Operations reported at 12:55 PM that the TeV end-of-store studies were complete.

At 2:23 PM, KRF2 tripped off. It reset.

At 2:37 PM, KRF4 tripped off. It reset.

At 2:50 PM, BRF9 tripped off for the first if two times in one hour. It reset.

At 2:55 PM, KRF1 tripped off. It reset.

NuMI resumed taking beam at 2:59 PM.

A Pbar expert began a Pbar Debuncher cooling study at 3:20 PM.

At 3:39 PM, KRF2 tripped off on a LCW flow fault.

At 3:59 PM, Operations reported that the 400MeV area beam permit tripped off due to bad beam positions. It reset.

Operations terminated store 5887 at 4:13 PM.

Operations reported at 4:18 PM that the Recycler had stashed antiprotons twice during the day shift.

At 4:23 PM, Operations reported that they wanted to put the TeV into shot setup, but they were waiting for the TeV sector E2 vacuum pump to turn on. It did, but then tripped off. A vacuum expert investigated and reported that the pump had failed.

A MI expert began a 20-minute study at 4:26 PM.

Operations put the TeV into shot setup at 4:35 PM.

At 4:46 PM, BRF9 tripped off for the first of three times in 30 minutes. It reset.

At 6:12 PM, BRF3 tripped off for the first of two times on a power supply overload. It reset.

Operations established store 5888 at 7:16 PM with an initial luminosity of 226.33E30.

At 8:48 PM, BRF9 tripped off. It reset.

At 10:22 PM, Operations reported problems with monitoring the TeV tune. Operators restarted a Front End (DCE04).

At 10:29 PM, Operations reported a sequencer malfunction during a Recycler stash. Operators contacted a controls expert. 30 minutes later she reported that the database was up and running, but it had run out of connections. A JAVA engine (DCE05), which works like a Front End, was trying to update the connection tables unsuccessfully and was using up all the available resources. She reported that the problem was due to this device.

Operations reported at midnight that the Recycler had stashed twice during the evening shift.

Thursday February 7

The midnight shift began with Operations monitoring the store 5888 (118.75E30) and stash (109.5E10), with Pbar stacking (64.5E10), and with NuMI and BNB taking beam.

At 12:05 AM, NuMI reported that they were having trouble gathering data. Operators contacted the expert who had been rebooting Front Ends and she rebooted DCE01. This fixed NuMI’s problem.

At 12:11 AM, a Booster vacuum pump tripped off. It reset.

At 12:22 AM, KRF4 tripped off. It reset.

At 1:02 AM, BRF9 tripped off for the first of two times in about 50 minutes. It reset.

At 2:04 AM, CDF reported that they were having problems with their solenoid and were lowering its current.

At 2:20 AM, KRF7 tripped off. It reset.

At 4:25 AM, Operations reported that a TeV sector F1 vacuum pump tripped off.

At 4:31 AM, LRF1 tripped off. It reset.

At 4:37 AM, the MI vacuum Front End tripped off. Operators rebooted it.

At 6:33 AM, LRF1 tripped off. It reset.

Operations reported at 8:01 AM that the Recycler had stashed five times during the midnight shift.

At 8:22 AM, LRF3 tripped off. It reset.

A controls expert began investigating all our controls issues at 8:23 AM.

SY experts began preparing for a SEM calibration run at 8:29 AM. The run was completed about two hours later.

TeV experts turned off their orbit stabilization program at 10:53 AM.

A MI expert began a collimator study at 11:06 AM.

A vacuum expert turned on the TeV sector F13 vacuum pump at 11:29 AM.

D0 experimenters reported at 11:35 AM that they were performing some luminosity calibration studies.

A SY expert began tuning beam to the MTest area at 11:57 AM.

Operations terminated store 5888 at 12:04 PM.

At 12:05 PM, LRF2 tripped off. It reset.

At 12:10 PM, the TeV F13 vacuum pump tripped off.

Operations reported at 12:21 PM that they were injecting protons into the TeV for studies.

At 12:47 PM, Operations reported that the 400 MeV area Debuncher phase was wandering around. An operator contacted an expert.

A Pbar expert began an AP10 study at 12:58 PM.

A MI expert began a study at 1:03 PM.

At 1:18 PM, BRF5 tripped off. It reset.

At 1:25 PM, KRF2 tripped off on a LCW flow fault. It would not reset. Operators contacted an expert who managed to get the station on and then tune its gain.

The Neutron Therapy Facility reported at 2:41 PM that they had completed treatment on the last patient for the day.

Operations reported at 4 PM that the Recycler had stashed once during the midnight shift.

Operations established store 5891 at 5:46 PM with an initial luminosity of 191.33E30.

At 6:43 PM, BRF9 tripped off. It reset.

A TeV expert informed Operations at 7:24 PM that the TeV had new optics and may not respond normally.

Operators added nitrogen pressure to the LRF3 water skid at 8:07 PM.

At 8:13 PM, LRF5 tripped off. It reset.

At 8:21 PM, KR4 tripped off. It reset.

At 9:29 PM, LRF5 tripped off and would not reset. Operators replaced a waveform generator module. Linac resumed sending beam at 9:40 PM.

At 9:52 PM, a vacuum pump at LRF3 tripped off. It reset.

At 9:57 PM, the MI beam began aborting due to bad beam positions. Operators adjusted the MI tunes.

At 10:14 PM, LRF5 tripped off. It reset.

At 11:01 PM, KRF1 tripped off. It reset.

At 11:19 PM, KRF7 tripped off. It reset.

Operations reported at 11:59 PM that the Recycler had stashed four times during the evening shift.

Friday February 8

The midnight shift began with Operations monitoring the store 5891 (70.49E30) and stash (218.6E10), with Pbar stacking (42.6E10), and with NuMI and BNB taking beam.

At 12:42 PM, KRF1 tripped off. It reset.

At 2:58 AM, Operations reported that the vacuum at TeV sector F1 was degrading. The F1 cryo-system was remaining stable.

At 3:28 AM, BRF9 tripped off. It reset.

At 4:08 AM, BRF6 tripped off. It reset.

At 4:09 AM, LRF4 tripped off. It reset.

At 4:19 AM, Operations received a fire alarm for the Booster. Operators halted Booster beam and prepared the area for access.

At 4:25 AM, KRF4 tripped off. It reset.

The Fermi Fire Department reported at 5:35 AM that they could not find any signs of a fire in Booster. Operators searched and secured the area at 6:15 AM.

Operations established beam to BNB at 6:48 AM.

At 5:52 AM, LRF3 didn’t trip, but its gradient went erratic.

At 7 AM, a Booster vacuum pump (IPRF17) tripped off and caused BRF7 & 8 to trip. Everything reset.

At 7:51 AM, a Booster vacuum pump (IPRF23) tripped off and caused BRF13 & 15 to trip. Everything reset.

Operations reported at 8:03 AM that the Recycler had stashed three times during the midnight shift.

The Plans for Friday and the Weekend

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.