Accelerator Update for April 7, 2008 - April 9, 2008

Monday April 7

The day shift began with Operations monitoring the Tevatron (TeV) store 6027 (71.41E30) and the Recycler stash (255.6E10), with the Antiproton Source (Pbar) stacking (45.9E10), with the Recycler, the Main Injector, the Booster and Linac operating normally, with the Collider Detector at Fermilab (CDF) and DZero (D0) using the colliding beams, and with the NuMI and SciBooNE / MiniBooNE (BNB) experiments taking beam.

An RF expert made adjustments to a MI RF station (MIRF6) at 8:21 AM that allowed MI to run 10 turns.

Operations prepared MI-31 for access at 8:26 AM to investigate the Pelletron temperature problems.

At 9:05 AM, Operations reported that the BLM errors, which were causing the TeV sequencer to crash, were fixed by a multibus reboot that synced the BLMs and BPMs.

At 9:40 AM, the MiniBooNE experiment reported that their horn status display was down. Operations contacted an expert.

At 9:43 AM, a Linac Klystron RF station (KRF3) tripped off. It reset.

MI-31 came out of access at 10:05 AM.

MI experts requested at 10:58 AM that Operations increase the intensity. It didn’t work. MIRF6 began tripping off. RF experts made adjustments to the station. Operations set MI to 7 turns for NuMI.

Operations terminated store 6027 at 1:39 PM.

Operations put the TeV into shot setup at 1:50 PM.

At 2:03 PM, KRF6 tripped off. It reset.

A MI expert began a coalescing check at 2:13 PM.

A Recycler expert began a study at 3:57 PM.

Operations established store 6028 at 3:58 PM with an initial luminosity of 268.27E30.

Operations reported at 4 PM that the Recycler had stashed antiprotons twice during the day shift.

Operations reported at 4:32 PM that two experts accessed the Meson M04 enclosure to replace a bad oxygen monitor.

The Recycler study ended at 4:46 PM.

At 5:41 PM, LRF4 tripped off and would not reset. An expert investigated and will make repairs.

Operations put Pbar into standby at 5:51 PM.

Operations loaded an energy conservation timeline into the Time Line Generator (TLG) at 6:09 PM.

Linac experts found and replaced a bad relay for LRF4 at 10:01 PM.

Linac resumed sending beam at 10:07 PM.

At 10:13 PM, MIRF13 tripped off and caused the NuMI beam permit to also trip.

At 10:14 PM, MIRF12 and 13 tripped off. They reset.

At 10:40 PM, a Booster kicker (MP02) tripped off with a ground fault indication and would not reset. Operations called in an expert.

At 10:46 PM, LRF2 tripped off. It reset.

Operations had Pbar back in standby and had loaded an energy conservation timeline by 10:56 PM.

Operations reported at midnight that the Recycler had stashed once during the evening shift.

Tuesday April 8

The midnight shift began with Operations monitoring store 6028 (103.15E30) and the stash (72.0E10), with Pbar not stacking (17.2E10), and with NuMI and BNB off due to problems with MP02.

A Booster expert reported at 12:12 AM that Operations could run Booster beam.

At 12:28 AM, KRF2 tripped off. It reset.

At 12:30 AM, MP02 tripped off. Experts investigated.

At 12:51 AM, a TeV vacuum pump (T:E0IP10) tripped off. Operations will turn it back on after the store ends.

Booster resumed sending beam at 1:29 AM.

At 1:46 AM, MP02 tripped off. Operations put Pbar into standby.

Operations reported at 2:40 AM that when operators attempted to resume stacking, the Pbar lithium lens tripped off three times before it reset and ran normally.

At 2:43 AM, Operations reported that the Recycler VSA scope was still not working properly. Operators rebooted the scope.

At 3:54 AM, LRF3 tripped off. It reset.

At 3:57 AM, LRF4 tripped off. It reset.

At 4:47 AM, Operations reported that the Recycler VSA was again reporting bad information. Operators rebooted the scope.

At 5:26 AM, MP02 tripped off. A Booster vacuum pump (B:IPMP02) also tripped off.

At 6:41 AM, the Recycler VSA went bad again. Operators rebooted the scope.

At 7:02 AM, Operations reported that they had turned off Booster, NuMI, and BNB for a Booster access.

Operations reported at 7:59 AM that the Recycler had stashed antiprotons four times during the midnight shift.

At 8:38 AM, Operations reported that they had tried to stash antiprotons into the Recycler, but that most of them had been lost. Operations investigated and discovered that they had conducted an orbit smooth at the wrong time and that had caused the antiprotons to hit the collimators. The bad changes were removed.

Booster experts reported at 9:46 AM that there was a problem with BRF5. Operations reduced intensity while they were investigating.

Operations began preparing NuMI for an access at 9:59 AM.

A MI expert reported at 11:37 AM that he would be conducting a transverse damper study during the next Recycler stash.

At 11:58 AM, LRF2 tripped off. An expert investigated.

At 1:04 PM, KRF7 tripped off. It reset.

At 2:50 PM, LRF2 tripped off. It reset.

Around 3:32 PM, a Booster vacuum pump (IP720) tripped off. It reset.

At 3:57 PM, Operations reported that Booster would not deliver beam to NuMI or BNB in preparation for a Booster access, but stacking was still allowed.

Operations reported at 4 PM that the Recycler had stashed six times during the day shift.

Operations reported at 4:22 PM that Booster experts were preparing to bakeout the spare MP02 kicker. Operations will monitor the bake.

MI experts began a MI RF study at 4:54 PM.

Around 8:50 PM, Operations reported that multiple lightning strikes had caused the TeV to quench at sectors B1 and D1. Fortunately, store 6028 was lost due to an abort before the TeV quenched. Operators investigated.

At 9:07 PM, cryo system technicians at CHL reported that they had lost communication with the TeV sector A2 cryo system. Operators investigated.

Operations reported at 9:13 PM that the cause of the TeV abort might have been due to a lightning induced interlock glitch. Operations called in interlock experts.

At 9:25 PM, Operators reported that the TeV sector A2 wet engine was down and the A2 thermometry crate was off and would not reset.

At 9:30 PM, Operations reported that a power supply expert would come in to replace the UPS at TeV sector A1.

At 9:33 PM, KRF7 tripped off. It reset.

Operations put MI into an energy conservation mode at 9:40 PM.

Cryo system techs reported at 9:41 PM that they were beginning the TeV sectors B1 and D1 cold compressor pump-downs.

At 10:09 PM, KRF7 tripped off. It reset.

Operations prepared D0 and CDF for an access at 10:56 PM.

Interlock experts reported at 10:59 PM that they would need to access CDF and the Transfer Hall to replace bad electronics.

Operators reported that they had completed searching and securing the TeV A sector at 11:40 PM.

Cryo system experts reported at 11:57 PM that they had replaced the thermometry crate power supply and CPU board at the TeV sector A2 cryo system building.

Operations reported at midnight that the Recycler had stashed four times during the evening shift.

Wednesday April 9

The midnight shift began with the TeV recovering from a quench, with Operations monitoring the stash (403.0E10) and the stack (32.9E10), with D0 in access, and with NuMI and BNB off.

At 12:02 AM, Operations reported that the Recycler had lost about 10E10 of antiprotons due to bad VSA information, which caused the antiprotons to get over cooled. Operations contacted experts.

Interlock experts reported at 12:14 AM that they had completed their TeV safety system repairs.

At 1:24 AM, LRF4 tripped off for the first of three times in 10 minutes. It reset.

At 3:49 AM, Operations reported that the lightning strikes had tripped off the TeV sectors A2 and A3 bulk power supplies and they would not reset.

D0 came out of access at 5:01 AM.

Operations reported at 5:33 AM that they had successfully tested and turned on the TeV low beta magnet power supplies.

Power supply experts began investigating the A2 and 3 bulk power supplies at 5:34 AM.

At 5:51 AM, BRF12 tripped off. It reset.

Operations started a TeV QBS ramp at 6:08 AM.

At 7:36 AM, Operations reported that the Pbar Debuncher RF stations 1-6 voltages went down late last night.

At 7:50 AM, Operations reported that there was a problem with the electrical safety system at sector A2. Operators turned off the power supplies.

Operations reported at 8 AM that cryo system experts were beginning work on the wet engines at TeV sectors C2 and 3.

The Plans for Wednesday

The Run Coordinator wants to put a new store in later today. That is the only time the Booster will be used. The Booster access to change out the MP02 magnet will begin on Thursday (4/10/08) morning. On Tuesday (4/8/08), Pbar set a new stacking record with a rate of 25.9mA/hr.

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.