Accelerator Update for May 25, 2007 - May 29, 2007

Friday May 25

The day shift began with Operations monitoring the Tevatron (TeV) store 5456 (239.3E30), the Recycler stash (162.E10), and the Antiproton Source (Pbar) stack (16.7E10), with Pbar not stacking due to lithium lens work, with the Linac off due to work on a RF station (LRF3), Booster, and Main Injector (MI) off due to Linac being off, with NuMI and MiniBooNE off, and with D-Zero (D0) and the Collider Detector at Fermilab (CDF) using the colliding beams.

Operations reported at 8:27 AM, that they were dumping the antiprotons in Pbar so technicians could access and repair a quadrupole shunt (QS925).

Operations prepared Booster for an access at 8:42 AM. The access was to repair BRF8.

Operations reported at 9:26 AM that a TeV expert was beginning a Tevatron Electron Lens (TEL1) end-of-store study.

Pbar experts reported at 9:48 AM that QS925 shunt was repaired.

Linac experts reported at 10:24 AM that LRF3 developed a nitrogen leak.

The Booster access to repair BRF8 was completed at 10:25 AM.

A Recycler expert began a 45-minute study at 11:02 AM.

Operations turned on Booster at 11:22 AM.

TeV experts began a 40-minute sextupole study at 11:29 AM.

Linac experts completed their repairs on LRF3 around 12:18 PM.

Operations began sending beam to NuMI and MiniBooNE around 12:37 PM.

Operations terminated store 5456 at 1:36 PM.

Operations put the TeV into shot setup at 1:52 PM.

At 5:21 PM, Operations reported the first of several problems transferring antiprotons from MI to the TeV. Transfers were clipped and lost due to the MI beam aborting. Experts investigated and determined that a problem has developed with running MiniBooNE while trying to load antiprotons into the TeV. The Run Coordinator authorized turning off MiniBooNE during this part of shot setup.

Operations established store 5457 at 6:10 PM with an initial luminosity of 149.42E30.

Operations reported at 6:31 PM that throughout the day they had been having ACNET controls problems, network storms. It just happened again and like the other times it cleared up before an expert could discover the cause. Experts continued to investigate.

Pbar experts reported that they had finished their lithium lens work at 6:36 PM. Experts tuned the lens.

Pbar resumed stacking at 7:53 PM.

At 11:14 PM, the H- Source power supply voltage dropped and would not run back up. Operators swapped out the regulator and oscillator, but it didn’t help. Operations called in an expert.

Saturday May 26

The midnight shift began with Operations monitoring store 5457 (74.71E30) and the stash (34.1E10), with Pbar stacking (8.3E10), and with NuMI and MiniBooNE waiting on Linac beam.

At 12:09 AM, Operations prepared the H- dome for an access.

Operations reported at 2:33 AM that the H- driver power supply was replaced. Operations resumed sending beam to NuMI, MiniBooNE and Pbar. The lithium lens tripped off. Operations contacted a Pbar expert and loaded a NuMI only time line into the Time Line Generator (TLG).

The lithium lens expert gave operations permission to reset the lens at 2:46 AM. Pbar resumed stacking.

Operations reported at 7:59 AM that the Recycler had stashed antiprotons once during the midnight shift.

At 8:22 AM, Operations reported that the MI beam permit had tripped off a couple of times due to bad beam positions on the MI-8 collimators. Everything reset.

At 9:04 AM, a Linac Klystron RF station (KRF5) tripped off. It reset.

At 9:12 AM, LRF3 tripped off. It reset.

MI experts began a study at 11:20 AM.

Operations prepared the Pbar AP-0 vault area for access at 1:06 PM.

Operations reported that they had added a MiniBooNE switch to the “inject antiprotons into the TeV” aggregate that would automatically turn off MiniBooNE to temporarily resolve the conflicts during that part of shot setup.

The Pbar AP-0 work was completed by 1:39 PM.

Pbar resumed stacking at 1:40 PM.

At 2:13 PM, KRF3 tripped off on a coil comparator fault. Operators could not reset the station. They called in an expert.

At 3:11 PM, Operations reported that the Recycler stash was lost. This occurred at the same time that MI aborted due to bad beam positions, but with no beam in the machine. Operations contacted an expert.

A Linac expert brought KRF3 back on line at 4:14 PM.

TeV experts began a sextupole study period at 6:18 PM.

Operations reported at 8:21 PM that the Recycler had stashed three times during the last 12 hours. Unfortunately, the first two stashes were lost. Accelerator status: store 5457 was at 9.56E30, the stack was at 26.5E10, and the stash was at 33.3E10.

Operations terminated store 5457 at 8:40 PM.

A Controls expert reported at 10:49 PM that he discovered the cause of the earlier network storms. He found that several people were running a similar kind of plot at the same time and this increased activity leaked into the VAX system, which had a narrower pipeline and caused key packets of information to be missed. Operations found four of these plots running in the MCR and stopped them. The expert will work on a fix next week.

Sunday May 27

The midnight shift began with TeV experts conducting studies, with Operations monitoring the stash, with Pbar stacking, and with NuMI and MiniBooNE taking beam.

TeV experts reported at 2:24 AM that they had completed their sextupole study.

At 3:28 AM, the Recycler stash was lost again. Operations contacted experts.

Operators reported at 6:08 AM that the stash loss might have been due to MECAR. Investigations continue. Pbar continued to stack and TeV experts continued with their studies.

Operations reported at 7:58 AM that the Recycler had stashed twice during the last 12 hours.

At 8:41 AM, Operations reported a Booster clock repeater problem. Operations called in controls experts.

The latest TeV study ended at 9:11 AM. Operations put the TeV through a dry squeeze.

A TeV expert began another study period at 10 AM.

Controls experts reported at 10:24 AM that they had replaced a repeater power supply.

At 11:44 AM, MI aborted its beam. This occurred at the same time LRF5 tripped off.

At 12:43 PM, the MI power supply interlock loop tripped off. Operations managed to reset everything, but Recycler lost some of its stash due to the trip. Experts thought that this might have been due to a MI-40 power supply problem because nothing else seemed to be at fault and the MI-40 devices don’t have readbacks.

Operations put the TeV into shot setup at 3:14 PM.

Operations established store 5463 at 5:18 PM with an initial luminosity of 88.69E30.

At 8:18 PM, KRF5 tripped off. It reset.

Operations reported at 8:19 PM that the Recycler had stashed three times during the last 12 hours. The first stash was lost.

Monday May 28

The midnight shift began with Operations monitoring store 5463 and the stash, with Pbar stacking, and with NuMI and MiniBooNE taking beam.

At 5:09 AM, KRF5 tripped off. It reset.

At 5:24 AM, BRF11 tripped off. It reset.

At 5:44 AM, BRF1 tripped off. It reset.

At 6:42 AM, KRF6 tripped off twice in two minutes. It reset both times, but after the second reset it was 2 amps lower that its nominal reading. Operators increased its current.

Operations terminated store 5463 at 7:54 AM.

Operations reported at 8:04 AM that the Recycler had stashed four times during the last 12 hours.

The Plans for Monday

The plans for today are to allow CDF and D0 to make an access at 10 AM. The Run Coordinator will allow some work and machine studies during the access. He also wants the MiniBooNE/antiproton injection problem solved.

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.