Accelerator Update for July 11, 2008 - July 14, 2008

Friday July 11

The day shift began with Operations monitoring Tevatron (TeV) store 6284 (160.51E30) and the Recycler (RR) stash (226.0E10), with the Antiproton Source stacking (29.9E10), with the Main Injector, Booster, and Linac operating normally, with the Collider Detector at Fermilab (CDF) and DZero (D0) using the colliding beams, and with the SciBooNE / MiniBooNE (BNB) and Meson MTest experiments taking beam.

Operations established beam to NuMI for target scans at 8:33 AM.

A MI expert began a coalescing study at 8:41 AM.

Operations began preparing BNB for an access at 8:43 AM.

At 8:59 AM, Operations reported a TeV leak (T:LEAKS) alarm that cleared after a few minutes. Operations investigated and reported high humidity in TeV sectors D4 and E4. Operations contacted experts.

At 11:03 AM, the TeV suffered a sector A2 quench. Operators investigated.

Operations reported at 11:32 AM that they had switched to using the Pbar Sweeper “A” magnet.

Operations reported at 11: 38 AM that the TeV sector A2 cryo system went unstable a minute before the quench.

At 11:42 AM, Operations reported that MI was down with a possible Low Level RF (LLRF) problem. Experts investigated and reported that changes were made during a coalescing study.

Operations reported at 11:49 AM that the TeV sector A2 vacuum had started going bad around midnight and this was the cause of the unstable cryo system.

A MI expert reported 11:51 AM that the coalescing problem last night was due to the first rotation angle being wrong for paraphasing. This was temporally fixed with beam tuning.

Operations began preparing the TeV for an access at 11:57 AM.

Operations disabled the MTest beam at 12:23 PM for a TeV transfer hall access.

At 1:41 PM, Operations halted the NuMI beam so that FESS personnel could access and work on an exhaust fan. The work took about 80 minutes.

Operations began preparing D0 for access around 12:39 PM.

MI experts began a 90-minute coalescing study at 2:06 PM.

Water experts reported at 3:51 PM that they did not find any LCW leaks in TeV sector D.

Operations reported at 4:10 PM that the Recycler had stashed antiprotons four times during the day shift.

FESS reported at 5:34 PM that the TeV sector C0 pond water pump was on and operating normally.

At 8:08 PM, BRF11 tripped off. It reset.

At 8:40 PM, the P1 transfer line beam permit tripped off. It reset.

Operations reported at 10:07 PM that all work in the TeV had ended and operators had search and secured sector A.

Operations established beam to MTest at 11:01 PM.

Operations reported at 11:56 PM that the Recycler had stashed two times during the evening shift.

Saturday July 12

The midnight shift began with Operations monitoring the stash (396.5E10), with Pbar stacking (99.4E10), and with BNB and MTest taking beam.

Cryo system experts reported at 3:34 AM that the TeV sector A2 cryo system was ready to go.

At 4:34 AM, Operations reported that a TeV power supply for the Tevatron Electron Lens correctors (TEL2) would not come on. Operations contacted experts.

Operations put the TeV through a dry squeeze at 4:55 AM.

At 5:27 AM, the MI and BNB beam permits tripped off. Operators tracked the problem to a bad control card. Operators replaced the card.

Pbar resumed stacking at 5:33 AM.

At 7:51 AM, the Linac 750 KeV area buncher tripped off. It reset.

TeV experts reported at 8:01 AM that they had replaced a chassis for TEL2.

Operations put the TeV through a dry squeeze at 8:30 AM.

At 9:02 AM, Operations reported two Switchyard (SY) trips due to the QXR not ramping correctly. Operators rebooted the QXR system.

Operations put the TeV through a wet squeeze at 9:59 AM.

Operations disabled the MTest beam at 10:21 AM.

NuMI came out of access at 10:24 AM.

Operations put the TeV into shot setup at 10:27 AM.

At 11:14 AM, Operations reported problems with MECAR. It reset.

At 11:19 AM, the TeV sector F23 vacuum pumps tripped off. Operators turned them back on and the vacuum began to recover, but then they tripped off again. Operations called in a vacuum expert.

At 11:58 AM, Operations received a TeV sector E1 sump alarm. Operators began preparing the TeV for an access.

Operations reported at 12:15 PM that operators were beginning a search and secure of MI-65.

At 12:46 PM, a vacuum expert reported that the vacuum was getting worse. He needed an access to replace the pump. Operators began preparing the area for an access.

Operations put MI into an energy conservation mode at 1:47 PM.

At 2:12 PM, the Duty Mechanic reported that the E1 sump pump had failed. He began installing a new pump. This work took about 45 minutes.

At 3:38 PM, Operations reported a possible TeV ground fault. Operations contacted an expert.

Operations established beam to BNB at 3:42 PM.

The TeV expert reported at 4:21 PM that the ground fault was in between TeV sectors F2 and E2. Operations prepared the TeV F-sector for an access to hunt for the fault.

The TeV expert reported at 6:25 PM that the ground fault was in F sector.

Operations reported at 10:23 PM that the ground fault cleared, came back again, and then cleared again. Everyone was out of access.

Operations reestablished beam to BNB at 11:15 PM.

Sunday July 13

The midnight shift began with Operations monitoring the stash and the stack, and with BNB taking beam.

Pbar resumed stacking at 12:17 AM.

Operations put the TeV through a dry squeeze at 12:52 AM.

Operations put the TeV into shot setup at 2:36 AM.

Operations established store 6290 at 4:10 AM with an initial luminosity of 293.97E30.

Operations established beam to MTest at 4:49 AM.

At 5:09 AM, store 6290 aborted on a TeV power supply trip (C:DQ7). An expert investigated and reported that the wall disconnect had failed and this would have to be replaced before he could investigate the problem with the power supply. Operations contacted the Duty Mechanic.

NuMI experts established beam to NuMI at 6 AM.

Operations reported at 7:46 AM that the Recycler had stashed two times during the midnight shift.

Operations reported at 8:18 AM that D0 and CDF were in access.

The TeV power supply expert reported at 8:38 AM that C:DQ7 was back online.

At 8:39 AM, MECAR aborted the MI beam for the first of two times in 14 minutes. It reset.

At 9 AM, Operations reported more problems with MECAR. Operations contacted experts.

At 9:43 AM, Operations reported lowering the MI beam intensity to BNB and NuMI while trying to resolve the MECAR problems.

Operations reported at 10:42 AM that the MI-10 upper power supply was causing the MECAR problems. Operations called in a power supply expert and he reported at 12:27 PM that he had made adjustments.

Operations disabled the MTest beam at 12:28 PM.

Operations put the TeV into shot setup at 12:58 PM.

Operations established store 6292 at 3:12 PM with an initial luminosity of 248.7E30.

LCW experts began work on the MI-30 heat exchanger around 3:51 PM. The MI power supply interlock loop tripped off due to a “water differential pressure” indication.

At 3:59 PM, Operations received a TeV sector E1 sump alarm. Operators investigated and reported that the discharge port was dry.

Operations put Pbar into standby at 4:08 PM.

The MI LCW work ended at 5:13 PM.

At 7:55 PM, Operations reported two NuMI trips on a water temperature (E:BAFT2) indication. Operations contacted an expert.

Operations reported at 8 PM that the Recycler had stashed three times during the last 12 hours.

The NuMI expert gave Operations permission at 8:07 PM to disregard the E:BAFT2 alarm for tonight.

At 8:12 PM, the MTest beam line tripped off due to too much beam, but there was no beam to the area; the SY beam permit was down and its power supplies had been off since noon. Operators investigated and found that a SEM (MIWSEM) had failed due to CAMAC crate problems or due to its current digitizer. Operations called in experts.

At 10:53 PM, Operations reported that a NuMI horn water skid had tripped off on an “expansion tank level low” indication. Operations contacted experts.

An instrumentation expert reported that the MIWSEM was bad and needed to be replaced. Operations prepared M01 for access at 11:40 PM

Monday July 14

The midnight shift began with Operations monitoring store 6292 and the stash, with Pbar stacking, with BNB taking beam, but with MTest off due to a SEM problem and with NuMI off due to a horn water skid problem.

Instrumentation experts reported at 12:51 AM that they had replaced the MW1SEM.

Operations established beam to MTest at 1:05 AM.

Operations established beam to NuMI at 1:17 AM.

At 2:41 AM, CDF reported that they had a problem that which was keeping them from using the store. They needed an access to make repairs.

At 3:12 AM, the NuMI horn tripped off when the recovery tank level went low. Operations contacted experts.

At 7 AM, Operations began preparing NuMI for an access.

Operations terminated store 6292 at 8 AM.

Operations reported at 8:01 AM that the Recycler had stashed eight times during the last 12 hours.

The Plans for Friday and the Weekend

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.