Fermi National Laboratory

Accelerator Update for August 31, 2007 - September 7, 2007


Fermilab Shutdown Report
8/24/07 – 8/31/07


Things Done This Week

  • Repaired LRF3 PA Crowbar Circuit.
  • Troubleshoot LRF1 Cavity Pump Trips
  • Finished Conditioning 7835 in LRF7
  • Started Cleaning the Gallery
  • Troubleshoot I- Dome Power Problem


Electrical Work:

  • Cable pull 100% complete

Water work:

  • LCW water work in the Booster West Gallery
    • Contractor work is complete
      • BWG is back to normal foot traffic. Thank you!
    • Several leaks were revealed during the pressure test. The main focus was to make the header leak free
    • Next week: Move along to make final connections from the header to the system
  • Tunnel water work:
    • 100% filtration system complete (L2, L4, L11, L14, L16, & L23)
  • CUB
    • Booster de-oxy system is expected to be up and running by the end of the day
    • VFD testing is done for now (one pump)
      • Cedric will request another period of testing close to start-up to exercise 2 pump system
  • Corrector installation
    • Number of new correctors in the tunnel: 11
      • L19, L17, L21, L23, L15, L9, L11, L13, L5, L7, & L1
        • L1 was not a standard installation
        • problems mating the wire scanner and corrector
      • All correctors are electrically connected
      • Making plans for the installation of the last corrector: L3
      • Working towards powering correctors in the tunnel
      • Booster vacuum slowly recovering
    • Next week
      • Continue monitoring vacuum
      • Start L3 installation
      • Keep up efforts for energizing the correctors
  • Additional Activities
    • Sink hole
      • Hand digging delayed a week
    • LLRF Room
      • a new ventilation duct system was installed at the LLRF room
    • RF maintenance work is proceeding well

Antiproton Source


  • Leak Check (4)
    • AP3 to be done
    • Finished checking D/A, A40, A30
      • D:ESEP, A30 IP fixed
  • D:ISEP Shielding
  • Ion Chamber removal
  • Deb DCCT motorized stand
    • Ready for alignment
  • D:EKIK ceramic tube
    • New tube on stand; ready for alignment
  • AP2 Water Hose Replacement
  • Leaking LQA
    • Surveyed into place
  • Replaced leaking Deb Sextuple
  • SMA water leak
    • Surveyed into place
  • 4 water leaks to fix
    • IQ8 fixed twice
  • A10 Obstruction Search

Target Station

  • Water Systems (2)
  • Temperature Monitoring (2)
    • 1 wiring done
  • Target Rotation
  • Storage
  • SEM Module
  • LVDTs
  • Sample Retrieval
  • Lens motion
  • Air Exhaust Stack
  • 10mm-2 into transformer
    • Prepping; await tools
  • 10mm-3 into transformer
    • Prepping, installing
  • Lens replacement
  • Pictures of PMAG


  • Valve Stem replace
  • Leak Check (1 of 2)
  • LN2 transfer line maintenance

Cooling Systems

  • Deb 3&4 Trans Notch Filters
  • Deb Double Notch Filters
  • Core 4-8GHz Equalizer
  • TWT balance & trip points
    • Need cryo for 1 week
  • Stacktail Filter 3 cable change
    • Need cryo for 1 day

Electrical (Pbar & EE Support)

  • Shunt Move
    • Progress
  • Shunt Repairs (6)
    • Removed and rebuilding
  • PS Maintenance
    • Started
  • Lens SCRs
  • Regulation Electronics
    • Started by EE
  • Ion Pump maintenance
    • Started; await fans
  • Deb Shunt re-arrangement
  • Deb Extraction kicker load adjustment


  • Interlocks/Safety System (3)
  • Repair cathodic protection
  • Shunt Move
    • Progress
  • Shunt Repairs (6)
    • Removed and rebuilding
  • PS Maintenance
    • Started
  • Lens SCRs
  • Regulation Electronics
    • Started by EE
  • Ion Pump maintenance
    • Started; await fans
  • Deb Shunt re-arrangement
  • Deb Extraction kicker load adjustment

Main Injector

  • 8 GeV line:
    • Replaced 3 vacuum valve solenoids
    • Installed multiwire
    • Leak checked
  • LEP Corrector coil replacement:
    • Complete
  • MI40 Lambertson beampipe replacement:
    • Making vacuum parts, waiting on delivery of cross pieces for ion pumps.
  • Leak checking:
    • Complete at 527 and 614
  • Wet magnets from C0:
    • 5 magnets Shipped to IB2 for drying
  • MI Status


  • Finished or underway
    • New DCCT section installed, leak checked and under vacuum since Wednesday last week; pumping down
      • DCCT section has been aligned
      • This section will be baked
    • V303 moved
    • H308 moved
    • Need to check ALL LEP dipole positioning respect to beampipe
    • STC tank 7 leak check finished
      • Second attempt has been made; detected air but found no external leak
    • Trombone additional quadrupoles installation issues
      • HLRF cooperation
      • Installation has been completed
    • Added support to old DCCT
    • Bulk supply upgrade (on-going)
    • DCCT cable connected
    • Stochastic cooling: Photodiodes aligned
    • Leak check for the 800 transfer line: two leaks have been found so far
    • Measured rough positions of trombone elements with respect to surveyed elements
  • Rough measurements of trim quadrupoles and LEP dipole in trombone section

  • Pelletron Status
    • Closed on Aug. 31 st
      • All maintenance finished except water
        • Replace filter and take water sample later on
    • Made access on Sept. 4 th to retrieve blue man-lift
    • U-bend mode running on Aug. 31 st
      • Very little gas activity (i.e. conditioning)
        • Although lost current was high at first
          • Appears to be back to nominal on Tuesday (Sept. 4)
    • Charging circuitry back to nominal (7.3 mA/kV)
    • Very good electric strength
      • Achieved 5 MV with only two vacuum bursts
    • Regulation loop parameters needed to be adjusted (new needles)
      • Will have to resave nominal file (for the full line) at start up
    • Got pulsing beam (full amplitude) to the collector with minimal tuning of a few correctors
    • Data taking with OTR started on August 31 st, in the afternoon
      • Camera timing was verified and the first pictures were acquired
    • Ran a good portion of the day on September 4 th
      • More to come
    • Issues:
      • We missed (during check out) what appears to be a broken resistor on the gun resistor chain
        • Only impact should be a lower resolution for BIASI
        • We won’t open the tank to fix it
          • Wait for a natural occurrence
      • One non-working BPM channel
        • Will be worked on today (September 5 th)
      • E50 is not working
    • Continue taking data with OTR
      • Intermittent data taking until we can go in the tunnel for YAG measurements
        • Primary goal (Arden): Polish data acquisition software and timing of the camera in preparation for YAG measurements
            • Secondary goal (Arden, Sasha, Lionel): Optics measurements, pepper-pot measurements (i.e. understand the drop of intensity seen on pepper-pot images)
    • Maybes (Lionel, Sasha): Check gun performance and calibration of the Gun CT
  • On the list
    • Magnet moves and survey /alignment pending calculations
    • Elevation survey after collimator installation
    • Electrical connection for new quadrupoles
    • Field check for new quadrupoles
    • LEP dipole field centering on beam pipes
    • Vacuum maintenance
    • Kicker repair; timing verification
    • TOR330 repair
    • BPM calibration
    • Diagnostics maintenance
    • Laser link vacuum maintenance


Tevatron status as of September 7


Continuing with maintenance work


  • Drain cleaning complete in all underground areas
    • Lower carrier, pre-target, support rooms, shaft, target hall, decay pipe walkway, absorber hall, muon alcoves, and absorber passageway
  • Installation Multi-Wire in Pre-target in process
    • Survey in process
  • Horn 2 water leak inspection
    • Remove six R and all T Blocks
    • Drain, remove, and cap Horn 2 water lines. Removed strip line
    • Pressurize water system and introduce smoke into suction line
    • verified a leak on Horn 2 using cameras with Horn in Target Pile
    • Move Horn 2 to work cell & complete Survey
    • Introduce smoke again and inspect Horn in Work Cell to localize leak
    • Develop repair and Safety plans
  • Develop plan to fill decay pipe with helium in place of vacuum
    • Held planning meeting on Sept. 6th
    • Identify engineer to lead design & procurement of helium filling system including development documentation & procedures
    • Complete ODH analysis of underground areas
    • Complete safety analysis evaluating possible water & air impacts
    • Develop NuMI Note on changes to system
    • Develop Addendum to shielding assessment and update SAD
  • Target Pile Dehumidification Installation Project (35%)
    • Continue installation of piping & electrical conduit upper labyrinth
    • Continued installation of chilled water pipe & electrical conduit in T-Hall
    • Install insulation on chilled water piping in Target Hall
    • Planning for electrical tie-ins during Sept.13 th power outage
    • Finalized location of dehumidifier deck off east side of work cell
    • Start piping layout in service building & mezzanine


No work was completed this last week.

  • Reverse Horn Polarity
    • Tale wipes
    • Reverse polarity of horn power supply
    • Test horn
  • Check Corrosion Tabs at 25m Absorber (27/28Aug07)
  • Change water DI Bottle (50%)
  • Dry out horn (09Aug07)
  • Check Leakage Current on Ion Pumps
  • Power Supply Maintenance
  • Toroid Work

MTA Beamline Installation

  • Shielding underneath hatch area is complete
    • Restacking complete on 09/04
    • Finalizing some safety requirements:
      • Added cover plates to aid S&S
      • Retrofit cover plate over the utility penetration
      • Re-used the slug of concrete to plug in the beam pipe penetration
  • Electrical Work
    • Cable pull ongoing
      • Load cables estimate to finish by Monday
      • Immediately start pulling instrumentation cables
    • Next week: Continue with cable pull
  • Mechanical Installation
    • All magnets have been stationed in place
      • Next step will be alignment!
    • Moving along with MW stands (45% complete).
  • Alignment
    • Outside corner of the concrete blocks were shot by surveyors
    • Next week: Rough alignment starts Monday, Sep 10 th
  • LCW work
    • Copper pipe installation complete (Linac side)
      • Brazing/installing taps
    • Next week: Contractors will be moving to the MTA side
  • Vacuum work
    • Making as much progress as possible with the assortment of the spool pieces and flanges
    • Ion pumps stands are complete
    • Beam pipe support stands are complete

Collider Detector at Fermilab

  • Silicon cooling leak repair
    • East fixed, may revisit if there’s time
    • Visual inspection shows some difference from the east side
      • Oddly the leak rate is worse, but this side was still functioning
      • One leak site is obvious
    • Expect work to continue through at least next week
  • Power outage at B0/CDF trailers
    • Recover started Wed.
    • Still some things to sort out
    • Upgraded firmware in controls modules, main iFix servers ®XP during outage
      • Contributor to the duration of the recover


Work continues


Completed Jobs

  • Recycler Task Manager: Oshinowo
    • 5527 Optically align Recycler DCCT after installation. Date: 8/13/2007 Complete
  • Tevatron Task Manager: Bocean
    • 5494 As found low beta quads and separators at B0 Date: 8/06/2007- 8/08/2007 Complete
    • 5495 As found low beta quads and separators at D0. Date: 8/09/2007- 8/14/2007 Complete
    • 5498 Measure rolls on 111 dipoles and quads in Tevatron (48) Complete
    • 5501 Un roll D16-1 quad when D-1 is warmed.(As Found) Date: 8/13/2007 Complete
    • 5591 As Founds of the Roman pots at D0 4 of 6 Date: 8/13/2007 85% Complete
    • 5598 A0 dump as found (no request for this job) Date: 8/23/2007 Complete
    • 5594 As found Schottkies, Separators and Beam pipe at A17 s Date: 8/24/2007 Complete
    • 5572 As found and re survey kicker replacement at F-17. Date: 8/28/2007 Complete
    • 5506 As found crystal collimator at E0. Date: 8/28/2007 Complete
    • 5506 level run through Tevatron tie rods. Date: 8/29/2007 Complete
  • Booster Task Manager: Oshinowo
    • 5534 Align 24 Booster Corrector Magnets.( Long 19 Long 17 ) Date: 8/13/2007 Complete
    • 5534 Align 24 Booster Corrector Magnets ( Long 21 Long 23 ) Date: 8/14/2007 Complete
    • 5534 Align 24 Booster Corrector Magnets ( Long13 Long15 ) Date: 8/20/2007 Complete
    • 5534 Align 24 Booster Corrector Magnets ( Long 11 Long 9 ) Date: 8/21/2007 Complete
    • 5534 Align 24 Booster Corrector Magnets ( Long 7 Long 5 ) Date: 8/23/2007 Complete
    • 5534 Align 24 Booster Corrector Magnets ( Long 1 ) Date: 8/30/2007 Complete
  • Main Injector Task Manager: Oshinowo
    • 5563 AS found survey of Quadrupoles at 400, 402, 404 and the three Lambertson. Date: 8/13/2007- 8/15/2007 Complete
    • 5554 307, 308 Collimators: Base plate survey Date: 8/16/2007 Complete
    • 5554 307, 308 Collimators: Set travel and limit switches Date: 8/22/2007 Complete
    • 5596 Magnet 614 had to be moved please realign. Date: 8/23/2007 Complete
    • 5562 The beam pipe on the downstream side of the MI40 Lambertson. Date 8/29/2007 Complete
  • CMS Task Manager: Oshinowo Field Crew: O’Boyle.
    • 5516 Survey the CMS F-pixel Half cylinder +Z2 in the Lab C Date: 8/09/2007 Complete
    • 5518 Survey the CMS F-pixel Half cylinder +Z2 in the Lab C Date: 9/06/2007 Complete
  • D0 Task Manager: Oshinowo
    • Stickmike survey of Muon system prior to opening of detector. This will include both CFs and EFs. Date: 8/06/2007 Complete
    • 5524 VStar or Laser Tracker survey of North/South EF pedestals with respect to the sidewalk monuments. Date: 8/07/2007 Complete
    • 5592 Elevation run in D0 before machine is opened. Date: 8/06/2007 Complete
  • P150 Task Manager: Oshinowo Late Request
    • 5593 We had to move several magnets and multiwires to make up the ends on the cans for vacuum connections. They are as follows: The multiwire cans at 708 and 710. B2 magnets at the interface of 710 - 711 and the interfaces of 709 - 710, the serial #'s are BDM 883-1 BDM 835-1 BDM 879-1 and BDM 397-1. Date: 9/07/2007 Complete
  • Linac Task Manager: Oshinowo
    • 5468 Stake out the MuCool beamline component locations in the Linac and MuCool Test Area (MTA). Date: 8/08/2007 Complete
  • P-Bar Task Manager: Oshinowo
    • 5488 As Found A4Q10 (LQA in Accumulator) has a water leak. Date: 8/09/2007 Complete
    • 5595 As Found An Accumulator multipole magnet has developed an leak. Date: 8/30/2007 Complete
    • 5488 Alignment A4Q10 (LQA in Accumulator) has a water leak. Date: 8/30/2007 Complete
    • 5595 Alignment An Accumulator multipole magnet has developed an leak. Date: 9/04/2007 Complete
  • Nova Task Manager: Oshinowo
    • 5475- Stake out new Recycler injection line (MI10) Date: 8/27/2007 Complete
  • MTA Task Manager: Oshinowo
    • 5469 Vertical check of the MuCool beam line components. Date: 8/29/2007 Complete
    • Shoot MTA shielding blocks after re-stacking. Date: 9/04/2007 Complete

Next week’s schedule of work

  • Tevatron Task Manager: Bocean
    • 5499 Measure and re align 29 dipoles and quads that have new stands. Date: 9/04/2007- 9/07/2007
    • As found TEL1 in transfer hall. Date: 9/10/2007
    • As found TEL2 in transfer hall. Date: 9/11/2007
  • Elevation Run Floor points
    • Manager: Kyle
    • A-F sector s Date: 9/4/2007- 9/07/2007
  • Booster Task Manager: Oshinowo
    • 5534 Align 24 Booster Corrector Magnets Date: 9/04/2007
  • Main Injector Task Manager: Oshinowo
    • 5554 4 collimators in the Main Injector at the 301, 303, 307, and 308 locations. Date: 9/04/2007-9/07/2007
  • Nova Task Manager: Oshinowo
    • 5477-Stake out new Recycler RR30 quad positions for the Nova ProjectDate: 9/12/2007
    • 5476-Stake out new Recycler extraction line (MI30) for the Nova Project Accelerator upgrade. Date: 9/13/2007
  • P-Bar Task Manager: Oshinowo
    • 5541 A new three-piece ceramic tube for the Debuncher Extraction Kicker (located between D6Q10 and D6Q9) will be installed. Want to assure the maximum alignment aperture is retained from production in Technical Division IB2 to installation on the stand (may have to take into account deflection of stand with/without kicker modules. Date: 9/10/2007
    • 5540 The Debuncher DCCT (located between D1Q2 and D1Q3). Date: 9/12/2007
  • MTA Task Manager: Oshinowo
    • 5469 Rough alignment of the MuCool beam line components. Date: 9/10/2007 - 9/14/2007
  • CMS Task Manager: Oshinowo
  • CDF Task Manager: Greenwood
    • 5450 Measure elevation of CDF central detector Elevation Run. Date: 9/25/2007
  • ILC Task Manager: Bocean Not Scheduled

Alignment and Metrology Home page


Work in the tunnels continues.

Cryogenic system

Tevatron at LN2 temperature except A1, C4, D1, and E1 at room temperature

  • C4 and D1 cooldown to LN2 temperature to commence later today.
  • Pbar Debuncher Stochastic tanks and S/C Notch Filter Dewar are being cooled down to LHe temperature.
  • Tunnel magnet cryostat vacuums at A25 and D35 regions are of concern.
  • CDF Cryo online with TeV Cryo Compr discharge/suction headers due planned power outages and electrical work on water pump.
  • 3 FRIG Mycom compressors (plus DZero) running with helium precooled to LN2 temperature flowing through magnet strings

Cryo Operations

  • 3 FRIG Mycom compressors (plus DZero & CDF Cryo) running with helium precooled to LN2 temperature flowing through magnet strings
  • CHL helium inventory management system operating (Tank Farm, Helium Purifier, A0 GHe Recovery) and Tank Farm has been purified; the helium losses are normal

FRIG Maintenance

  • 10 of 13 wet expander flywheels have been replaced
  • 18 of 26 wet/dry expander overhauls of cold ends are completed (A1 thru D1, plus Pbar)
  • Various vacuum, instrumentation, and controls maintenance continuing

CHL Maintenance

  • CHL helium inventory management system operating on clean Helium Tank Farm, though A0 GHe Recovery being vented due to CHL Main Helium Purifier CHL helium inventory management system operating on clean Helium Tank Farm, though A0 GHe Recovery being vented due to CHL Main Helium Purifier being offline while Screw Compressors on Mobile Helium Purifier
  • Coldbox-1:
    • Charcoal Adsorber west unit regeneration at high temperature being started.
  • Nitrogen Reliquefier plant maintenance in progress:
    • R22 precooler skid compressor being inspected.
    • TSI Compressor 3000 hp motor replaced but bearing seal problems being addressed.
  • Various compressor and facility maintenance in progress

Current Operations

  • CHL Nitrogen Reliquefier plant off and LN2 supply dependent on vendor deliveries to CHL LN2 Dewar tanks; deliveries are fine


This Week

  • Linux File Server Upgrade – Complete
  • Fileshare process port to Linux - Complete
  • BSSB version 8.8 Update - Complete
  • MI-30 CAMAC Link Reconfiguration – Complete
  • MI-30 Collimator Controls Installation – In Progress
  • MADC II Reset Updates – In Progress
  • Routine maintenance

Next Week

  • Console Node OS Upgrades Continue
    • Individual console nodes unavailable ~1 hr.
  • Central System Processes port to Linux as ready

Other Upcoming Disruptions

  • 9/18/07
    • Database Disk Array Firmware Patch Installations
  • 9/19/07
    • Unix Webserver Maintenance
  • 9/20/07
    • Save/Restore and SDA Database Updates
  • 9/27/07
    • Application and Device Database Updates


Casey’s Pond

  • OPERATING 3 – 100HP PUMPS & C-4.
  • FLOWS/PRESSURE STABLE [flow thru 14 inch strainer].
  • WILL CONTINUE INTO THRU TODAY. EXPECT TO TRANSFER BACK TO THE 400HP PUMPS THIS PM(C-4 Power Outage on 9/8/07 – will continue to run on C-4 ICW pumps except for ~1 hr. transfer switch maintenance)
  • ICW FLOW METER WIP. Expected completion/transfer back to the 16 inch strainer by/before 9/28/07


  • ALL of the19 poles are installed with conductors attached.
  • Project completion is ahead of schedule at ~90% complete

TeV pond silt


Sink Holes


Other Jobs

  • FEEDER 37 PROJECT – PS-2 & K-TEV [2 Hr] POWER OUTAGE SHEDULED FOR MON 9/10 @ 8:00 – 10:00 AM [to switch cables – temporary feed]
  • During the first week in October, FESS will conduct an ICW flush


  • Safety System Status
    • Upgrade M02 - M05 Enclosure Interlocks 40%
    • Upgrade M01 Enclosure Interlocks 0%
    • Upgrade Main Injector PLC Code and replace PLC Batteries 35%
    • Replace Safety System DEC talk with ATT Natural Voices 30%
    • Linux kernal 2.4 to 2.6 upgrade to Safety Data Acquisition System 25%
    • Upgrade Meson Area Oxygen Monitors 0%
    • Replace Pbar Vault Key Tree Door Latches & add status LEDs 0%
    • Route Key Tree release signals from MCR to Meson 100%
    • Install Radiation Monitor Cables for the Muon Test Area 100%
    • Replace oxygen monitor cells in the Tevatron 100%
  • A successful shutdown is getting our work done safely

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.

last modified 09/07/2007    email Fermilab