Tevatron - Sept. 30
These are some recollections by operators, or people who were operators, about the Tevatron. They show how close people can become to a machine, its installation, operation, innovation and obstacles. It's a love-hate relationship. Operators remember their mistakes and successes, and the challenges they faced to get the Tevatron up and keep it running. Today the machine is turning off and so the challenges go with it into history. The Tevatron will be missed.
1983: Marvin Olson
There was little need for operators during the Tevatron installation so we all got farmed out to various groups as additional manpower. I was assigned to Bob Brooker in EE Support to help with the Quench Bypass Switches. I helped with the assembly testing and installation of the switches. I was also assigned the job of assembling the QBS Controller Boxes. After the entire ring was installed and I was back on shift work, I was proud when I observed my first Tevatron quench and the switches performed perfectly saving the magnets from destruction.
Mid 1980s: Bruce Hanna
In the mid 1980s , Tevatron dipole magnets were blowing up weekly due to a lead problem at the ends of the Top Coil dipoles. We had no way of telling whether or not the leads were tied. After one failure , I went over to the Industrial area and was looking at the latest fatality with Brian Smith. He was using a bore scope to show me some of the internal damage. I asked if we could use this from the outside, going in through the Kautzky port. We tried , but found the scope would not get through. We found another flexible bore scope with a smaller diameter , but it was too flimsy and wouldn't get around all the nooks and crannies. I called up the manufacturer, Olympus, and explained our problem. The representative said he might have something. The bore scope he brought was a 7mm diameter scope and we were able to get into the magnet and inspect the leads all the way into the cryostat. We called Helen Edwards over and had her look at this. She saw what we were able to do and quietly told us that this bore scope was NOT TO LEAVE THIS SITE! When I asked the rep about purchasing this scope he said that there was a small problem; only 100 of these scopes were made and the US AIR Force bought 99 of them; this was the only one left and it was a demo. We persisted and finally gave him a short order form for $10,000.00. The bore scope was a great success. About a week after buying the bore scope I got a call from Carolyn Hines. To say that she was upset with me was an understatement. She was furious about this $10,000.00 short order form. When I told her that Helen told me to buy the scope, the line got quiet. After a moment she told me to never do that again and hung up.
3:10 AM, October 13, 1985: Jerry Annala
I guess my most defining memory of the Tevatron was the first collision detected back in 1985. I remember people working the low level RF systems that were spread out on tables. Operators and System Experts were at F0 and AP10 and on the phone with the MCR whenever we tried to do a shot. I remember stacking forever to get 1 mA in the Accumulator and then only getting a fraction of that into the Tevatron. It seemed like such a challenge to get any antiprotons to Low Beta. When CDF brought over the picture of the first collision, I thought about what a great accomplishment it was for everyone to have made that happen and what a huge milestone that was.
October 13, 1985: Dave Capista
I was around on some of the first collider shots, if not the very first collider shot. What I remember are the hours of checklists and typing in parameters by hand and checking and rechecking to the point that we all had a headache by the end of the shift. It is amazing how far we have come with automation and how smoothly and efficiently shots go today.
Early 1990s: Wally Kissel
I remember the long hours in the tunnel taking pictures of the insides of the Tevatron magnets. During the era of Tevatron Fixed Target operations, the Tevatron magnets were ramped from low energy to high energy approximately once per minute. Weakness in some of the conductor insulating and bundling materials allowed some conductor movement during this cyclic operation. Not good. What to do? Inspect! How? Well here comes the part in which I was involved. Once the hundreds of magnets had been warmed to room temperature and the cryogenic connections had been removed, it was then possible to look into these areas through a cryogenics pipe approximately 2 inches in diameter. It was possible to snake through a bore scope fiber optic image device to peek into the bowels of the magnet. Yes, analogous to proctologist’s instruments of the time. Unfortunately the pathway was obstructed. There was a one way, aluminum flapper valve in the way. Of course the one way it operated was not the way to allow entrance! It was only few inches from the opening but also around a corner, maybe two corners. It had to be pulled open to allow the bore scope in. OK, so how does one open this door? It was aluminum so a magnet wouldn’t work. Use any sticky substance would be bad. This problem really sucked! Hey wait! That’s it! Duane Plant came up with a clever little device using a spring-loaded de-soldering tool, an appropriate length of small diameter flexible tubing, and the business end of a toy suction dart from Toys R Us. Oh what fun it was to manipulate the bore scope into a position to see the flapper valve and then move the suction device into the necessary alignment. Proper contact rarely occurred on the first try or even the second. Fortunately there were two people involved in this adventure. As one person would be getting cramps coercing the bore scope and sucker tube into position, the other would be prepared to trigger the de-soldering tool. And then re-arm the tool and trigger again. After a while, this process was better understood and bursts of “Woo Hoo!” could be heard frequently through the tunnel. Once inside the opening, it was necessary to maneuver the scope to the right places and take pictures while trying to figure out which way is up? And was this the first corner or the second? I believe those hours playing video games like Doom, Quake, Duke Nukem, and the like, gave some of us an advantage. Operators took the pictures and others did the repairs. The success of this and many other Tevatron tasks were due to on the spot innovation and dedicated work.
July 17 & 18, 1996: Bob Mau
I remember the Flood of 1996. It took me an extra hour just to get into work that day. I was diverted from my regular route and ended up at strange flooded intersections where I would have to wait to see if an SUV would try to get through and then gauge the depth of the water to see if I could get through. When I finally got to Fermilab, I found waterfalls flowing into the Tevatron; water was flowing into the Tevatron from everywhere. My biggest worries were if sump pumps started to fail. You have to understand that the Tevatron magnets were only three inches off of the floor. Also, lakes were starting to surround the power. If they tripped off there would be no power to the sumps. A bunch of people from Roads and Grounds personnel, FESS, Mechanical Support, and Operations all worked filling and placing sandbags. They worked hard and stayed late, even those who had flooding problems at home. The loyalty and hard work of these people saved the Tevatron.
4:30 AM, June 25, 2002: Max Monningh
We began that owl shift in the middle of a painful Tevatron recovery, with lots of hardware replacements and compounded by mystery injection failures into the P1 transfer line. Finally, at 4:38 AM, we were ready to put beam in the Tevatron. I was the Tevatron operator, and I was eager to get a store into the machine. "OK, I got it!" I said. In those days the Tevatron sequencer was much less friendly and there was a tendency to try to do the sequencer's tasks by hand, which was quicker and more convenient. As long as you did it right. I prepared the Tevatron for injection, which required enabling and disabling several timers, a simple enough task. Except that I was in a rush to get the machine back on and get that store in and self-assured enough to think I knew what I was doing and it was 4:38 AM on the last day of a midnight rotation. So , I forgot to enable a timer that would ensure the Tevatron was empty of beam when we injected fresh beam. When we injected the first blob of beam everything was great. A special kicker magnet kicked the beam onto a circular orbit and around it went. Then, the next blob came along and it too was injected. Unfortunately , the original circulating beam was still there and it got another kick, which pushed the old beam straight into the side of a magnet called F11U, quenching the Tevatron. So what I learned from that experience was to take my time, and to be very , very , cautious of ever letting my guard down. The Tevatron does not forgive mistakes and it punishes those who have the temerity to think they know what they're doing. The Tevatron taught me humility and gave me paranoia.
4:30 PM, November 3, 2002: Todd Johnson
Of course I remember very well the night Duane Plant called me at home and told me that he had been reading the Main Control Room Electronic Log Book and saw that we had quenched the Tevatron due to losses, but the cause of the losses was not obvious. At the same time he was talking with me on the phone , a news reporter on his TV said something about a magnitude 9 earthquake that had just happened in Alaska. He and I checked the earthquake page on the USGS site. I managed to get a plot of the low beta magnet tilt-meters to see if the quadrupole magnets had moved. The tilt -meter signals revealed that the magnets began moving many seconds before the losses occurred and caused the quench; it was at precisely the correct time the waves from the quake in Alaska should have arrived here. The remaining problem was coming up with a convincing way to present the information the next morning!
Dec 5, 2003: Dean Still
This marks a date that will stick with me when I think about failures in the Tevatron. It is the date of the 16-house quench that occurred while I was setting up to do studies that lead to the roman pot moving all the way through beam causing damage to spools pieces and the most impressive beam "etching" of a collimator we had ever seen. It was significant in that it illuminated a failure mode (fast quench) of the QPM system that was not really thought to have been able to happen. It lead to re-engineering the QPM and BLM systems in the Tevatron as well as testing that failure for the LHC. It was also significant in demonstrating the destructive power of uncontrolled beam.