... It’s a blooming miracle - a special LHC Report

Wrestling a 27 km superconducting collider under control is not easy. Throw in high intensity beams and it can sometimes seem a continual, frustrating battle with the vagaries of a hugely complex problem space.

This problem space has been thoroughly explored over the last three or four years. The myriad of potential obstacles to smooth operation ranges from unidentified falling objects (UFOs), electron clouds, beam dynamics, radiation to electronics, vacuum instabilities, “transparent” software changes, Radio Frequency (RF) trips, electrical network glitches etc. etc. The huge extended systems, such as the beam loss monitors, cryogenics, and quench protection systems, have a phenomenal number of components that inevitably have occasional failures (there is, of course, a higher probability that these occur late Friday evening and over the weekend). On the cryogenics front, cooling and keeping 36,000 tonnes of magnets at 1.9 K is a challenging prerequisite for everything else that follows.

Despite their complexity, we have very well behaved magnets and overall the machine is stable and magnetically reproducible. The magnets are well understood following a long and careful measurement campaign carried out during production. A sophisticated magnet model is even capable of dealing with the once feared dynamic effects. Together with the accuracy and stability of the power converters, our carefully optimized machine stays optimized. The injection, ramp and squeeze have been mastered and, as a general rule, injected beam makes it into collisions.

Exploitation of the LHC’s potential is helped by the presence of excellent beam instrumentation and powerful high-level software architecture. Together with some applied intelligence, these have allowed the development of tools (e.g. measurement and correction of optics, an on-line aperture model) which have opened the way to maximizing the performance of the machine. Most notably, accurate measurements of the aperture in the regions adjacent to the experiments have allowed us to reduce the beam size at the interaction points to unexpectedly low values. The better-than-expected aperture reflects good observance of tolerances during installation and very good alignment of all elements by the survey group.

The LHC has enjoyed from the start very good beam quality (both protons and ions) from the injector complex. The bunch currents have been well above, and the beam sizes well below, the nominal values quoted in the design report. The production of beam in the LINAC, Booster, PS, and SPS is distinctly non-trivial and requires continual care and attention to maintain the beam parameters, however this diligence is reflected directly in delivered luminosity.

Naturally, a cautious approach marked the re-starting in November 2009 following recovery from the 2008 incident. This was most clearly reflected in the choice to run at an initial beam energy of 3.5 TeV. Having experienced first hand the destructiveness of magnetic energy, awareness of the damage potential of the beam to the machine has underpinned the operational approach and marked the subsequent evolution in beam intensity. The full and proper functioning of the extended machine protection system (MPS) has always been an absolute priority.

The MPS consists of a federation of inputs from various systems into a beam interlock system (BIS). When the BIS is triggered it provokes a beam dump within 3 to 4 turns (that is, in a few hundred millionths of a second). The MPS has worked flawlessly, always pulling a beam abort when called upon to do so.

In addition to the MPS, the beam drives a subtle interplay of the beam dump system, the collimation system and protection devices, all of which rely on a well-defined aperture, orbit and optics for guaranteed safe operation. Assuring this throughout high intensity operation remains paramount. Numerous interlocks are in place to ensure that posted limits are always respected.

One notable feature during LHC commissioning and operation is the collective ability of CERN teams to resolve problems. There is in-depth expertise and experience on all systems, including vacuum, collimation, RF, fast kicker magnets and so on. Serious issues, such as radiation to electronics, and the lack of redundancy in protection systems, are targeted rigorously as they become apparent.  

Although precision and rigour are needed when dealing with the tightly synchronized choreography and the ever present dangers of magnetic and beam energy, an open and mostly friendly atmosphere pervades. A recent visitor to an 8:30 meeting noted the lack of defensiveness, and a willingness to directly engage a problem without needing to assign blame. There is amazing dedication from everybody involved. Problems that stop the operation of the machine happen anytime, with a slight preference for nighttime and the weekend. Despite this, there is unfailing support from all teams. Coming out of the technical stop this last weekend the Machine Protection and Electrical Integrity team worked until 5 in the morning on Saturday, and the control and timing teams were in from 2 to gone 6 on Sunday morning dealing with the effects of an innocent leap second.

Finally, trying to hold everything together across the accelerator complex is a talented, smart, moderately good-looking operations team who have necessarily developed an advanced sense of humour.

The LHC came out of a five-day technical stop on the evening of Friday 29 June. For real-time information on the operation of the machine, visit the LHC Page 1 status page.

by Mike Lamont for the LHC Team