LHC Report: A tough restart

The third LHC Technical Stop of five days took place in the week of September 17. Getting back to normal operation after a technical stop  can sometimes be difficult, with debugging, testing and requalification required on the systems that have seen interventions. Folding in a selection of other problems can make for a frustrating time.


The new injector magnet is transported to the LHC. Photo: TE/ABT group.

The restart experienced over the last days was one of the tougher ones. Many problems occurred, both small and large, one after the other; in the end it took until Sunday afternoon, 9 days after the end of the technical stop, to have a physics fill in the machine that delivered an initial luminosity similar to those before the technical stop. Most problems encountered were, in fact, not related to the technical stop.

The technical stop consisted of the usual maintenance and consolidation of the various systems, but two items stand out: the replacement of the mirrors and supports of the Beam Synchrotron Light Monitors (BSRTs), which had to be put out of operation because of heating by the beam; and the replacement of one fast-pulsed kicker magnet used to inject the beam. The injection magnets have also suffered from beam-induced heating, sometimes resulting in a delay of the injection process of up to several hours during a good running period. In total there are eight of these magnets in the machine. The 'hottest' of these was replaced during the technical stop with a new version of the magnet with improved impedance reduction measures. The machine will gain some up-time from this replacement, and allow the new design to be checked under operational conditions. The injection magnet replacement was carefully planned and managed successfully in four and a half days, requiring around the clock work from all the teams involved.

Then the series of unfortunate events started: on Friday afternoon, 21 September, right at the end of the technical stop, the cryogenics in Point 8 was stopped by a technical fault. It took until Sunday for the cryogenic conditions to allow operation with beam to be re-established. Monday was spent with the standard test ramp with pilot beams to check the correct functionality of the LHC coming back from the technical stop. Commissioning with beam continued normally until Tuesday when a transformer switch on the external electrical network resulted in a glitch which caused another  cryogenics trip, this time in Point 2. The cryogenics system recovered overnight (from Tuesday to Wednesday) and the planned vacuum cleaning with beam of the newly installed injection magnet could finally start.

Some 13 hours of dedicating 'scrubbing' with beam were required to improve the vacuum conditions in the newly installed magnet. This was followed by the normal ramp-up in the number of bunches in the machine: a first fill had 452 bunches per beam in the machine, next 840 bunches and finally 2 fills with 1374 bunches per beam before nominal operational conditions were re-established on Sunday 30 September. Beam time during this ramp-up in intensity was interrupted by an earth fault on a quadrupole power converter due to a water leak that took 8 hours to be resolved, and a problem with an Uninterruptible Power Supply (UPS) on Saturday, which took out another 12 hours of beam time.

Since then, the LHC has been back to normal physics production, interrupted only on Monday 1 October for tests with 25 ns spaced bunches. This will continue until Monday 8 October when a five-day machine development period will start. The good news is that the newly installed injection magnet heats significantly less than the other injection magnets installed and it provides important positive information for the application of similar measures to the other injection magnets in the machine during the long shutdown in 2013 - 2014.

by Jan Uythoven for the LHC team