It's a kind of Magic - On Shift at CMS
An experiment like CMS, where I have been working for six years now, is a complex apparatus. So many components and subsystems, both hard- and software, developed all over the world by so many institutes. It's almost hard to believe that it actually works as planned. The designers of the data acquisition system (DAQ) of CMS must have had similar thoughts, and so Freddie Mercury proclaims 'It's a kind of magic' every time the data taking is started.
During data taking, a shift crew of five is present in the control room at all times. Every one of those has an eye on different aspects of the whole system, eight hours at a time in three shifts, 7am-3pm, 3pm-11pm, and 11pm-7am. My job these days are so called trigger shifts. During normal operations, the LHC delivers proton-proton collisions to our detector every 25 nano-seconds. However, the great majority of data generated by CMS from these collisions is never stored by us. Mostly because we could never handle the huge amounts of data, but also because most of the stuff happening when two protons collide is pretty well known after decades of particle phyiscs experiment and we are not interested in it very much. The intersting processes, like the production of Higgs bosons, is pretty rare. So we have a trigger system, consisting of both hardware and software components, that can make the decision to keep the data from a collisions in fractions of a second. That makes it crucial for our success that this system works as intendend and one shifter is assigned the tasks to make sure it does. Which mostly involves finding out which sub-system of our detector is acting up when things go south, and call the relevant expert so they can fix the problem.
At this moment, the LHC has not started to deliver real collisions for this year, so we are still very muich in the preparation phase. Which means that more things go wrong than usual, but also that we are not missing any crucial data if somethings goes wrong, so things are a bit more relaxed in the control room compared to the real deal later this year.
Despite all my years in CMS, I have never done any actual control room shifts so far. So it's quite an interesting experience for me, given that I am normally just doing data analysis and software development. Therefore, I am enjoying working a bit closer to the detector for once. Also, our control room is just looks kinda cool:
Of course there is more going on during the day, when all the experts are present and working on their sub-systems. In the evening and at night, the five shifters are holding the line and make sure everything is running smoothly. And as long as it does, there actually is not much to do and one can work on other tasks or enjoy the boredom.
As there are not proton-proton collisions yet, we are measuring cosmic muons right now. Even 100 meters under ground, there are still a couple of them per second running through our detector and we can use the signals they create to calibrate and align the detector.
Six shifts down, 9 to go for this months, then I will have fulfilled my quota for this year, and I can relax, knowing that it is someone elses turn to wake up at 5am to be in the control room in time for the shift.