The optical Synchronization and Link Board project, oSLB

The calorimeter trigger synchronization of the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider (LHC) uses a synchronization method implemented in the synchronization and link board (SLB). These boards allow the synchronization of electromagnetic and hadronic trigger primitives at the LHC frequency (40.08 MHz) and its transmission to the Regional Calorimeter Trigger. The upgrade of the Calorimeter Trigger system dictates the use of input optical links at a rate of 4.8 Gb/s. In this paper we present the design options and technological choices for the optical part of new optical Synchronization and Link Boards (oSLB).


Overview
The upgrade of the CMS Calorimeter Trigger (CT) system implies the use of input optical links at a rate of 4.8 Gb/s, which should allow the implementation of a compact trigger system based on uTCA technology with better performance compared to the present system. This choice will force the replacement of the mezzanines that currently synchronize and transmit the calorimeter trigger primitives from Electromagnetic Calorimeter (ECAL) and Hadron Calorimeter (HCAL) to the Regional Calorimeter Trigger using high speed cables, the Synchronization and Link Board (SLBs), by new ones using optical links, the oSLBs. Figure 1 shows the connections diagram of the calorimeter trigger boards foreseen to be installed during the Long Shutdown 1 (2013-2014).
The present four high speed 1.2 GB/s electrical links used on the SLB will be replaced by two optical links per board, running at 4.8 Gb/s. These two links have the same trigger data that we sent to the old and to the new Trigger systems, allowing to run the present Calorimeter Trigger system in parallel with the development and commissioning of the new uTCA Trigger.
On the receiver side, on the Regional Calorimeter Trigger, all electrical mezzanines (RM) have to be also replaced with an optical version. To simplify the design of these new optical receivers boards (oRM) we have decide to use the same FPGA and design approach as for the oSLB.
In this paper we describe the main functionality of the oSLB mezzanine, the steps on the optical device selection as well the studies to reduce costs, and the planning for installation on LS1 are presented.

The synchronization method
The trigger system of the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider (LHC) is a synchronous and pipelined system working at 40.08 MHz. The system can be viewed as a massive parallel processor that computes local trigger objects, followed by a tree like structure that selects the highest rank objects in the detector or performs global energy sums. At each processing stage data must be synchronized.
Calorimeter trigger data are primarily generated at the Trigger Primitive Generators and must be processed by the Regional Calorimeter Trigger (RCT) processors housed in the RCT crates. Non-negligible differences on the trigger primitives arrival time to the processors are introduced by different particle flight paths, different optical transmission fiber lengths and different phase lock delays in the electronic serializers. A synchronization stage therefore must exist before data transmission.
Channel synchronization relies on the Timing Trigger and Control (TTC) Bunch Crossing zero (BC0) broadcast command that can be adjusted relatively to the LHC orbit signal, on a synchronization FIFO and on the accumulator histogram that reflects the LHC bunch crossing structure [1]. A common hardwired control distribution guarantees aligned trigger data transmission over the high speed optics.
3 Description of the oSLB

Trigger synchronization with oSLB
The oSLB receives data of up to 8 Trigger Towers at 40 MHz on two PMC 64-pin connectors, the minimum partition of the input data being one Trigger Tower. Input data per Trigger Tower is 9-bit data, with 8 bits coding the tower transverse energy and 1 bit coding the fine-grain information.
-2 - The synchronization of the data is performed by eight Sync Core blocks, each one responsible for the synchronization and data alignment of a single Trigger Tower.
An adjustable synchronization FIFO allows to change the position of the received trigger data (to compensate the delays of the data reception), an energy histogram is used to check the correct alignment during operation and Error counters check the stability of the reference signals (clock and BC0) on every orbit [2]. Any variation of the energy histogram will show that there was a jump on the phase of the input data.

oSLB design choices
The technical choices for the oSLB design are presented in this section. The most important parameter is whatever we choose for the oSLB design has to comply with the present constraints of the mainboards of the ECAL trigger, from the communication and configuration interface up to the available power per slot.
There are four main points on the oSLB design: 1. We have to maintain the same form factor PCB, the connectors interface type, pin out to the existing system, and use the present maximum power available.
2. The FPGA to be used has to be compatible with the existing interface signal levels.
3. The optical device selection, has to fit on the existing "free" space on the Trigger Concentrator Cards (ECAL Trigger boards, TCCs) 4. Price per oSLB. We have to produce 650 units of these boards, so the cost of the main components (FPGA, optics) becomes the most important point.
We have to make sure that, with minimum impact on the existing hardware, we can replace the SLB mezzanines in use by a new one (oSLB) that uses optical transceivers instead of copper cable connectors, using the available mechanical space and constraints, same connectors, same form factor and use the same logical interface and signal levels. Also, both, the performances of the optical device and its costs are relevant for its choice.

Form factor constraints
A fundamental problem when upgrading just a component in a system is that we have to be careful with the form factor, the placement and size of the components, the overlap with the components existing on the other board, the limits of the space available and all choices have to be carefully studied. The original connectors for the interface to the main board have to be maintained as well the pin assignments and logical levels. After all this requirements met the oSLB PCB will match the previous design dimensions, 122 × 38 mm. Figure 2 shows a *D view of the oSLB mezzanine board.

FPGA choice
We have waited for the latest generation devices of the Xilinx FPGAs Kintex7 family to be available, that meet the requirements on the high speed links, and that are much more cost effective, to choose the final FPGA. We will use the Kintex7 70T devices, these are low cost high performance -3 -  devices, with 8 Transceivers up to 6.125 Gb/s implemented on a fine pitch BGA 676 pins package that fits on the PCB dimensions and complies with the power budget available.

The oSLB optical device
For more than 2 years a number of optical devices as been evaluated. After this long process we have finally chosen to use a modified version of the "versatile link" being developed at CERN [3]. This modified version, showed on figure 3, has the receiver module replaced by a transmitter module that can operate up to 8gbps, housed on a short "SFP" [4] cage version that will fit the requirements for the oSLB pcb design and its integration on the existing trigger boards.
Another important aspect of using a dual transmitter device, with separated optical connector is that this greatly simplifies the optical connections between the oSLB and the new designs, both, uTCA or oRMs. The LC connectors of these devices will allow a point-to-point connection avoiding the need of an optical dispatcher that would greatly increase the number of optical connections.