The IBL readout system

The first upgrade for the ATLAS Pixel Detector will be an additional layer, which is called IBL (Insertable B-Layer). To readout this new layer, built from new electronics, an update of the readout electronics is necessary. The aim is to develop a system which is capable to read out at a higher bandwidth, but also compatible with the existing system to be integrated into it. This paper describes the necessary development to reach a new readout system, concentrating on the requirements of a newly designed Back of Crate card as the optical interface in the counting room.


Introduction
In the scope of the LHC phase I upgrade, increased luminosity is expected within the experiments, leading to a decrease of the ATLAS Pixel Detector [1] performance. In particular, separation of pileup events will be increasingly difficult and higher module occupancies will decrease the readout efficiency.
To overcome these deficiencies, a fourth pixelised layer, the Insertable B-Layer [2] (IBL), will be inserted into the current Pixel Detector during the LHC phase I upgrade. Insertion is possible due to a new beampipe design with smaller radius, that will come with the IBL. The new layer will provide an increased intrinsic resolution in z and increased b-tagging performance.
The IBL will be built in 14 staves with 16 modules each. A module unit will be built from two FrontEnd I4 (FE-I4) [3] integrated circuits, bump bonded to pixel sensors. Command and readout will be transmitted optically between the off-detector system and a transceiver system located close to the detector, like in the current Pixel subsystem.

The Pixel Detector readout system
As the IBL will be run as part of the Pixel Detector subsystem, this section will introduce boundary conditions of the pixel system and limitations to be surpassed within the IBL readout system. The Pixel readout system off-detector electronics are built on the basis of VME. 1 Each readout system building block can supply up to 26 Pixel modules with clock and command data and read them out via one or two data links at either 40 or 80 Mbit/s each. Each module is given its separate transmission line(s) for both, command and data transfer. A readout system building block can deliver a maximum total data rate of 160 MByte/s. Each building block contains a ReadOut Driver (ROD) delivering the VME bus interface, and configuration and control of the detector. It can process the incoming Pixel data frames and align that data for multiple modules, delivering an event fragment to the ATLAS TDAQ 2 systems.
Optical interfaces within the building blocks are delivered by the Back Of Crate (BOC) card, connected to fibres which are routed to the detector as well as the ATLAS TDAQ system. The BOC also serves all needed functionality within the analogue link frame -laser drivers with forward current control, receiver thresholds and phase adjustment of data.

TTC connection
The Timing, Trigger and Control connection (TTC), serves clock and command to the detector. For the ATLAS Pixel Detector it is delivered by a custom made component, which can be plugged into the BOC, the TX-plugin. The plugin can supply a maximum of 12 modules with optical signals through a 12-way VCSEL array. Each module is given its own optical transmission line.
The TTC connection delivers clock and command to the detector, encoded into a single stream using BiPhase Mark (BPM 3 ) encoding. The encoder chip on the TX-plugin, the BPM-12 [4], includes timing adjustments on the signal and an output laser driver. Timing adjustment allows to place the detector timing relative to the LHC collisions in steps of about 320 ps. As the laser driver does not support pre-emphasis, a mark to space regulation with the same step size is also embedded within the chip to correct the outgoing signals duty cycle manually.

Data connection
The Pixel data connection comes in Non-Return to Zero (NRZ) encoded lines, either one or two per module depending on the location of the module within the detector. The outgoing data transmission modes can be selected to be either 1 × 40 Mb/s, 2 × 40 Mb/s, 1 × 80 Mb/s or 2 × 80 Mb/s, delivering a maximum of 160 Mb/s per module. The current receiver system does not allow for automatic adjustment of receiver thresholds or phases, to sample the incoming NRZ streams. This is done instead by varying the threshold and phase parameter spaces while transmitting a known pattern from the detector to the off-detector electronics and measuring the error rate. Pixel data links tend to be unstable, in case noise is inserted into the on-detector transmitters power adjustment. Due to missing pre-emphasis, the error free transmission range in terms of phase adjustment turns out to be very limited at 80 Mb/s line speed.

IBL readout requirements
Each IBL module needs to be supplied with a single input command line and a clock source, synchronising it with the LHC machine. In addition there are two data links, one coming from each frontend chip. The Timing Trigger and Command (TTC) link, controlling the detector, runs at 40 Mbit/s. It uses BiPhase Mark encoding in the optical path, to keep the same system as used for the ATLAS Pixel Detector. As the frontend does not support decoding of that transmission, it will be decoded by an opto-electrical converter outside the ATLAS inner detector. Data return per frontend runs with a 160 Mbit/s transmission, encoded using 8B/10B coding scheme.
Running as part of the Pixel subsystem, IBL will be placed within the same readout frame, i.e. VME. Using more powerful electronics than in the current system, the IBL readout building block will deliver an increased bandwidth of 640 MB/s, serving a maximum of 16 modules with 32 frontend ICs per building block.

Back of crate card
The IBL back of crate card will be equipped with 24 TTC transmitters and 48 data receivers to establish the aforementioned module connections. Spare optical channels will be available for exchanging a malfunctioning transmission line, by embedding multiplexing functionality in all transmitters and receivers.
A central programmable device, probably a Virtex 5 series FPGA, will be connected to all I/Os, both on the crates backplane and the BOC cards faceplate, thereby giving maximum routing and programming flexibility. Aside the central device, a controlling bus interface will be delivered by a small programmable device, allowing safe in-system firmware upgrades of the central device and basic monitoring functionality.

Firmware encoded frontend connection
As the Pixel subsystem suffered major issues with its TX-plugins, the IBL should not use them. Instead the BiPhase Mark encoder will be placed within the central devices firmware. It has been successfully tested and delivers not only the encoded stream at minimal latency of less than one clock cycle, but also embeds capabilities to shift the outgoing signal in phase versus the LHC bunch crossing clock with a step size as low as 75 ps.
An adjustment of the mark to space ratio has been programmed and tested, but will probably not be needed within the IBL system. This is due to the replacement of the BPM-12 laser driver with an of the shelf electro-optical converter, delivering a one-to-one translation of the electrical input signal into the optical output signal on sub-nanosecond scale. Buying optical components from an industrial market will not increase cost of the system, yet making it a lot more reliable and maintainable.

Data receiver
As the IBL will return a balanced 8B/10B encoded stream at 160 Mbit/s per link, the new receiver circuitry can implement several advantages over the previous system: • AC coupled receiver logic: Due to the balanced encoding, the receiver circuitry can run AC coupled. It therefore delivers a bias free stream, automatically adapting to subtle changes in the data transmission line. A TIA 4 can be utilised in the first receiver stage, increasing the maximum frequency performance.
• Transition based code: Regular transitions within the received stream allow to automatically synchronise the received data with a local clock, embedding standard synchronisation techniques into the BOC cards central programmable device.
• Bytewise encoding: The receiver can be split into an 8B/10B decoder delivering bytewise output, and a formatting section knowing the frontend data coding scheme. Therefore the individual logic blocks will get simpler in structure and easier to debug than was in the current Pixel Detector readout system.
For all of this, the receiver logic block will be constructed as shown in figure 2. Opposed to the previous BOCs usage, the IBL BOC will deliver packaged data to the ROD and take care of initial buffering.

Embedded S-Link
Increasing the data density of the readout system building blocks requires an increased output bandwidth towards the ATLAS TDAQ systems. This will be delivered using multiple S-Links. 5 The previous S-Link implementation was a daughterboard to the BOC card. Due to space limitations, the BOC card will generate the high speed S-Link signal itself, using a firmware based encoder and Figure 2. The IBL data receiver logic block. Input data from the detector comes in from the left, is aligned, sampled and decoded. Afterwards all IDLE statements are dropped and only data within a Start of Frame/End of Frame are handed to the output buffer block. Two blocks aside, deliver error monitoring of the stream and timeout logic to determine whether an expected packet did not arrive in time (replacing it with a timeout packet).
MGTs, 6 available in the central programmable device. A 1 × 4 SFP 7 cage will connect to the AT-LAS TDAQ system. A second cage of the same type will hold copies of the outgoing data streams to be delivered to a future FTK. 8

Testbed
A testbed for readout system firmware implementation will be set up from two individual evaluation boards (c.f. figure 3). The right hand side board will emulate ROD functionality, including a PowerPC R processor to develop a first set of firmware to run the initial system. The left hand side board is to implement the BOC functionality checking communication with the detector frontend, as well as with the readout driver. First versions of the IBL readout will thereby soon be tested for operation, needs and misconception.

Conclusions
We are proposing a system that delivers the full IBL readout within 14 building blocks, fitting into a single 9U VME crate. Within this paper we described the basic functional design for the I/O interface of the IBL readout system, introducing automated operation of all links and an increased total bandwidth. ! Figure 3. Two different development boards will serve for an initial test setup, delivering a PowerPC R processor on the ROD emulation and large logic space as well as a custom I/O board on the BOC emulation.