ATLAS Phase-II-Upgrade Pixel data transmission development

The ATLAS tracking system will be replaced by an all-silicon detector in the course of the planned upgrade of the Large Hadron Collider around 2025. The readout of the new pixel system will be most challenging in terms of data rate and readout speed. Simulations of the on-detector electronics based on the currently foreseen trigger rate of 1 MHz indicate that a readout speed of up to 5 Gbit/s per data link is necessary. Due to radiation levels, the first part of transmission has to be implemented electrically. System simulation and test results of cable candidates will be presented.


Introduction
The Large Hadron Collider (LHC) is undergoing a series of upgrades, which will result in the High Luminosity LHC (HL-LHC) [1] with a luminosity of 5 · 10 34 cm −2 s −1 . To cope with the higher luminosity level, ATLAS [2] is also undergoing a series of upgrades. The "Phase-0" ATLAS upgrade is complete and included the addition of a new inner pixel layer. The "Phase-I" upgrade in 2018 will include mostly trigger system upgrades, and the "Phase-II" upgrade, planned for 2024-2026, will see the replacement of the complete inner detector by the new Inner Tracker (ITk) [3,4]. Currently, there are several different layouts under discussion (see [5,6]). One possible layout candidate for the ITk is shown in figure 1a. It consists of a pixel detector with five barrel layers and four endcap rings and a strip detector with four layers and six endcap disks. This layout covers a volume up to 1 m radius around the interaction point and provides nine space points up to |η| ≈ 4. The active area of both detectors is built from about 200 m 2 silicon. Currently, it is planned to have a pixel size of 50 × 50 µm 2 for the innermost layers of the pixel detector. During its operation the ITk has to withstand a huge radiation dose. Figure 1b shows a simulation of the total ionising dose after the expected HL-LHC runtime. The HL-LHC will deliver about 3000 fb −1 of proton-proton collisions to ATLAS. For the innermost layer of the ITk Pixel Detector a total ionising dose of about 10 MGy will be reached. This high level of radiation imposes strong requirements on all on-detector components including cables, support structures, and electronics.
For Phase-II the ATLAS trigger system will also be upgraded. The new trigger system will use data from the tracking detectors as input to the trigger decision. The high luminosity of the HL-LHC requires new mechanisms to reduce backgrounds and to stabilise the trigger efficiency. Currently, ATLAS is investigating two possible options for the trigger system architecture. The first option is a two stage trigger system, with the first trigger stage (L0) running at at 1 MHz with 10 µs  Picture of a possible ITk layout with nine space points up to |η| ≈ 4 consisting of a pixel detector (blue, R < 300 mm) and a silicon strip detector (red, R > 300 mm) [8] together with a FLUKA 2011.2c.4 simulation of the total ionising dose in the ITk region normalised to 3000 fb −1 of pp collisions at 14 TeV [9] latency and the second trigger stage (L1) running at at 400 kHz with 60 µs latency. The hardware track trigger system [7] would use data from the outer pixel layers and the strip detector as input for an L1 trigger decision. The second option for the trigger architecture is a single stage trigger with up to 4 MHz trigger rate with a maximum latency of 25 µs. In this architecture the hardware track trigger would be replaced by a software track trigger in the event handler system. This paper will discuss the current developments around the data transmission for the ITk Pixel Detector. First, the data transmission scheme will be presented. Then an overview of the status of the cable developments for the data transmission will be given. Finally, the paper will also show simulation results of expected hit occupancies with data rate estimations.

Data transmission scheme
The data transmission scheme for the ITk foresees several stages of different cables and transmission technologies [10]. The detector needs a bidirectional data link, where the link to the detector will carry the trigger and control signals (TTC) running at a data rate of 160 Mbit/s. These signals are shared between a group of front-end chips. Each individual front-end chip is then addressed in the commands of the TTC data stream. The cables from the detector will carry the hit information from the detector at rate of about 5 Gbit/s. In the innermost layer of the detector each front-end chip will have a dedicated uplink towards the off-detector region. The data links of the outer layers will be shared between two or four front-end chips, as the bandwidth requirement per front-end chip decreases with distance to the interaction point.
A schematic view of this transmission scheme is shown in figure 2. In total the cables need to cover a distance between the detector and the off-detector readout electronics of about 90 m. The transmission is divided into three sections separated by patch panels (PP). The first stage of data transmission will be done via flex cables or twisted-pair cables. These cables are terminated at the first patch panel (PP0) and depending on its position they have to be about 1.5 m long. For reasons of the high radiation and to keep them accessible the opto-electrical converters cannot be placed in the detector volume but at the transition from the Inner Detector to the calorimeters (ID-endplate, PP1). The distance between the stave and the opto-electrical converters is about 5 to 7 m and will be bridged by copper cables. Finally, the connection to the off-detector readout hardware is done with about 80 m of optical fibres. The next section will briefly describe the three main cable types, which are currently under evaluation.

Cables
For the electrical data transmission from the modules to the opto-electrical converters different types of cables are under evaluation [11]. The main goals are to provide stable communication at data rates up to 5 Gbit/s over 7 m cable with a target bit error rate of 10 −12 . Also the the required material must be kept as low as possible. Another important aspect is the resistance against the high radiation level in the detector volume, which all cables have to fulfill. Due to the high data rates running through the cables only differential signalling will be considered and this is also reflected in the types of cables.

Twisted pair
The easiest solution to transfer differential signals is to use twisted pair cables. In these cables two conductors are twisted together, which reduces the impact of electromagnetic interference by canceling common-mode interference. Figure 3 shows the profile of such a twisted pair wire evaluated for the ITk Pixel Detector. Particular things to note about this cable are the thinness of the conductors (0.127 mm, 36 AWG1) and the integral shielding with 10 µm aluminium foil, aiming for a balance between good performance and low material radiation length -the material budget contribution in percent of the radiation length is about X/X 0 = 0.027 %. During tests of a 1.15 m prototype a data rate of 6.2 Gbit/s has been reached by using 8b10b encoding and pre-emphasis in the transmitter.
Also the effect of having equalisation techniques in the receiver has been evaluated [12,13]. Figure 4 shows the eye diagram of a 3 m twisted pair cable prototype before and after equalisation. The opening of the eye diagram is much bigger due to intersymbol interference (ISI) being reduced.

TwinAx
The twinaxial cable (TwinAx) is an enhancement of the twisted pair cable as it is a dual coaxial cable with a common shield. Therefore, it has much better transmission properties at the cost of    [12]. It can be seen that adding equalisation to the signal improves the opening of the eye significantly and therefore is required for reliable data transmission through the twisted pair cable. more material inside the detector volume. For a 30 AWG copper-cladded aluminium TwinAx cable the material budget in percent of the radiation length is about X/X 0 = 0.076 % smeared over the stave, which is a factor of 3 higher than the twisted pair cable. Figure 5 shows a photo and a labeled schematic of a TwinAx prototype. Different prototypes have been tested with a bit error rate tester to up to 10 Gbit/s over 6 m cable. In these tests the bit error rate was always below the target rate of 10 −12 . Table 1 shows the results of the measurements. With these measurements it was proven that the TwinAx cables can be used for transmitting the data from the detector towards the opto-electrical converters. However, due to their large radiation length they must be combined with lightweight cables or flexes in the inner detector regions.

Flex cable
For the data transmission on the staves of the ITk pixel detector flex cables can also be used [14]. These cables are made of layers of flexible plastic substrates (polyimide, PEEK) alternating with thin copper planes and they are directly glued to the mechanical support structures. Through the   flex cables all data signals as well as the module powering and the high voltage for the sensors are distributed. A sectional drawing through a flex is shown in figure 6. The flex is constructed out of polyimide and designed to have a differential impedance of 100 Ω.
A 1 m flex cable prototype has been evaluated and a bit error measurement has been performed. The prototype has been rated up to 6.2 Gbit/s with a bit error rate of 2.6 · 10 −15 . In this test a commercial FPGA board was used for generating 8b10b encoded signals and measure the bit error rate. Additionally, a pre-emphasis was added in the transmitter. Figure 6. Sectional drawing of the layer stacking inside the flex cable prototype. All distances are optimised to achieve a differential impedance of 100 Ω.

Simulations
The ITk front-end chip will be designed to provide a maximum data rate of 5 Gbit/s. With this data rate and the assumption of 1 MHz trigger rate each event can occupy up to 5000 bits on the data link. If multiple front-end chips share the same physical transmission line the hit information from all connected front-end chips has to fit into these 5000 bits. Therefore, it is important to know how many hits per front-end chip are expected and how these are distributed over the front-end chip. Near the interaction point (z=0) there are fewer hits than in the forward regions (z=max.). However, the forward region has larger clusters allowing a better compression due to cluster-based encoding. Table 2 shows the results of two possible encoding schemes (hit-based and cluster-based encoding). The numbers given in the table are based on simulations of the current baseline design, and as the design is likely to change somewhat before we actually build it. Therefore, the predictions should be taken as representative of what we can achieve, not as exact.
Summarizing the results from the simulations it is required to have different encodings for the different regions in the detector. Especially, in the inner layers of the detector additional compression will be needed to get the data out of the detector. The development of an adequate transmission protocol is a challenging task for ITk-DAQ development and the front-end design. If it is not possible to directly switch between the two different encoding schemes in the front-end chip a compromise between the hit-based and cluster based encoding has to be found. The most likely solution will be the encoding of hits inside a 2 × 2, 3 × 3 or 4 × 4 pixel region into a cluster in the front-end chip.

Summary
To cope with the new requirements of the HL-LHC, ATLAS will install a new tracking detector, the ITk. The ITk Pixel Detector will be the innermost part and has to cope with high hit occupancies and a harsh radiation environment. One topic in the research and development for the ITk Pixel Detector is the data transmission strategy. Different cables, including twisted pair, TwinAx and flex cables, are currently being evaluated. These cables have to allow a stable data transmission at 5 Gbit/s, but should not bring in too much material into the detector volume. The most recent measurements show that most likely the cable will be a hybrid between twisted pair or flex cables and TwinAx cables. However, to achieve good signal quality the data transmission must use pre-emphases, equalisation and signal balancing.
In terms of available bandwidth for the data transmission of the hit data, simulations with the latest ITk layout have been performed. In these simulations the hit occupancies for all front-end chips has been calculated. The highest hit-occupancies will be in the innermost barrel layers. It is already clear that for this layer the data stream has to be compressed to read out all the data. For the outer layers the situation is more relaxed, as the occupancies are lower.