CERN Accelerating science

ATLAS Slides
Report number	ATL-SOFT-SLIDE-2022-550
Title	ARC-CE+HyperQueue based submission system of ATLAS jobs for the Karolina HPC
Author(s)	Svatos, Michal (Czech Academy of Sciences (CZ)) ; Chudoba, Jiri (Czech Academy of Sciences (CZ)) ; Vokac, Petr (Czech Technical University in Prague (CZ))
Corporate author(s)	The ATLAS collaboration
Collaboration	ATLAS Collaboration
Submitted to	6th Users' Conference of IT4Innovations, Ostrava, Czech Republic, 3 - 4 Nov 2022
Submitted by	michal.svatos@cern.ch on 20 Oct 2022
Subject category	Particle Physics - Experiment
Accelerator/Facility, Experiment	CERN LHC ; ATLAS
Free keywords	computing ; hpc
Abstract	For several years, the distributed computing of the ATLAS experiment at the LHC has been granted opportunistic use of computing resources of the Czech national HPC centre, IT4Innovations. With the introduction of Karolina HPC, resources provided to ATLAS significantly increased, but with lower efficiency. The inefficiency arose because ATLAS jobs, designed for 8 cores, have a rather short multiprocessing phase on the 128 cores of Karolina's worker nodes in comparison with initialization and finalization running on a single core. To ensure efficient usage, HyperQueue was implemented in the ARC-CE based submission system. This enables four 32-core jobs to be sent to each worker node, significantly improving CPU efficiency without leaving empty resources.