Normalized to: Tsaregorodtsev, A.
[1]
oai:arXiv.org:1403.7221 [pdf] - 1208663
DIRAC framework evaluation for the $\boldsymbol{Fermi}$-LAT and CTA
experiments
Arrabito, Luisa;
Cohen-Tanugi, Johann;
Diaz, Ricardo Graciani;
Longo, Francesco;
Kuss, Michael;
Piron, Frédéric;
Renaud, Matthieu;
Rolland, Vincent;
Sapunov, Matvey;
Tsaregorodtsev, Andreï;
Zimmer, Stephan
Submitted: 2014-03-27
DIRAC (Distributed Infrastructure with Remote Agent Control) is a general
framework for the management of tasks over distributed heterogeneous computing
environments. It has been originally developed to support the production
activities of the LHCb (Large Hadron Collider Beauty) experiment and today is
extensively used by several particle physics and biology communities. Current
($Fermi$ Large Area Telescope -- LAT) and planned (Cherenkov Telescope Array --
CTA) new generation astrophysical/cosmological experiments, with very large
processing and storage needs, are currently investigating the usability of
DIRAC in this context. Each of these use cases has some peculiarities:
$Fermi$-LAT will interface DIRAC to its own workflow system to allow the access
to the grid resources, while CTA is using DIRAC as workflow management system
for Monte Carlo production and analysis on the grid. We describe the prototype
effort that we lead toward deploying a DIRAC solution for some aspects of
$Fermi$-LAT and CTA needs.
[2]
oai:arXiv.org:1212.4115 [pdf] - 603688
Extending the Fermi-LAT Data Processing Pipeline to the Grid
Submitted: 2012-12-17
The Data Handling Pipeline ("Pipeline") has been developed for the Fermi
Gamma-Ray Space Telescope (Fermi) Large Area Telescope (LAT) which launched in
June 2008. Since then it has been in use to completely automate the production
of data quality monitoring quantities, reconstruction and routine analysis of
all data received from the satellite and to deliver science products to the
collaboration and the Fermi Science Support Center. Aside from the
reconstruction of raw data from the satellite (Level 1), data reprocessing and
various event-level analyses are also reasonably heavy loads on the pipeline
and computing resources. These other loads, unlike Level 1, can run
continuously for weeks or months at a time. In addition it receives heavy use
in performing production Monte Carlo tasks.
The software comprises web-services that allow online monitoring and provides
charts summarizing work flow aspects and performance information. The server
supports communication with several batch systems such as LSF and BQS and
recently also Sun Grid Engine and Condor. This is accomplished through
dedicated job control services that for Fermi are running at SLAC and the other
computing site involved in this large scale framework, the Lyon computing
center of IN2P3. While being different in the logic of a task, we evaluate a
separate interface to the Dirac system in order to communicate with EGI sites
to utilize Grid resources, using dedicated Grid optimized systems rather than
developing our own. (abstract abridged)