Normalized to: Demichev, A.
[1]
oai:arXiv.org:1812.01324 [pdf] - 1791980
Using Binary File Format Description Languages for Documenting, Parsing,
and Verifying Raw Data in TAIGA Experiment
Bychkov, I.;
Demichev, A.;
Dubenskaya, J.;
Fedorov, O.;
Hmelnov, A.;
Kazarina, Y.;
Korosteleva, E.;
Kostunin, D.;
Kryukov, A.;
Mikhailov, A.;
Nguyen, M. D.;
Polyakov, S.;
Postnikov, E.;
Shigarov, A.;
Shipilov, D.;
Zhurov, D.
Submitted: 2018-12-04
The paper is devoted to the issues of raw binary data documenting, parsing
and verifying in astroparticle data lifecycle. The long-term preservation of
raw data of astroparticle experiments as originally generated is essential for
re-running analyses and reproducing research results. The selected high-quality
raw data should have detailed documentation and accompanied by open software
tools for access to them. We consider applicability of binary file format
description languages to specify, parse and verify raw data of the Tunka
Advanced Instrument for cosmic rays and Gamma Astronomy (TAIGA) experiment. The
formal specifications are implemented for five data formats of the experiment
and provide automatic generation of source code for data reading libraries in
target programming languages (e.g. C++, Java, and Python). These libraries were
tested on TAIGA data. They showed a good performance and help us to locate the
parts with corrupted data. The format specifications can be used as metadata
for exchanging of astroparticle raw data. They can also simplify software
development for data aggregation from various sources for the multi-messenger
analysis.
[2]
oai:arXiv.org:1811.12086 [pdf] - 1791770
Russian-German Astroparticle Data Life Cycle Initiative
Bychkov, Igor;
Demichev, Andrey;
Dubenskaya, Julia;
Fedorov, Oleg;
Haungs, Andreas;
Heiss, Andreas;
Kang, Donghwa;
Kazarina, Yulia;
Korosteleva, Elena;
Kostunin, Dmitriy;
Kryukov, Alexander;
Mikhailov, Andrey;
Nguyen, Minh-Duc;
Polyakov, Stanislav;
Postnikov, Evgeny;
Shigarov, Alexey;
Shipilov, Dmitry;
Streit, Achim;
Tokareva, Victoria;
Wochele, Doris;
Wochele, Jürgen;
Zhurov, Dmitry
Submitted: 2018-11-29
Modern large-scale astroparticle setups measure high-energy particles, gamma
rays, neutrinos, radio waves, and the recently discovered gravitational waves.
Ongoing and future experiments are located worldwide. The data acquired have
different formats, storage concepts, and publication policies. Such differences
are a crucial point in the era of Big Data and of multi-messenger analysis in
astroparticle physics. We propose an open science web platform called
ASTROPARTICLE.ONLINE which enables us to publish, store, search, select, and
analyze astroparticle data. In the first stage of the project, the following
components of a full data life cycle concept are under development: describing,
storing, and reusing astroparticle data; software to perform multi-messenger
analysis using deep learning; and outreach for students, post-graduate
students, and others who are interested in astroparticle physics. Here we
describe the concepts of the web platform and the first obtained results,
including the meta data structure for astroparticle data, data analysis by
using convolution neural networks, description of the binary data, and the
outreach platform for those interested in astroparticle physics. The
KASCADE-Grande and TAIGA cosmic-ray experiments were chosen as pilot examples.
[3]
oai:arXiv.org:1811.02403 [pdf] - 1785865
Architecture of Distributed Data Storage for Astroparticle Physics
Submitted: 2018-11-06
For the successful development of the astrophysics and, accordingly, for
obtaining more complete knowledge of the Universe, it is extremely important to
combine and comprehensively analyze information of various types (e.g., about
charged cosmic particles, gamma rays, neutrinos, etc.) obtained by using divers
large-scale experimental setups located throughout the world. It is obvious
that all kinds of activities must be performed continually across all stages of
the data life cycle to help support effective data management, in particular,
the collection and storage of data, its processing and analysis, refining the
physical model, making preparations for publication, and data reprocessing
taking refinement into account. In this paper we present a general approach to
construction and the architecture of a system to be able to collect, store, and
provide users' access to astrophysical data. We also suggest a new approach to
the construction of a metadata registry based on the blockchain technology.