Normalized to: Emad, N.
[1]
oai:arXiv.org:1805.01844 [pdf] - 1677365
Polynomial data compression for large-scale physics experiments
Submitted: 2018-05-03
The new generation research experiments will introduce huge data surge to a
continuously increasing data production by current experiments. This data surge
necessitates efficient compression techniques. These compression techniques
must guarantee an optimum tradeoff between compression rate and the
corresponding compression /decompression speed ratio without affecting the data
integrity.
This work presents a lossless compression algorithm to compress physics data
generated by Astronomy, Astrophysics and Particle Physics experiments.
The developed algorithms have been tuned and tested on a real use case~: the
next generation ground-based high-energy gamma ray observatory, Cherenkov
Telescope Array (CTA), requiring important compression performance.
Stand-alone, the proposed compression method is very fast and reasonably
efficient. Alternatively, applied as pre-compression algorithm, it can
accelerate common methods like LZMA, keeping close performance.