Normalized to: Oberto, A.
[1]
oai:arXiv.org:1611.01312 [pdf] - 1508841
An Hierarchical Approach to Big Data
Allen, M. G.;
Fernique, P.;
Boch, T.;
Durand, D.;
Oberto, A.;
Merin, B.;
Stoehr, F.;
Genova, F.;
Pineau, F-X.;
Salgado, J.
Submitted: 2016-11-04
The increasing volumes of astronomical data require practical methods for
data exploration, access and visualisation. The Hierarchical Progressive Survey
(HiPS) is a HEALPix based scheme that enables a multi-resolution approach to
astronomy data from the individual pixels up to the whole sky. We highlight the
decisions and approaches that have been taken to make this scheme a practical
solution for managing large volumes of heterogeneous data. Early implementors
of this system have formed a network of HiPS nodes, with some 250 diverse data
sets currently available, with multiple mirror implementations for important
data sets. This hierarchical approach can be adapted to expose Big Data in
different ways. We describe how the ease of implementation, and local
customisation of the Aladin Lite embeddable HiPS visualiser have been keys for
promoting collaboration on HiPS.
[2]
oai:arXiv.org:1611.01374 [pdf] - 1508848
MocServer: What & Where in a few milliseconds
Submitted: 2016-11-04
The MocServer is an astronomical service dedicated to the manipulation of
data set coverages. This server sets together about 15 000 spatial footprints
associated to catalogs, data bases and pixel surveys. Thanks to the Multi-Order
Coverage map coding method (MOC1.0 IVOA standard), the MocServer is able to
provide in a few milliseconds the list of data set identifiers intersecting any
polygon on the sky. The MOC server has been deployed in June 2015 by the Centre
de Donnees astronomiques de Strasbourg. It is operational and already in use by
Aladin Desktop and Aladin Lite prototype versions.
[3]
oai:arXiv.org:1505.02291 [pdf] - 1183004
Hierarchical progressive surveys. Multi-resolution HEALPix data
structures for astronomical images, catalogues, and 3-dimensional data cubes
Fernique, P.;
Allen, M. G.;
Boch, T.;
Oberto, A.;
Pineau, F-X.;
Durand, D.;
Bot, C.;
Cambresy, L.;
Derriere, S.;
Genova, F.;
Bonnarel, F.
Submitted: 2015-05-09
Scientific exploitation of the ever increasing volumes of astronomical data
requires efficient and practical methods for data access, visualisation, and
analysis. Hierarchical sky tessellation techniques enable a multi-resolution
approach to organising data on angular scales from the full sky down to the
individual image pixels. Aims. We aim to show that the Hierarchical progressive
survey (HiPS) scheme for describing astronomical images, source catalogues, and
three-dimensional data cubes is a practical solution to managing large volumes
of heterogeneous data and that it enables a new level of scientific
interoperability across large collections of data of these different data
types. Methods. HiPS uses the HEALPix tessellation of the sphere to define a
hierarchical tile and pixel structure to describe and organise astronomical
data. HiPS is designed to conserve the scientific properties of the data
alongside both visualisation considerations and emphasis on the ease of
implementation. We describe the development of HiPS to manage a large number of
diverse image surveys, as well as the extension of hierarchical image systems
to cube and catalogue data. We demonstrate the interoperability of HiPS and
Multi-Order Coverage (MOC) maps and highlight the HiPS mechanism to provide
links to the original data. Results. Hierarchical progressive surveys have been
generated by various data centres and groups for ~200 data collections
including many wide area sky surveys, and archives of pointed observations.
These can be accessed and visualised in Aladin, Aladin Lite, and other
applications. HiPS provides a basis for further innovations in the use of
hierarchical data structures to facilitate the description and statistical
analysis of large astronomical data sets.