The Complete Calibration of the Colour-Redshift Relation survey (C3R2) is a spectroscopic effort involving ESO and Keck facilities designed to empirically calibrate the galaxy colour-redshift relation - P(z|C) to the Euclid depth (i_AB=24.5) and is intimately linked to upcoming Stage IV dark energy missions based on weak lensing cosmology. The aim is to build a spectroscopic calibration sample that is as representative as possible of the galaxies of the Euclid weak lensing sample. In order to minimise the number of spectroscopic observations to fill the gaps in current knowledge of the P(z|C), self-organising map (SOM) representations of the galaxy colour space have been constructed. Here we present the first results of an ESO@ VLT Large Programme approved in the context of C3R2, which makes use of the two VLT optical and near-infrared multi-object spectrographs, FORS2 and KMOS. This paper focuses on high-quality spectroscopic redshifts of high-z galaxies observed with the KMOS spectrograph in the H- and K-bands. A total of 424 highly-reliable z are measured in the 1.3<=z<=2.5 range, with total success rates of 60.7% in the H-band and 32.8% in the K-band. The newly determined z fill 55% of high and 35% of lower priority empty SOM grid cells. We measured Halpha fluxes in a 1."2 radius aperture from the spectra of the spectroscopically confirmed galaxies and converted them into star formation rates. In addition, we performed an SED fitting analysis on the same sample in order to derive stellar masses, E(B-V), total magnitudes, and SFRs. We combine the results obtained from the spectra with those derived via SED fitting, and we show that the spectroscopic failures come from either weakly star-forming galaxies (at z<1.7, i.e. in the H-band) or low S/N spectra (in the K-band) of z>2 galaxies.

The Probability Density Function (PDF) provides an estimate of the photometric redshift (zphot) prediction error. It is crucial for current and future sky surveys, characterized by strict requirements on the zphot precision, reliability and completeness. The present work stands on the assumption that properly defined rejection criteria, capable of identifying and rejecting potential outliers, can increase the precision of zphot estimates and of their cumulative PDF, without sacrificing much in terms of completeness of the sample. We provide a way to assess rejection through proper cuts on the shape descriptors of a PDF, such as the width and the height of the maximum PDF's peak. In this work we tested these rejection criteria to galaxies with photometry extracted from the Kilo Degree Survey (KiDS) ESO Data Release 4, proving that such approach could lead to significant improvements to the zphot quality: e.g., for the clipped sample showing the best trade-off between precision and completeness, we achieve a reduction in outliers fraction of $\simeq 75\%$ and an improvement of $\simeq 6\%$ for NMAD, with respect to the original data set, preserving the $\simeq 93\%$ of its content.

Astronomy has entered the multi-messenger data era and Machine Learning has found widespread use in a large variety of applications. The exploitation of synoptic (multi-band and multi-epoch) surveys, like LSST (Large Synoptic Survey Telescope), requires an extensive use of automatic methods for data processing and interpretation. With data volumes in the petabyte domain, the discrimination of time-critical information has already exceeded the capabilities of human operators and crowds of scientists have extreme difficulty to manage such amounts of data in multi-dimensional domains. This work is focused on an analysis of critical aspects related to the approach, based on Machine Learning, to variable sky sources classification, with special care to the various types of Supernovae, one of the most important subjects of Time Domain Astronomy, due to their crucial role in Cosmology. The work is based on a test campaign performed on simulated data. The classification was carried out by comparing the performances among several Machine Learning algorithms on statistical parameters extracted from the light curves. The results make in evidence some critical aspects related to the data quality and their parameter space characterization, propaedeutic to the preparation of processing machinery for the real data exploitation in the incoming decade.

Weak lensing, which is the deflection of light by matter along the line of sight, has proven to be an efficient method for constraining models of structure formation and reveal the nature of dark energy. So far, most weak-lensing studies have focused on the shear field that can be measured directly from the ellipticity of background galaxies. However, within the context of forthcoming full-sky weak-lensing surveys such as Euclid, convergence maps (mass maps) offer an important advantage over shear fields in terms of cosmological exploitation. While it carry the same information, the lensing signal is more compressed in the convergence maps than in the shear field. This simplifies otherwise computationally expensive analyses, for instance, non-Gaussianity studies. However, the inversion of the non-local shear field requires accurate control of systematic effects caused by holes in the data field, field borders, shape noise, and the fact that the shear is not a direct observable (reduced shear). We present the two mass-inversion methods that are included in the official Euclid data-processing pipeline: the standard Kaiser & Squires method (KS), and a new mass-inversion method (KS+) that aims to reduce the information loss during the mass inversion. This new method is based on the KS method and includes corrections for mass-mapping systematic effects. The results of the KS+ method are compared to the original implementation of the KS method in its simplest form, using the Euclid Flagship mock galaxy catalogue. In particular, we estimate the quality of the reconstruction by comparing the two-point correlation functions and third- and fourth- order moments obtained from shear and convergence maps, and we analyse each systematic effect independently and simultaneously...

Relic galaxies are thought to be the progenitors of high-redshift red nuggets that for some reason missed the channels of size growth and evolved passively and undisturbed since the first star formation burst (at $z>2$). These local ultracompact old galaxies are unique laboratories for studying the star formation processes at high redshift and thus the early stage of galaxy formation scenarios. Counterintuitively, theoretical and observational studies indicate that relics are more common in denser environments, where merging events predominate. To verify this scenario, we compared the number counts of a sample of ultracompact massive galaxies (UCMGs) selected within the third data release of the Kilo Degree Survey, that is, systems with sizes $R_{\rm e} < 1.5 \, \rm kpc$ and stellar masses $M_{\rm \star} > 8 \times 10^{10}\, \rm M_{\odot}$, with the number counts of galaxies with the same masses but normal sizes in field and cluster environments. Based on their optical and near-infrared colors, these UCMGs are likely to be mainly old, and hence representative of the relic population. We find that both UCMGs and normal-size galaxies are more abundant in clusters and their relative fraction depends only mildly on the global environment, with denser environments penalizing the survival of relics. Hence, UCMGs (and likely relics overall) are not special because of the environment effect on their nurture, but rather they are just a product of the stochasticity of the merging processes regardless of the global environment in which they live.

Every field of Science is undergoing unprecedented changes in the discovery process, and Astronomy has been a main player in this transition since the beginning. The ongoing and future large and complex multi-messenger sky surveys impose a wide exploiting of robust and efficient automated methods to classify the observed structures and to detect and characterize peculiar and unexpected sources. We performed a preliminary experiment on KiDS DR4 data, by applying to the problem of anomaly detection two different unsupervised machine learning algorithms, considered as potentially promising methods to detect peculiar sources, a Disentangled Convolutional Autoencoder and an Unsupervised Random Forest. The former method, working directly on images, is considered potentially able to identify peculiar objects like interacting galaxies and gravitational lenses. The latter instead, working on catalogue data, could identify objects with unusual values of magnitudes and colours, which in turn could indicate the presence of singularities.

Among the many challenges posed by the huge data volumes produced by the new generation of astronomical instruments there is also the search for rare and peculiar objects. Unsupervised outlier detection algorithms may provide a viable solution. In this work we compare the performances of six methods: the Local Outlier Factor, Isolation Forest, k-means clustering, a measure of novelty, and both a normal and a convolutional autoencoder. These methods were applied to data extracted from SDSS stripe 82. After discussing the sensitivity of each method to its own set of hyperparameters, we combine the results from each method to rank the objects and produce a final list of outliers.

DEAP-3600 is a single-phase liquid argon detector aiming to directly detect Weakly Interacting Massive Particles (WIMPs), located at SNOLAB (Sudbury, Canada). After analyzing data taken during the first year of operation, a null result was used to place an upper bound on the WIMP-nucleon spin-independent, isoscalar cross section. This study reinterprets this result within a Non-Relativistic Effective Field Theory framework, and further examines how various possible substructures in the local dark matter halo may affect these constraints. Such substructures are hinted at by kinematic structures in the local stellar distribution observed by the Gaia satellite and other recent astronomical surveys. These include the Gaia Sausage (or Enceladus), as well as a number of distinct streams identified in recent studies. Limits are presented for the coupling strength of the effective contact interaction operators $\mathcal{O}_1$, $\mathcal{O}_3$, $\mathcal{O}_5$, $\mathcal{O}_8$, and $\mathcal{O}_{11}$, considering isoscalar, isovector, and xenonphobic scenarios, as well as the specific operators corresponding to millicharge, magnetic dipole, electric dipole, and anapole interactions. The effects of halo substructures on each of these operators are explored as well, showing that the $\mathcal{O}_5$ and $\mathcal{O}_8$ operators are particularly sensitive to the velocity distribution, even at dark matter masses above 100 GeV/$c^2$.

The data from the Euclid mission will enable the measurement of the photometric redshifts, angular positions, and weak lensing shapes for over a billion galaxies. This large dataset will allow for cosmological analyses using the angular clustering of galaxies and cosmic shear. The cross-correlation (XC) between these probes can tighten constraints and it is therefore important to quantify their impact for Euclid. In this study we carefully quantify the impact of XC not only on the final parameter constraints for different cosmological models, but also on the nuisance parameters. In particular, we aim at understanding the amount of additional information that XC can provide for parameters encoding systematic effects, such as galaxy bias or intrinsic alignments (IA). We follow the formalism presented in Euclid Collaboration: Blanchard et al. (2019) and make use of the codes validated therein. We show that XC improves the dark energy Figure of Merit (FoM) by a factor $\sim 5$, whilst it also reduces the uncertainties on galaxy bias by $\sim 17\%$ and the uncertainties on IA by a factor $\sim 4$. We observe that the role of XC on the final parameter constraints is qualitatively the same irrespective of the galaxy bias model used. We also show that XC can help in distinguishing between different IA models, and that if IA terms are neglected then this can lead to significant biases on the cosmological parameters. We find that the XC terms are necessary to extract the full information content from the data in future analyses. They help in better constraining the cosmological model, and lead to a better understanding of the systematic effects that contaminate these probes. Furthermore, we find that XC helps in constraining the mean of the photometric-redshift distributions, but it requires a more precise knowledge of this mean in order not to degrade the final FoM. [Abridged]

Proportional electroluminescence (EL) in noble gases is used in two-phase detectors for dark matter search to record (in the gas phase) the ionization signal induced by particle scattering in the liquid phase. The "standard" EL mechanism is considered to be due to noble gas excimer emission in the vacuum ultraviolet (VUV). In addition to it, there are two alternative mechanisms, producing light in the visible and near infrared (NIR) range. The first is due to bremsstrahlung of electrons scattered on neutral atoms (so-called neutral bremsstrahlung, NBrS). The second, responsible for electron avalanche scintillations in the NIR at higher electric fields, is due to transitions between excited atomic states. In this work, we have for the first time demonstrated two alternative techniques of the optical readout of two-phase argon detectors, in the visible and NIR range, using a SiPM matrix and electroluminescence due to either neutral bremsstrahlung or avalanche scintillations effects. The amplitude yield and position resolution were measured for these readout techniques, which allowed to assess the detection threshold for electron and nuclear recoils in two-phase argon detectors for dark matter search. It should be emphasized that this is the first practical application of the NBrS effect in detection science.

Stage IV weak lensing experiments will offer more than an order of magnitude leap in precision. We must therefore ensure that our analyses remain accurate in this new era. Accordingly, previously ignored systematic effects must be addressed. In this work, we evaluate the impact of the reduced shear approximation and magnification bias, on the information obtained from the angular power spectrum. To first-order, the statistics of reduced shear, a combination of shear and convergence, are taken to be equal to those of shear. However, this approximation can induce a bias in the cosmological parameters that can no longer be neglected. A separate bias arises from the statistics of shear being altered by the preferential selection of galaxies and the dilution of their surface densities, in high-magnification regions. The corrections for these systematic effects take similar forms, allowing them to be treated together. We calculated the impact of neglecting these effects on the cosmological parameters that would be determined from Euclid, using cosmic shear tomography. To do so, we employed the Fisher matrix formalism, and included the impact of the super-sample covariance. We also demonstrate how the reduced shear correction can be calculated using a lognormal field forward modelling approach. These effects cause significant biases in Omega_m, sigma_8, n_s, Omega_DE, w_0, and w_a of -0.53 sigma, 0.43 sigma, -0.34 sigma, 1.36 sigma, -0.68 sigma, and 1.21 sigma, respectively. We then show that these lensing biases interact with another systematic: the intrinsic alignment of galaxies. Accordingly, we develop the formalism for an intrinsic alignment-enhanced lensing bias correction. Applying this to Euclid, we find that the additional terms introduced by this correction are sub-dominant.

The Euclid mission will observe well over a billion galaxies out to $z\sim6$ and beyond. This will offer an unrivalled opportunity to investigate several key questions for understanding galaxy formation and evolution. The first step for many of these studies will be the selection of a sample of quiescent and star-forming galaxies, as is often done in the literature by using well known colour techniques such as the `UVJ' diagram. However, given the limited number of filters available for the Euclid telescope, the recovery of such rest-frame colours will be challenging. We therefore investigate the use of observed Euclid colours, on their own and together with ground-based u-band observations, for selecting quiescent and star-forming galaxies. The most efficient colour combination, among the ones tested in this work, consists of the (u-VIS) and (VIS-J) colours. We find that this combination allows users to select a sample of quiescent galaxies complete to above $\sim70\%$ and with less than 15$\%$ contamination at redshifts in the range $0.75<z<1$. For galaxies at high-z or without the u-band complementary observations, the (VIS-Y) and (J-H) colours represent a valid alternative, with $>65\%$ completeness level and contamination below 20$\%$ at $1<z<2$ for finding quiescent galaxies. In comparison, the sample of quiescent galaxies selected with the traditional UVJ technique is only $\sim20\%$ complete at $z<3$, when recovering the rest-frame colours using mock Euclid observations. This shows that our new methodology is the most suitable one when only Euclid bands, along with u-band imaging, are available.

Ultra-compact massive galaxies UCMGs, i.e. galaxies with stellar masses $M_{*} > 8 \times 10^{10} M_{\odot}$ and effective radii $R_{e} < 1.5$ kpc, are very rare systems, in particular at low and intermediate redshifts. Their origin as well as their number density across cosmic time are still under scrutiny, especially because of the paucity of spectroscopically confirmed samples. We have started a systematic census of UCMG candidates within the ESO Kilo Degree Survey, together with a large spectroscopic follow-up campaign to build the largest possible sample of confirmed UCMGs. This is the third paper of the series and the second based on the spectroscopic follow-up program. Here, we present photometrical and structural parameters of 33 new candidates at redshifts $0.15 \lesssim z \lesssim 0.5$ and confirm 19 of them as UCMGs, based on their nominal spectroscopically inferred $M_{*}$ and $R_{e}$. This corresponds to a success rate of $\sim 58\%$, nicely consistent with our previous findings. The addition of these 19 newly confirmed objects, allows us to fully assess the systematics on the system selection, and finally reduce the number density uncertainties. Moreover, putting together the results from our current and past observational campaigns and some literature data, we build the largest sample of UCMGs ever collected, comprising 92 spectroscopically confirmed objects at $0.1 \lesssim z \lesssim 0.5$. This number raises to 116, allowing for a $3\sigma$ tolerance on the $M_{*}$ and $R_{e}$ thresholds for the UCMG definition. For all these galaxies we have estimated the velocity dispersion values at the effective radii which have been used to derive a preliminary mass-velocity dispersion correlation.

Large liquid argon detectors offer one of the best avenues for the detection of galactic weakly interacting massive particles (WIMPs) via their scattering on atomic nuclei. The liquid argon target allows exquisite discrimination between nuclear and electron recoil signals via pulse-shape discrimination of the scintillation signals. Atmospheric argon (AAr), however, has a naturally occurring radioactive isotope, $^{39}$Ar, a $\beta$ emitter of cosmogenic origin. For large detectors, the atmospheric $^{39}$Ar activity poses pile-up concerns. The use of argon extracted from underground wells, deprived of $^{39}$Ar, is key to the physics potential of these experiments. The DarkSide-20k dark matter search experiment will operate a dual-phase time projection chamber with 50 tonnes of radio-pure underground argon (UAr), that was shown to be depleted of $^{39}$Ar with respect to AAr by a factor larger than 1400. Assessing the $^{39}$Ar content of the UAr during extraction is crucial for the success of DarkSide-20k, as well as for future experiments of the Global Argon Dark Matter Collaboration (GADMC). This will be carried out by the DArT in ArDM experiment, a small chamber made with extremely radio-pure materials that will be placed at the centre of the ArDM detector, in the Canfranc Underground Laboratory (LSC) in Spain. The ArDM LAr volume acts as an active veto for background radioactivity, mostly $\gamma$-rays from the ArDM detector materials and the surrounding rock. This article describes the DArT in ArDM project, including the chamber design and construction, and reviews the background required to achieve the expected performance of the detector.

Many scientific investigations of photometric galaxy surveys require redshift estimates, whose uncertainty properties are best encapsulated by photometric redshift (photo-z) posterior probability density functions (PDFs). A plethora of photo-z PDF estimation methodologies abound, producing discrepant results with no consensus on a preferred approach. We present the results of a comprehensive experiment comparing twelve photo-z algorithms applied to mock data produced for the Large Synoptic Survey Telescope (LSST) Dark Energy Science Collaboration (DESC). By supplying perfect prior information, in the form of the complete template library and a representative training set as inputs to each code, we demonstrate the impact of the assumptions underlying each technique on the output photo-z PDFs. In the absence of a notion of true, unbiased photo-z PDFs, we evaluate and interpret multiple metrics of the ensemble properties of the derived photo-z PDFs as well as traditional reductions to photo-z point estimates. We report systematic biases and overall over/under-breadth of the photo-z PDFs of many popular codes, which may indicate avenues for improvement in the algorithms or implementations. Furthermore, we raise attention to the limitations of established metrics for assessing photo-z PDF accuracy; though we identify the conditional density estimate (CDE) loss as a promising metric of photo-z PDF performance in the case where true redshifts are available but true photo-z PDFs are not, we emphasize the need for science-specific performance metrics.

The recent data collected by {\it Herschel} have confirmed that interstellar structures with filamentary shape are ubiquitously present in the Milky Way. Filaments are thought to be formed by several physical mechanisms acting from the large Galactic scales down to the sub-pc fractions of molecular clouds, and they might represent a possible link between star formation and the large-scale structure of the Galaxy. In order to study this potential link, a statistically significant sample of filaments spread throughout the Galaxy is required. In this work we present the first catalogue of $32,059$ candidate filaments automatically identified in the Hi-GAL survey of the entire Galactic Plane. For these objects we determined morphological (length, $l^{a}$, and geometrical shape) and physical (average column density, $N_{\rm H_{2}}$, and average temperature, $T$) properties. We identified filaments with a wide range of properties: 2$'$\,$\leq l^{a}\leq$\, 100$'$, $10^{20} \leq N_{\rm H_{2}} \leq 10^{23}$\,cm$^{-2}$ and $10 \leq T\leq$ 35\,K. We discuss their association with the Hi-GAL compact sources, finding that the most tenuous (and stable) structures do not host any major condensation and we also assign a distance to $\sim 18,400$ filaments for which we determine mass, physical size, stability conditions and Galactic distribution. When compared to the spiral arms structure, we find no significant difference between the physical properties of on-arm and inter-arm filaments. We compared our sample with previous studies, finding that our Hi-GAL filament catalogue represents a significant extension in terms of Galactic coverage and sensitivity. This catalogue represents an unique and important tool for future studies devoted to understanding the filament life-cycle.

We provide predictions of the yield of $7<z<9$ quasars from the Euclid wide survey, updating the calculation presented in the Euclid Red Book in several ways. We account for revisions to the Euclid near-infrared filter wavelengths; we adopt steeper rates of decline of the quasar luminosity function (QLF; $\Phi$) with redshift, $\Phi\propto10^{k(z-6)}$, $k=-0.72$, and a further steeper rate of decline, $k=-0.92$; we use better models of the contaminating populations (MLT dwarfs and compact early-type galaxies); and we use an improved Bayesian selection method, compared to the colour cuts used for the Red Book calculation, allowing the identification of fainter quasars, down to $J_{AB}\sim23$. Quasars at $z>8$ may be selected from Euclid $OYJH$ photometry alone, but selection over the redshift interval $7<z<8$ is greatly improved by the addition of $z$-band data from, e.g., Pan-STARRS and LSST. We calculate predicted quasar yields for the assumed values of the rate of decline of the QLF beyond $z=6$. For the case that the decline of the QLF accelerates beyond $z=6$, with $k=-0.92$, Euclid should nevertheless find over 100 quasars with $7.0<z<7.5$, and $\sim25$ quasars beyond the current record of $z=7.5$, including $\sim8$ beyond $z=8.0$. The first Euclid quasars at $z>7.5$ should be found in the DR1 data release, expected in 2024. It will be possible to determine the bright-end slope of the QLF, $7<z<8$, $M_{1450}<-25$, using 8m class telescopes to confirm candidates, but follow-up with JWST or E-ELT will be required to measure the faint-end slope. Contamination of the candidate lists is predicted to be modest even at $J_{AB}\sim23$. The precision with which $k$ can be determined over $7<z<8$ depends on the value of $k$, but assuming $k=-0.72$ it can be measured to a 1 sigma uncertainty of 0.07.

We report measurements of the charged daughter fraction of $^{218}$Po as a result of the $^{222}$Rn alpha decay, and the mobility of $^{218}$Po$^+$ ions, using radon-polonium coincidences from the $^{238}$U chain identified in 532 live-days of DarkSide-50 WIMP-search data. The fraction of $^{218}$Po that is charged is found to be 0.37$\pm$0.03 and the mobility of $^{218}$Po$^+$ is (8.6$\pm$0.1)$\times$10$^{-4}$$\frac{\text{cm}^2}{\text{Vs}}$.

Our aim is to quantify the impact of systematic effects on the inference of cosmological parameters from cosmic shear. We present an end-to-end approach that introduces sources of bias in a modelled weak lensing survey on a galaxy-by-galaxy level. Residual biases are propagated through a pipeline from galaxy properties (one end) through to cosmic shear power spectra and cosmological parameter estimates (the other end), to quantify how imperfect knowledge of the pipeline changes the maximum likelihood values of dark energy parameters. We quantify the impact of an imperfect correction for charge transfer inefficiency (CTI) and modelling uncertainties of the point spread function (PSF) for Euclid, and find that the biases introduced can be corrected to acceptable levels.

The Euclid space telescope will measure the shapes and redshifts of galaxies to reconstruct the expansion history of the Universe and the growth of cosmic structures. Estimation of the expected performance of the experiment, in terms of predicted constraints on cosmological parameters, has so far relied on different methodologies and numerical implementations, developed for different observational probes and for their combination. In this paper we present validated forecasts, that combine both theoretical and observational expertise for different cosmological probes. This is presented to provide the community with reliable numerical codes and methods for Euclid cosmological forecasts. We describe in detail the methodology adopted for Fisher matrix forecasts, applied to galaxy clustering, weak lensing and their combination. We estimate the required accuracy for Euclid forecasts and outline a methodology for their development. We then compare and improve different numerical implementations, reaching uncertainties on the errors of cosmological parameters that are less than the required precision in all cases. Furthermore, we provide details on the validated implementations that can be used by the reader to validate their own codes if required. We present new cosmological forecasts for Euclid. We find that results depend on the specific cosmological model and remaining freedom in each setup, i.e. flat or non-flat spatial cosmologies, or different cuts at nonlinear scales. The validated numerical implementations can now be reliably used for any setup. We present results for an optimistic and a pessimistic choice of such settings. We demonstrate that the impact of cross-correlations is particularly relevant for models beyond a cosmological constant and may allow us to increase the dark energy Figure of Merit by at least a factor of three.

In the last years, Astroinformatics has become a well defined paradigm for many fields of Astronomy. In this work we demonstrate the potential of a multidisciplinary approach to identify globular clusters (GCs) in the Fornax cluster of galaxies taking advantage of multi-band photometry produced by the VLT Survey Telescope using automatic self-adaptive methodologies. The data analyzed in this work consist of deep, multi-band, partially overlapping images centered on the core of the Fornax cluster. In this work we use a Neural-Gas model, a pure clustering machine learning methodology, to approach the GC detection, while a novel feature selection method ($\Phi$LAB) is exploited to perform the parameter space analysis and optimization. We demonstrate that the use of an Astroinformatics based methodology is able to provide GC samples that are comparable, in terms of purity and completeness with those obtained using single band HST data (Brescia et al. 2012) and two approaches based respectively on a morpho-photometric (Cantiello et al. 2018b) and a PCA analysis (D'Abrusco et al. 2015) using the same data discussed in this work.

With the launch of eROSITA (extended Roentgen Survey with an Imaging Telescope Array), successfully occurred on 2019 July 13, we are facing the challenge of computing reliable photometric redshifts for 3 million of active galactic nuclei (AGNs) over the entire sky, having available only patchy and inhomogeneous ancillary data. While we have a good understanding of the photo-z quality obtainable for AGN using spectral energy distribution (SED)-fitting technique, we tested the capability of machine learning (ML), usually reliable in computing photo-z for QSO in wide and shallow areas with rich spectroscopic samples. Using MLPQNA as example of ML, we computed photo-z for the X-ray-selected sources in Stripe 82X, using the publicly available photometric and spectroscopic catalogues. Stripe 82X is at least as deep as eROSITA will be and wide enough to include also rare and bright AGNs. In addition, the availability of ancillary data mimics what can be available in the whole sky. We found that when optical, and near- and mid-infrared data are available, ML and SED fitting perform comparably well in terms of overall accuracy, realistic redshift probability density functions, and fraction of outliers, although they are not the same for the two methods. The results could further improve if the photometry available is accurate and including morphological information. Assuming that we can gather sufficient spectroscopy to build a representative training sample, with the current photometry coverage we can obtain reliable photo-z for a large fraction of sources in the Southern hemisphere well before the spectroscopic follow-up, thus timely enabling the eROSITA science return. The photo-z catalogue is released here.

DEAP-3600 is a single-phase liquid argon (LAr) direct-detection dark matter experiment, operating 2 km underground at SNOLAB (Sudbury, Canada). The detector consists of 3279 kg of LAr contained in a spherical acrylic vessel. This paper reports on the analysis of a 758 tonne\cdot day exposure taken over a period of 231 live-days during the first year of operation. No candidate signal events are observed in the WIMP-search region of interest, which results in the leading limit on the WIMP-nucleon spin-independent cross section on a LAr target of $3.9\times10^{-45}$ cm$^{2}$ ($1.5\times10^{-44}$ cm$^{2}$) for a 100 GeV/c$^{2}$ (1 TeV/c$^{2}$) WIMP mass at 90\% C. L. In addition to a detailed background model, this analysis demonstrates the best pulse-shape discrimination in LAr at threshold, employs a Bayesian photoelectron-counting technique to improve the energy resolution and discrimination efficiency, and utilizes two position reconstruction algorithms based on PMT charge and photon arrival times.

Star Formation Rates or SFRs are crucial to constrain theories of galaxy formation and evolution. SFRs are usually estimated via spectroscopic observations requiring large amounts of telescope time. We explore an alternative approach based on the photometric estimation of global SFRs for large samples of galaxies, by using methods such as automatic parameter space optimisation, and supervised Machine Learning models. We demonstrate that, with such approach, accurate multi-band photometry allows to estimate reliable SFRs. We also investigate how the use of photometric rather than spectroscopic redshifts, affects the accuracy of derived global SFRs. Finally, we provide a publicly available catalogue of SFRs for more than 27 million galaxies extracted from the Sloan Digital Sky survey Data Release 7. The catalogue is available through the Vizier facility at the following link ftp://cdsarc.u-strasbg.fr/pub/cats/J/MNRAS/486/1377.

The DEAP-3600 experiment is searching for WIMP dark matter with a 3.3 tonne single phase liquid argon (LAr) target, located 2.1 km underground at SNOLAB. The experimental signature of dark matter interactions is keV-scale $^{40}$Ar nuclear recoils (NR) producing 128 nm LAr scintillation photons observed by PMTs. The largest backgrounds in DEAP-3600 are electronic recoils (ER) induced by $\beta$ and $\gamma$-rays originating from internal and external radioactivity in the detector material. A background model of the ER interactions in DEAP-3600 was developed and is described in this work. The model is based on several components which are expected from radioisotopes in the LAr, from ex-situ material assay measurements, and from dedicated independent in-situ analyses. This prior information is used in a Bayesian fit of the ER components to a 247.2 d dataset to model the radioactivity in the surrounding detector materials. While excellent discrimination between ERs and NRs is reached with pulse shape discrimination, utilizing the large difference between fast and slow components of LAr scintillation light, detailed knowledge of the ER background and activity of detector components, sets valuable constraints on other key types of backgrounds in the detector: neutrons and alphas. In addition, the activity of $^{42}$Ar in LAr in DEAP-3600 is determined by measuring the daughter decay of $^{42}$K. This cosmogenically activated trace isotope is a relevant background at higher energies for other rare event searches using atmospheric argon e.g. DarkSide-20k, GERDA or LEGEND. The specific activity of $^{42}$Ar in the atmosphere is found to be $40.4 \pm 5.9$ $\mu$Bq/kg of argon.

Future radio surveys will generate catalogues of tens of millions of radio sources, for which redshift estimates will be essential to achieve many of the science goals. However, spectroscopic data will be available for only a small fraction of these sources, and in most cases even the optical and infrared photometry will be of limited quality. Furthermore, radio sources tend to be at higher redshift than most optical sources and so a significant fraction of radio sources hosts differ from those for which most photometric redshift templates are designed. We therefore need to develop new techniques for estimating the redshifts of radio sources. As a starting point in this process, we evaluate a number of machine-learning techniques for estimating redshift, together with a conventional template-fitting technique. We pay special attention to how the performance is affected by the incompleteness of the training sample and by sparseness of the parameter space or by limited availability of ancillary multi-wavelength data. As expected, we find that the quality of the photometric-redshift degrades as the quality of the photometry decreases, but that even with the limited quality of photometry available for all sky-surveys, useful redshift information is available for the majority of sources, particularly at low redshift. We find that a template-fitting technique performs best with high-quality and almost complete multi-band photometry, especially if radio sources that are also X-ray emitting are treated separately. When we reduced the quality of photometry to match that available for the EMU all-sky radio survey, the quality of the template-fitting degraded and became comparable to some of the machine learning methods. Machine learning techniques currently perform better at low redshift than at high redshift, because of incompleteness of the currently available training data at high redshifts.

Global Stellar Formation Rates or SFRs are crucial to constrain theories of galaxy formation and evolution. SFR's are usually estimated via spectroscopic observations which require too much previous telescope time and therefore cannot match the needs of modern precision cosmology. We therefore propose a novel method to estimate SFRs for large samples of galaxies using a variety of supervised ML models.

The DarkSide-50 direct-detection dark matter experiment is a dual-phase argon time projection chamber operating at Laboratori Nazionali del Gran Sasso. This paper reports on the blind analysis of a (16,660+-270) kg d exposure using a target of low-radioactivity argon extracted from underground sources. We find no events in the dark matter selection box and set a 90% C.L. upper limit on the dark matter-nucleon spin-independent cross section of 1.14E-44 cm^2 (3.78E-44 cm^2, 3.43E-43 cm^2) for a WIMP mass of 100 GeV/c^2 (1 TeV/c^2, 10 TeV/c^2).

Despite the high accuracy of photometric redshifts (zphot) derived using Machine Learning (ML) methods, the quantification of errors through reliable and accurate Probability Density Functions (PDFs) is still an open problem. First, because it is difficult to accurately assess the contribution from different sources of errors, namely internal to the method itself and from the photometric features defining the available parameter space. Second, because the problem of defining a robust statistical method, always able to quantify and qualify the PDF estimation validity, is still an open issue. We present a comparison among PDFs obtained using three different methods on the same data set: two ML techniques, METAPHOR (Machine-learning Estimation Tool for Accurate PHOtometric Redshifts) and ANNz2, plus the spectral energy distribution template fitting method, BPZ. The photometric data were extracted from the KiDS (Kilo Degree Survey) ESO Data Release 3, while the spectroscopy was obtained from the GAMA (Galaxy and Mass Assembly) Data Release 2. The statistical evaluation of both individual and stacked PDFs was done through quantitative and qualitative estimators, including a dummy PDF, useful to verify whether different statistical estimators can correctly assess PDF quality. We conclude that, in order to quantify the reliability and accuracy of any zphot PDF method, a combined set of statistical estimators is required.

We present new constraints on sub-GeV dark matter particles scattering off electrons in argon based on an analysis of ionization signal data from the DarkSide-50 detector.

We present results from an ongoing investigation using the Kilo Degree Survey (KiDS) on the VLT Survey Telescope (VST) to provide a census of ultra-compact massive galaxies (UCMGs), defined as galaxies with stellar masses $M_{\rm \star} > 8 \times 10^{10} \rm M_{\odot}$ and effective radii $R_{\rm e} < 1.5\,\rm kpc$. UCMGs, which are expected to have undergone very few merger events, provide a unique view on the accretion history of the most massive galaxies in the Universe. Over an effective sky area of nearly 330 square degrees, we select UCMG candidates from KiDS multi-colour images, which provide high quality structural parameters, photometric redshifts and stellar masses. Our sample of $\sim 1000$ photometrically selected UCMGs at $z < 0.5$ represents the largest sample of UCMG candidates assembled to date over the largest sky area. In this paper we present the first effort to obtain their redshifts using different facilities, starting with first results for 28 candidates with redshifts $z < 0.5$, obtained at NTT and TNG telescopes. We confirmed, as bona fide UCMGs, 19 out of the 28 candidates with new redshifts. A further 46 UCMG candidates are confirmed with literature spectroscopic redshifts (35 at $z < 0.5$), bringing the final cumulative sample of spectroscopically-confirmed lower-z UCMGs to 54 galaxies, which is the largest sample at redshifts below $0.5$. We use these spectroscopic redshifts to quantify systematic errors in our photometric selection, and use these to correct our UCMG number counts. We finally compare the results to independent datasets and simulations.

We present the results of a search for dark matter WIMPs in the mass range below 20 GeV/c^2 using a target of low-radioactivity argon. The data were obtained using the DarkSide-50 apparatus at Laboratori Nazionali del Gran Sasso (LNGS). The analysis is based on the ionization signal, for which the DarkSide-50 time projection chamber is fully efficient at 0.1 keVee. The observed rate in the detector at 0.5 keVee is about 1.5 events/keVee/kg/day and is almost entirely accounted for by known background sources. We obtain a 90% C.L. exclusion limit above 1.8 GeV/c^2 for the spin-independent cross section of dark matter WIMPs on nucleons, extending the exclusion region for dark matter below previous limits in the range 1.8-6 GeV/c^2.

We present a visual analytics tool, based on the VisIVO suite, to exploit a combination of all new-generation surveys of the Galactic Plane to study the star formation process of the Milky Way. The tool has been developed within the VIALACTEA project, founded by the 7th Framework Programme of the European Union, that creates a common forum for the major new-generation surveys of the Milky Way Galactic Plane from the near infrared to the radio, both in thermal continuum and molecular lines. Massive volumes of data are produced by space missions and ground-based facilities and the ability to collect and store them is increasing at a higher pace than the ability to analyze them. This gap leads to new challenges in the analysis pipeline to discover information contained in the data. Visual analytics focuses on handling these massive, heterogeneous, and dynamic volumes of information accessing the data previously processed by data mining algorithms and advanced analysis techniques with highly interactive visual interfaces offering scientists the opportunity for in-depth understanding of massive, noisy, and high-dimensional data.

We have obtained structural parameters of about 340,000 galaxies from the Kilo Degree Survey (KiDS) in 153 square degrees of data release 1, 2 and 3. We have performed a seeing convolved 2D single S\'ersic fit to the galaxy images in the 4 photometric bands (u, g, r, i) observed by KiDS, by selecting high signal-to-noise ratio (S/N > 50) systems in every bands. We have classified galaxies as spheroids and disc-dominated by combining their spectral energy distribution properties and their S\'ersic index. Using photometric redshifts derived from a machine learning technique, we have determined the evolution of the effective radius, \Re\ and stellar mass, \mst, versus redshift, for both mass complete samples of spheroids and disc-dominated galaxies up to z ~ 0.6. Our results show a significant evolution of the structural quantities at intermediate redshift for the massive spheroids ($\mbox{Log}\ M_*/M_\odot>11$, Chabrier IMF), while almost no evolution has found for less massive ones ($\mbox{Log}\ M_*/M_\odot < 11$). On the other hand, disc dominated systems show a milder evolution in the less massive systems ($\mbox{Log}\ M_*/M_\odot < 11$) and possibly no evolution of the more massive systems. These trends are generally consistent with predictions from hydrodynamical simulations and independent datasets out to redshift z ~ 0.6, although in some cases the scatter of the data is large to drive final conclusions. These results, based on 1/10 of the expected KiDS area, reinforce precedent finding based on smaller statistical samples and show the route toward more accurate results, expected with the the next survey releases.

Astronomy has entered the big data era and Machine Learning based methods have found widespread use in a large variety of astronomical applications. This is demonstrated by the recent huge increase in the number of publications making use of this new approach. The usage of machine learning methods, however is still far from trivial and many problems still need to be solved. Using the evaluation of photometric redshifts as a case study, we outline the main problems and some ongoing efforts to solve them.

We present a machine-learning photometric redshift analysis of the Kilo-Degree Survey Data Release 3, using two neural-network based techniques: ANNz2 and MLPQNA. Despite limited coverage of spectroscopic training sets, these ML codes provide photo-zs of quality comparable to, if not better than, those from the BPZ code, at least up to zphot<0.9 and r<23.5. At the bright end of r<20, where very complete spectroscopic data overlapping with KiDS are available, the performance of the ML photo-zs clearly surpasses that of BPZ, currently the primary photo-z method for KiDS. Using the Galaxy And Mass Assembly (GAMA) spectroscopic survey as calibration, we furthermore study how photo-zs improve for bright sources when photometric parameters additional to magnitudes are included in the photo-z derivation, as well as when VIKING and WISE infrared bands are added. While the fiducial four-band ugri setup gives a photo-z bias $\delta z=-2e-4$ and scatter $\sigma_z<0.022$ at mean z = 0.23, combining magnitudes, colours, and galaxy sizes reduces the scatter by ~7% and the bias by an order of magnitude. Once the ugri and IR magnitudes are joined into 12-band photometry spanning up to 12 $\mu$, the scatter decreases by more than 10% over the fiducial case. Finally, using the 12 bands together with optical colours and linear sizes gives $\delta z<4e-5$ and $\sigma_z<0.019$. This paper also serves as a reference for two public photo-z catalogues accompanying KiDS DR3, both obtained using the ANNz2 code. The first one, of general purpose, includes all the 39 million KiDS sources with four-band ugri measurements in DR3. The second dataset, optimized for low-redshift studies such as galaxy-galaxy lensing, is limited to r<20, and provides photo-zs of much better quality than in the full-depth case thanks to incorporating optical magnitudes, colours, and sizes in the GAMA-calibrated photo-z derivation.

The explosion of data in recent years has generated an increasing need for new analysis techniques in order to extract knowledge from massive datasets. Machine learning has proved particularly useful to perform this task. Fully automatized methods have recently gathered great popularity, even though those methods often lack physical interpretability. In contrast, feature based approaches can provide both well-performing models and understandable causalities with respect to the correlations found between features and physical processes. Efficient feature selection is an essential tool to boost the performance of machine learning models. In this work, we propose a forward selection method in order to compute, evaluate, and characterize better performing features for regression and classification problems. Given the importance of photometric redshift estimation, we adopt it as our case study. We synthetically created 4,520 features by combining magnitudes, errors, radii, and ellipticities of quasars, taken from the SDSS. We apply a forward selection process, a recursive method in which a huge number of feature sets is tested through a kNN algorithm, leading to a tree of feature sets. The branches of the tree are then used to perform experiments with the random forest, in order to validate the best set with an alternative model. We demonstrate that the sets of features determined with our approach improve the performances of the regression models significantly when compared to the performance of the classic features from the literature. The found features are unexpected and surprising, being very different from the classic features. Therefore, a method to interpret some of the found features in a physical context is presented. The methodology described here is very general and can be used to improve the performance of machine learning models for any regression or classification task.

Within scientific and real life problems, classification is a typical case of extremely complex tasks in data-driven scenarios, especially if approached with traditional techniques. Machine Learning supervised and unsupervised paradigms, providing self-adaptive and semi-automatic methods, are able to navigate into large volumes of data characterized by a multi-dimensional parameter space, thus representing an ideal method to disentangle classes of objects in a reliable and efficient way. In Astrophysics, the identification of candidate Globular Clusters through deep, wide-field, single band images, is one of such cases where self-adaptive methods demonstrated a high performance and reliability. Here we experimented some variants of the known Neural Gas model, exploring both supervised and unsupervised paradigms of Machine Learning for the classification of Globular Clusters. Main scope of this work was to verify the possibility to improve the computational efficiency of the methods to solve complex data-driven problems, by exploiting the parallel programming with GPU framework. By using the astrophysical playground, the goal was to scientifically validate such kind of models for further applications extended to other contexts.

In the last decade, astronomers have found a new type of supernova called `superluminous supernovae' (SLSNe) due to their high peak luminosity and long light-curves. These hydrogen-free explosions (SLSNe-I) can be seen to z~4 and therefore, offer the possibility of probing the distant Universe. We aim to investigate the possibility of detecting SLSNe-I using ESA's Euclid satellite, scheduled for launch in 2020. In particular, we study the Euclid Deep Survey (EDS) which will provide a unique combination of area, depth and cadence over the mission. We estimated the redshift distribution of Euclid SLSNe-I using the latest information on their rates and spectral energy distribution, as well as known Euclid instrument and survey parameters, including the cadence and depth of the EDS. We also applied a standardization method to the peak magnitudes to create a simulated Hubble diagram to explore possible cosmological constraints. We show that Euclid should detect approximately 140 high-quality SLSNe-I to z ~ 3.5 over the first five years of the mission (with an additional 70 if we lower our photometric classification criteria). This sample could revolutionize the study of SLSNe-I at z>1 and open up their use as probes of star-formation rates, galaxy populations, the interstellar and intergalactic medium. In addition, a sample of such SLSNe-I could improve constraints on a time-dependent dark energy equation-of-state, namely w(a), when combined with local SLSNe-I and the expected SN Ia sample from the Dark Energy Survey. We show that Euclid will observe hundreds of SLSNe-I for free. These luminous transients will be in the Euclid data-stream and we should prepare now to identify them as they offer a new probe of the high-redshift Universe for both astrophysics and cosmology.

In Astrophysics, the identification of candidate Globular Clusters through deep, wide-field, single band HST images, is a typical data analytics problem, where methods based on Machine Learning have revealed a high efficiency and reliability, demonstrating the capability to improve the traditional approaches. Here we experimented some variants of the known Neural Gas model, exploring both supervised and unsupervised paradigms of Machine Learning, on the classification of Globular Clusters, extracted from the NGC1399 HST data. Main focus of this work was to use a well-tested playground to scientifically validate such kind of models for further extended experiments in astrophysics and using other standard Machine Learning methods (for instance Random Forest and Multi Layer Perceptron neural network) for a comparison of performances in terms of purity and completeness.

Photometric redshifts (photo-z's) provide an alternative way to estimate the distances of large samples of galaxies and are therefore crucial to a large variety of cosmological problems. Among the various methods proposed over the years, supervised machine learning (ML) methods capable to interpolate the knowledge gained by means of spectroscopical data have proven to be very effective. METAPHOR (Machine-learning Estimation Tool for Accurate PHOtometric Redshifts) is a novel method designed to provide a reliable PDF (Probability density Function) of the error distribution of photometric redshifts predicted by ML methods. The method is implemented as a modular workflow, whose internal engine for photo-z estimation makes use of the MLPQNA neural network (Multi Layer Perceptron with Quasi Newton learning rule), with the possibility to easily replace the specific machine learning model chosen to predict photo-z's. After a short description of the software, we present a summary of results on public galaxy data (Sloan Digital Sky Survey - Data Release 9) and a comparison with a completely different method based on Spectral Energy Distribution (SED) template fitting.

Hi-GAL is a large-scale survey of the Galactic plane, performed with Herschel in five infrared continuum bands between 70 and 500 $\mu$m. We present a band-merged catalogue of spatially matched sources and their properties derived from fits to the spectral energy distributions (SEDs) and heliocentric distances, based on the photometric catalogs presented in Molinari et al. (2016a), covering the portion of Galactic plane $-71.0^{\circ}< \ell < 67.0^{\circ}$. The band-merged catalogue contains 100922 sources with a regular SED, 24584 of which show a 70 $\mu$m counterpart and are thus considered proto-stellar, while the remainder are considered starless. Thanks to this huge number of sources, we are able to carry out a preliminary analysis of early stages of star formation, identifying the conditions that characterise different evolutionary phases on a statistically significant basis. We calculate surface densities to investigate the gravitational stability of clumps and their potential to form massive stars. We also explore evolutionary status metrics such as the dust temperature, luminosity and bolometric temperature, finding that these are higher in proto-stellar sources compared to pre-stellar ones. The surface density of sources follows an increasing trend as they evolve from pre-stellar to proto-stellar, but then it is found to decrease again in the majority of the most evolved clumps. Finally, we study the physical parameters of sources with respect to Galactic longitude and the association with spiral arms, finding only minor or no differences between the average evolutionary status of sources in the fourth and first Galactic quadrants, or between "on-arm" and "inter-arm" positions.

The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products, adding further area, homogenized photometric calibration, photometric redshifts and weak lensing shear measurements to the first two releases. A dedicated pipeline embedded in the Astro-WISE information system is used for the production of the main release. Modifications with respect to earlier releases are described in detail. Photometric redshifts have been derived using both Bayesian template fitting, and machine-learning techniques. For the weak lensing measurements, optimized procedures based on the THELI data reduction and lensfit shear measurement packages are used. In DR3 stacked ugri images, weight maps, masks, and source lists for 292 new survey tiles (~300 sq.deg) are made available. The multi-band catalogue, including homogenized photometry and photometric redshifts, covers the combined DR1, DR2 and DR3 footprint of 440 survey tiles (447 sq.deg). Limiting magnitudes are typically 24.3, 25.1, 24.9, 23.8 (5 sigma in a 2 arcsec aperture) in ugri, respectively, and the typical r-band PSF size is less than 0.7 arcsec. The photometric homogenization scheme ensures accurate colors and an absolute calibration stable to ~2% for gri and ~3% in u. Separately released are a weak lensing shear catalogue and photometric redshifts based on two different machine-learning techniques.

We present the VST Optical Imaging of the CDFS and ES1 Fields (VOICE) Survey, a VST INAF Guaranteed Time program designed to provide optical coverage of two 4 deg$^2$ cosmic windows in the Southern hemisphere. VOICE provides the first, multi-band deep optical imaging of these sky regions, thus complementing and enhancing the rich legacy of longer-wavelength surveys with VISTA, Spitzer, Herschel and ATCA available in these areas and paving the way for upcoming observations with facilities such as the LSST, MeerKAT and the SKA. VOICE exploits VST's OmegaCAM optical imaging capabilities and completes the reduction of WFI data available within the ES1 fields as part of the ESO-Spitzer Imaging Extragalactic Survey (ESIS) program providing $ugri$ and $uBVR$ coverage of 4 and 4 deg$^2$ areas within the CDFS and ES1 field respectively. We present the survey's science rationale and observing strategy, the data reduction and multi-wavelength data fusion pipeline. Survey data products and their future updates will be released at http://www.mattiavaccari.net/voice/ and on CDS/VizieR.

We present METAPHOR (Machine-learning Estimation Tool for Accurate PHOtometric Redshifts), a method able to provide a reliable PDF for photometric galaxy redshifts estimated through empirical techniques. METAPHOR is a modular workflow, mainly based on the MLPQNA neural network as internal engine to derive photometric galaxy redshifts, but giving the possibility to easily replace MLPQNA with any other method to predict photo-z's and their PDF. We present here the results about a validation test of the workflow on the galaxies from SDSS-DR9, showing also the universality of the method by replacing MLPQNA with KNN and Random Forest models. The validation test include also a comparison with the PDF's derived from a traditional SED template fitting method (Le Phare).

In the current data-driven science era, it is needed that data analysis techniques has to quickly evolve to face with data whose dimensions has increased up to the Petabyte scale. In particular, being modern astrophysics based on multi-wavelength data organized into large catalogues, it is crucial that the astronomical catalog cross-matching methods, strongly dependant from the catalogues size, must ensure efficiency, reliability and scalability. Furthermore, multi-band data are archived and reduced in different ways, so that the resulting catalogues may differ each other in formats, resolution, data structure, etc, thus requiring the highest generality of cross-matching features. We present $C^{3}$ (Command-line Catalogue Cross-match), a multi-platform application designed to efficiently cross-match massive catalogues from modern surveys. Conceived as a stand-alone command-line process or a module within generic data reduction/analysis pipeline, it provides the maximum flexibility, in terms of portability, configuration, coordinates and cross-matching types, ensuring high performance capabilities by using a multi-core parallel processing paradigm and a sky partitioning algorithm.

In the modern galaxy surveys photometric redshifts play a central role in a broad range of studies, from gravitational lensing and dark matter distribution to galaxy evolution. Using a dataset of about 25,000 galaxies from the second data release of the Kilo Degree Survey (KiDS) we obtain photometric redshifts with five different methods: (i) Random forest, (ii) Multi Layer Perceptron with Quasi Newton Algorithm, (iii) Multi Layer Perceptron with an optimization network based on the Levenberg-Marquardt learning rule, (iv) the Bayesian Photometric Redshift model (or BPZ) and (v) a classical SED template fitting procedure (Le Phare). We show how SED fitting techniques could provide useful information on the galaxy spectral type which can be used to improve the capability of machine learning methods constraining systematic errors and reduce the occurrence of catastrophic outliers. We use such classification to train specialized regression estimators, by demonstrating that such hybrid approach, involving SED fitting and machine learning in a single collaborative framework, is capable to improve the overall prediction accuracy of photometric redshifts.

Euclid is a Europe-led cosmology space mission dedicated to a visible and near infrared survey of the entire extra-galactic sky. Its purpose is to deepen our knowledge of the dark content of our Universe. After an overview of the Euclid mission and science, this contribution describes how the community is getting organized to face the data analysis challenges, both in software development and in operational data processing matters. It ends with a more specific account of some of the main contributions of the Swiss Science Data Center (SDC-CH).

Photometric redshifts (photo-z's) are fundamental in galaxy surveys to address different topics, from gravitational lensing and dark matter distribution to galaxy evolution. The Kilo Degree Survey (KiDS), i.e. the ESO public survey on the VLT Survey Telescope (VST), provides the unprecedented opportunity to exploit a large galaxy dataset with an exceptional image quality and depth in the optical wavebands. Using a KiDS subset of about 25,000 galaxies with measured spectroscopic redshifts, we have derived photo-z's using i) three different empirical methods based on supervised machine learning, ii) the Bayesian Photometric Redshift model (or BPZ), and iii) a classical SED template fitting procedure (Le Phare). We confirm that, in the regions of the photometric parameter space properly sampled by the spectroscopic templates, machine learning methods provide better redshift estimates, with a lower scatter and a smaller fraction of outliers. SED fitting techniques, however, provide useful information on the galaxy spectral type which can be effectively used to constrain systematic errors and to better characterize potential catastrophic outliers. Such classification is then used to specialize the training of regression machine learning models, by demonstrating that a hybrid approach, involving SED fitting and machine learning in a single collaborative framework, can be effectively used to improve the accuracy of photo-z estimates.

Modern Astrophysics is based on multi-wavelength data organized into large and heterogeneous catalogues. Hence, the need for efficient, reliable and scalable catalogue cross-matching methods plays a crucial role in the era of the petabyte scale. Furthermore, multi-band data have often very different angular resolution, requiring the highest generality of cross-matching features, mainly in terms of region shape and resolution. In this work we present $C^{3}$ (Command-line Catalogue Cross-match), a multi-platform application designed to efficiently cross-match massive catalogues. It is based on a multi-core parallel processing paradigm and conceived to be executed as a stand-alone command-line process or integrated within any generic data reduction/analysis pipeline, providing the maximum flexibility to the end-user, in terms of portability, parameter configuration, catalogue formats, angular resolution, region shapes, coordinate units and cross-matching types. Using real data, extracted from public surveys, we discuss the cross-matching capabilities and computing time efficiency also through a direct comparison with some publicly available tools, chosen among the most used within the community, and representative of different interface paradigms. We verified that the $C^{3}$ tool has excellent capabilities to perform an efficient and reliable cross-matching between large datasets. Although the elliptical cross-match and the parametric handling of angular orientation and offset are known concepts in the astrophysical context, their availability in the presented command-line tool makes $C^{3}$ competitive in the context of public astronomical tools.

The most valuable asset of a space mission like Euclid are the data. Due to their huge volume, the automatic quality control becomes a crucial aspect over the entire lifetime of the experiment. Here we focus on the design strategy for the Science Ground Segment (SGS) Data Quality Common Tools (DQCT), which has the main role to provide software solutions to gather, evaluate, and record quality information about the raw and derived data products from a primarily scientific perspective. The SGS DQCT will provide a quantitative basis for evaluating the application of reduction and calibration reference data, as well as diagnostic tools for quality parameters, flags, trend analysis diagrams and any other metadata parameter produced by the pipeline. In a large programme like Euclid, it is prohibitively expensive to process large amount of data at the pixel level just for the purpose of quality evaluation. Thus, all measures of quality at the pixel level are implemented in the individual pipeline stages, and passed along as metadata in the production. In this sense most of the tasks related to science data quality are delegated to the pipeline stages, even though the responsibility for science data quality is managed at a higher level. The DQCT subsystem of the SGS is currently under development, but its path to full realization will likely be different than that of other subsystems. Primarily because, due to a high level of parallelism and to the wide pipeline processing redundancy, for instance the mechanism of double Science Data Center for each processing function, the data quality tools have not only to be widely spread over all pipeline segments and data levels, but also to minimize the occurrences of potential diversity of solutions implemented for similar functions, ensuring the maximum of coherency and standardization for quality evaluation and reporting in the SGS.

The emerging need for efficient, reliable and scalable astronomical catalog cross-matching is becoming more pressing in the current data-driven science era, where the size of data has rapidly increased up to the Petabyte scale. C3 (Command-line Catalogue Cross-matching) is a multi-platform tool designed to efficiently cross-match massive catalogues from modern astronomical surveys, ensuring high-performance capabilities through the use of a multi-core parallel processing paradigm. The tool has been conceived to be executed as a stand-alone command-line process or integrated within any generic data reduction/analysis pipeline, providing the maximum flexibility to the end user, in terms of parameter configuration, coordinates and cross-matching types. In this work we present the architecture and the features of the tool. Moreover, since the modular design of the tool enables an easy customization to specific use cases and requirements, we present also an example of a customized C3 version designed and used in the FP7 project ViaLactea, dedicated to cross-correlate Hi-GAL clumps with multi-band compact sources.

A variety of fundamental astrophysical science topics require the determination of very accurate photometric redshifts (photo-z's). A wide plethora of methods have been developed, based either on template models fitting or on empirical explorations of the photometric parameter space. Machine learning based techniques are not explicitly dependent on the physical priors and able to produce accurate photo-z estimations within the photometric ranges derived from the spectroscopic training set. These estimates, however, are not easy to characterize in terms of a photo-z Probability Density Function (PDF), due to the fact that the analytical relation mapping the photometric parameters onto the redshift space is virtually unknown. We present METAPHOR (Machine-learning Estimation Tool for Accurate PHOtometric Redshifts), a method designed to provide a reliable PDF of the error distribution for empirical techniques. The method is implemented as a modular workflow, whose internal engine for photo-z estimation makes use of the MLPQNA neural network (Multi Layer Perceptron with Quasi Newton learning rule), with the possibility to easily replace the specific machine learning model chosen to predict photo-z's. We present a summary of results on SDSS-DR9 galaxy data, used also to perform a direct comparison with PDF's obtained by the Le Phare SED template fitting. We show that METAPHOR is capable to estimate the precision and reliability of photometric redshifts obtained with three different self-adaptive techniques, i.e. MLPQNA, Random Forest and the standard K-Nearest Neighbors models.

We present an innovative method called FilExSeC (Filaments Extraction, Selection and Classification), a data mining tool developed to investigate the possibility to refine and optimize the shape reconstruction of filamentary structures detected with a consolidated method based on the flux derivative analysis, through the column-density maps computed from Herschel infrared Galactic Plane Survey (Hi-GAL) observations of the Galactic plane. The present methodology is based on a feature extraction module followed by a machine learning model (Random Forest) dedicated to select features and to classify the pixels of the input images. From tests on both simulations and real observations the method appears reliable and robust with respect to the variability of shape and distribution of filaments. In the cases of highly defined filament structures, the presented method is able to bridge the gaps among the detected fragments, thus improving their shape reconstruction. From a preliminary "a posteriori" analysis of derived filament physical parameters, the method appears potentially able to add a sufficient contribution to complete and refine the filament reconstruction.

The VIALACTEA project has a work package dedicated to Tools and Infrastructure and, inside it, a task for the Database and Virtual Observatory Infrastructure. This task aims at providing an infrastructure to store all the resources needed by the, more purposely, scientific work packages of the project itself. This infrastructure includes a combination of: storage facilities, relational databases and web services on top of them, and has taken, as a whole, the name of VIALACTEA Knowledge Base (VLKB). This contribution illustrates the current status of this VLKB. It details the set of data resources put together; describes the database that allows data discovery through VO inspired metadata maintenance; illustrates the discovery, cutout and access services built on top of the former two for the users to exploit the data content.

Astronomy is undergoing through a methodological revolution triggered by an unprecedented wealth of complex and accurate data. DAMEWARE (DAta Mining & Exploration Web Application and REsource) is a general purpose, Web-based, Virtual Observatory compliant, distributed data mining framework specialized in massive data sets exploration with machine learning methods. We present the DAMEWARE (DAta Mining & Exploration Web Application REsource) which allows the scientific community to perform data mining and exploratory experiments on massive data sets, by using a simple web browser. DAMEWARE offers several tools which can be seen as working environments where to choose data analysis functionalities such as clustering, classification, regression, feature extraction etc., together with models and algorithms.

Due to the necessity to evaluate photo-z for a variety of huge sky survey data sets, it seemed important to provide the astronomical community with an instrument able to fill this gap. Besides the problem of moving massive data sets over the network, another critical point is that a great part of astronomical data is stored in private archives that are not fully accessible on line. So, in order to evaluate photo-z it is needed a desktop application that can be downloaded and used by everyone locally, i.e. on his own personal computer or more in general within the local intranet hosted by a data center. The name chosen for the application is PhotoRApToR, i.e. Photometric Research Application To Redshift (Cavuoti et al. 2015, 2014; Brescia 2014b). It embeds a machine learning algorithm and special tools dedicated to preand post-processing data. The ML model is the MLPQNA (Multi Layer Perceptron trained by the Quasi Newton Algorithm), which has been revealed particularly powerful for the photo-z calculation on the base of a spectroscopic sample (Cavuoti et al. 2012; Brescia et al. 2013, 2014a; Biviano et al. 2013). The PhotoRApToR program package is available, for different platforms, at the official website (http://dame.dsf.unina.it/dame_photoz.html#photoraptor).

The abundance of compact, massive, early-type galaxies (ETGs) provides important constraints to galaxy formation scenarios. Thanks to the area covered, depth, excellent spatial resolution and seeing, the ESO Public optical Kilo Degree Survey (KiDS), carried out with the VLT Survey Telescope (VST), offers a unique opportunity to conduct a complete census of the most compact galaxies in the Universe. This paper presents a first census of such systems from the first 156 square degrees of KiDS. Our analysis relies on g-, r-, and i-band effective radii ($R_{\rm e}$), derived by fitting galaxy images with PSF-convolved S\'ersic models, high-quality photometric redshifts, $z_{\rm phot}$, estimated from machine learning techniques, and stellar masses, $M_{\rm \star}$, calculated from KiDS aperture photometry. After massiveness ($M_{\rm \star} > 8 \times 10^{10}\, \rm M_{\odot}$) and compactness ($R_{\rm e} < 1.5 \, \rm kpc$ in g-, r- and i-bands) criteria are applied, a visual inspection of the candidates plus near-infrared photometry from VIKING-DR1 are used to refine our sample. The final catalog, to be spectroscopically confirmed, consists of 92 systems in the redshift range $z \sim 0.2-0.7$. This sample, which we expect to increase by a factor of ten over the total survey area, represents the first attempt to select massive super-compact ETGs (MSCGs) in KiDS. We investigate the impact of redshift systematics in the selection, finding that this seems to be a major source of contamination in our sample. A preliminary analysis shows that MSCGs exhibit negative internal colour gradients, consistent with a passive evolution of these systems. We find that the number density of MSCGs is only mildly consistent with predictions from simulations at $z>0.2$, while no such system is found at $z < 0.2$.

The exploitation of present and future synoptic (multi-band and multi-epoch) surveys requires an extensive use of automatic methods for data processing and data interpretation. In this work, using data extracted from the Catalina Real Time Transient Survey (CRTS), we investigate the classification performance of some well tested methods: Random Forest, MLPQNA (Multi Layer Perceptron with Quasi Newton Algorithm) and K-Nearest Neighbors, paying special attention to the feature selection phase. In order to do so, several classification experiments were performed. Namely: identification of cataclysmic variables, separation between galactic and extra-galactic objects and identification of supernovae.

The VIALACTEA project aims at building a predictive model of star formation in our galaxy. We present the innovative integrated framework and the main technologies and methodologies to reach this ambitious goal.

Calibrating the photometric redshifts of >10^9 galaxies for upcoming weak lensing cosmology experiments is a major challenge for the astrophysics community. The path to obtaining the required spectroscopic redshifts for training and calibration is daunting, given the anticipated depths of the surveys and the difficulty in obtaining secure redshifts for some faint galaxy populations. Here we present an analysis of the problem based on the self-organizing map, a method of mapping the distribution of data in a high-dimensional space and projecting it onto a lower-dimensional representation. We apply this method to existing photometric data from the COSMOS survey selected to approximate the anticipated Euclid weak lensing sample, enabling us to robustly map the empirical distribution of galaxies in the multidimensional color space defined by the expected Euclid filters. Mapping this multicolor distribution lets us determine where - in galaxy color space - redshifts from current spectroscopic surveys exist and where they are systematically missing. Crucially, the method lets us determine whether a spectroscopic training sample is representative of the full photometric space occupied by the galaxies in a survey. We explore optimal sampling techniques and estimate the additional spectroscopy needed to map out the color-redshift relation, finding that sampling the galaxy distribution in color space in a systematic way can efficiently meet the calibration requirements. While the analysis presented here focuses on the Euclid survey, similar analysis can be applied to other surveys facing the same calibration challenge, such as DES, LSST, and WFIRST.

The Shapley Supercluster Survey is a multi-wavelength survey covering an area of ~23 deg^2 (~260 Mpc^2 at z=0.048) around the supercluster core, including nine Abell and two poor clusters, having redshifts in the range 0.045-0.050. The survey aims to investigate the role of the cluster-scale mass assembly on the evolution of galaxies, mapping the effects of the environment from the cores of the clusters to their outskirts and along the filaments. The optical (ugri) imaging acquired with OmegaCAM on the VLT Survey Telescope is essential to achieve the project goals providing accurate multi-band photometry for the galaxy population down to m*+6. We describe the methodology adopted to construct the optical catalogues and to separate extended and point-like sources. The catalogues reach average 5sigma limiting magnitudes within a 3\arcsec diameter aperture of ugri=[24.4,24.6,24.1,23.3] and are 93% complete down to ugri=[23.8,23.8,23.5,22.0] mag, corresponding to ~m*_r+8.5. The data are highly uniform in terms of observing conditions and all acquired with seeing less than 1.1 arcsec full width at half-maximum. The median seeing in r-band is 0.6 arcsec, corresponding to 0.56 kpc h^{-1}_{70} at z=0.048. While the observations in the u, g and r bands are still ongoing, the i-band observations have been completed, and we present the i-band catalogue over the whole survey area. The latter is released and it will be regularly updated, through the use of the Virtual Observatory tools. This includes 734,319 sources down to i=22.0 mag and it is the first optical homogeneous catalogue at such a depth, covering the central region of the Shapley supercluster.

The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the survey is mapping the large-scale matter distribution in the Universe back to a redshift of ~0.5. Secondary science cases are manifold, covering topics such as galaxy evolution, Milky Way structure, and the detection of high-redshift clusters and quasars. KiDS is an ESO Public Survey and dedicated to serving the astronomical community with high-quality data products derived from the survey data, as well as with calibration data. Public data releases will be made on a yearly basis, the first two of which are presented here. For a total of 148 survey tiles (~160 sq.deg.) astrometrically and photometrically calibrated, coadded ugri images have been released, accompanied by weight maps, masks, source lists, and a multi-band source catalog. A dedicated pipeline and data management system based on the Astro-WISE software system, combined with newly developed masking and source classification software, is used for the data production of the data products described here. The achieved data quality and early science projects based on the data products in the first two data releases are reviewed in order to validate the survey data. Early scientific results include the detection of nine high-z QSOs, fifteen candidate strong gravitational lenses, high-quality photometric redshifts and galaxy structural parameters for hundreds of thousands of galaxies. (Abridged)

We estimated photometric redshifts (zphot) for more than 1.1 million galaxies of the ESO Public Kilo-Degree Survey (KiDS) Data Release 2. KiDS is an optical wide-field imaging survey carried out with the VLT Survey Telescope (VST) and the OmegaCAM camera, which aims at tackling open questions in cosmology and galaxy evolution, such as the origin of dark energy and the channel of galaxy mass growth. We present a catalogue of photometric redshifts obtained using the Multi Layer Perceptron with Quasi Newton Algorithm (MLPQNA) model, provided within the framework of the DAta Mining and Exploration Web Application REsource (DAMEWARE). These photometric redshifts are based on a spectroscopic knowledge base which was obtained by merging spectroscopic datasets from GAMA (Galaxy And Mass Assembly) data release 2 and SDSS-III data release 9. The overall 1 sigma uncertainty on Delta z = (zspec - zphot) / (1+ zspec) is ~ 0.03, with a very small average bias of ~ 0.001, a NMAD of ~ 0.02 and a fraction of catastrophic outliers (| Delta z | > 0.15) of ~0.4%.

The ESO Public Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will scan 1500 square degrees in four optical filters (u, g, r, i). Designed to be a weak lensing survey, it is ideal for galaxy evolution studies, thanks to the high spatial resolution of VST, the good seeing and the photometric depth. The surface photometry have provided with structural parameters (e.g. size and S\'ersic index), aperture and total magnitudes have been used to derive photometric redshifts from Machine learning methods and stellar masses/luminositites from stellar population synthesis. Our project aimed at investigating the evolution of the colour and structural properties of galaxies with mass and environment up to redshift $z \sim 0.5$ and more, to put constraints on galaxy evolution processes, as galaxy mergers.

We discuss whether modern machine learning methods can be used to characterize the physical nature of the large number of objects sampled by the modern multi-band digital surveys. In particular, we applied the MLPQNA (Multi Layer Perceptron with Quasi Newton Algorithm) method to the optical data of the Sloan Digital Sky Survey - Data Release 10, investigating whether photometric data alone suffice to disentangle different classes of objects as they are defined in the SDSS spectroscopic classification. We discuss three groups of classification problems: (i) the simultaneous classification of galaxies, quasars and stars; (ii) the separation of stars from quasars; (iii) the separation of galaxies with normal spectral energy distribution from those with peculiar spectra, such as starburst or starforming galaxies and AGN. While confirming the difficulty of disentangling AGN from normal galaxies on a photometric basis only, MLPQNA proved to be quite effective in the three-class separation. In disentangling quasars from stars and galaxies, our method achieved an overall efficiency of 91.31% and a QSO class purity of ~95%. The resulting catalogue of candidate quasars/AGNs consists of ~3.6 million objects, of which about half a million are also flagged as robust candidates, and will be made available on CDS VizieR facility.

Photometric redshifts (photo-z) are crucial to the scientific exploitation of modern panchromatic digital surveys. In this paper we present PhotoRApToR (Photometric Research Application To Redshift): a Java/C++ based desktop application capable to solve non-linear regression and multi-variate classification problems, in particular specialized for photo-z estimation. It embeds a machine learning algorithm, namely a multilayer neural network trained by the Quasi Newton learning rule, and special tools dedicated to pre- and postprocessing data. PhotoRApToR has been successfully tested on several scientific cases. The application is available for free download from the DAME Program web site.

We forecast dark energy constraints that could be obtained from a new large sample of Type Ia supernovae where those at high redshift are acquired with the Euclid space mission. We simulate a three-prong SN survey: a z<0.35 nearby sample (8000 SNe), a 0.2<z<0.95 intermediate sample (8800 SNe), and a 0.75<z<1.55 high-z sample (1700 SNe). The nearby and intermediate surveys are assumed to be conducted from the ground, while the high-z is a joint ground- and space-based survey. This latter survey, the "Dark Energy Supernova Infra-Red Experiment" (DESIRE), is designed to fit within 6 months of Euclid observing time, with a dedicated observing program. We simulate the SN events as they would be observed in rolling-search mode by the various instruments, and derive the quality of expected cosmological constraints. We account for known systematic uncertainties, in particular calibration uncertainties including their contribution through the training of the supernova model used to fit the supernovae light curves. Using conservative assumptions and a 1-D geometric Planck prior, we find that the ensemble of surveys would yield competitive constraints: a constant equation of state parameter can be constrained to sigma(w)=0.022, and a Dark Energy Task Force figure of merit of 203 is found for a two-parameter equation of state. Our simulations thus indicate that Euclid can bring a significant contribution to a purely geometrical cosmology constraint by extending a high-quality SN Hubble diagram to z~1.5. We also present other science topics enabled by the DESIRE Euclid observations

We review some aspects of the current state of data-intensive astronomy, its methods, and some outstanding data analysis challenges. Astronomy is at the forefront of "big data" science, with exponentially growing data volumes and data rates, and an ever-increasing complexity, now entering the Petascale regime. Telescopes and observatories from both ground and space, covering a full range of wavelengths, feed the data via processing pipelines into dedicated archives, where they can be accessed for scientific analysis. Most of the large archives are connected through the Virtual Observatory framework, that provides interoperability standards and services, and effectively constitutes a global data grid of astronomy. Making discoveries in this overabundance of data requires applications of novel, machine learning tools. We describe some of the recent examples of such applications.

Accurate photometric redshifts for large samples of galaxies are among the main products of modern multiband digital surveys. Over the last decade, the Sloan Digital Sky Survey (SDSS) has become a sort of benchmark against which to test the various methods. We present an application of a new method to the estimation of photometric redshifts for the galaxies in the SDSS Data Release 9 (SDSS-DR9). Photometric redshifts for more than 143 million galaxies were produced and made available at the URL: http://dame.dsf.unina.it/catalog/DR9PHOTOZ/. The MLPQNA (Multi Layer Perceptron with Quasi Newton Algorithm) model provided within the framework of the DAMEWARE (DAta Mining and Exploration Web Application REsource) is an interpolative method derived from machine learning models. The obtained redshifts have an overall uncertainty of sigma=0.023 with a very small average bias of about 3x10^-5, and a fraction of catastrophic outliers of about 5%. This result is slightly better than what was already available in the literature, particularly in terms of the smaller fraction of catastrophic outliers.

Astronomy is undergoing through a methodological revolution triggered by an unprecedented wealth of complex and accurate data. The new panchromatic, synoptic sky surveys require advanced tools for discovering patterns and trends hidden behind data which are both complex and of high dimensionality. We present DAMEWARE (DAta Mining & Exploration Web Application REsource): a general purpose, web-based, distributed data mining environment developed for the exploration of large datasets, and finely tuned for astronomical applications. By means of graphical user interfaces, it allows the user to perform classification, regression or clustering tasks with machine learning methods. Salient features of DAMEWARE include its capability to work on large datasets with minimal human intervention, and to deal with a wide variety of real problems such as the classification of globular clusters in the galaxy NGC1399, the evaluation of photometric redshifts and, finally, the identification of candidate Active Galactic Nuclei in multiband photometric surveys. In all these applications, DAMEWARE allowed to achieve better results than those attained with more traditional methods. With the aim of providing potential users with all needed information, in this paper we briefly describe the technological background of DAMEWARE, give a short introduction to some relevant aspects of data mining, followed by a summary of some science cases and, finally, we provide a detailed description of a template use case.

In the last decade a new generation of telescopes and sensors has allowed the production of a very large amount of data and astronomy has become a data-rich science. New automatic methods largely based on machine learning are needed to cope with such data tsunami. We present some results in the fields of photometric redshifts and galaxy classification, obtained using the MLPQNA algorithm available in the DAMEWARE (Data Mining and Web Application Resource) for the SDSS galaxies (DR9 and DR10). We present PhotoRApToR (Photometric Research Application To Redshift): a Java based desktop application capable to solve regression and classification problems and specialized for photo-z estimation.

In this paper we discuss an application of machine learning based methods to the identification of candidate AGN from optical survey data and to the automatic classification of AGNs in broad classes. We applied four different machine learning algorithms, namely the Multi Layer Perceptron (MLP), trained respectively with the Conjugate Gradient, Scaled Conjugate Gradient and Quasi Newton learning rules, and the Support Vector Machines (SVM), to tackle the problem of the classification of emission line galaxies in different classes, mainly AGNs vs non-AGNs, obtained using optical photometry in place of the diagnostics based on line intensity ratios which are classically used in the literature. Using the same photometric features we discuss also the behavior of the classifiers on finer AGN classification tasks, namely Seyfert I vs Seyfert II and Seyfert vs LINER. Furthermore we describe the algorithms employed, the samples of spectroscopically classified galaxies used to train the algorithms, the procedure followed to select the photometric parameters and the performances of our methods in terms of multiple statistical indicators. The results of the experiments show that the application of self adaptive data mining algorithms trained on spectroscopic data sets and applied to carefully chosen photometric parameters represents a viable alternative to the classical methods that employ time-consuming spectroscopic observations.

MLPQNA stands for Multi Layer Perceptron with Quasi Newton Algorithm and it is a machine learning method which can be used to cope with regression and classification problems on complex and massive data sets. In this paper we give the formal description of the method and present the results of its application to the evaluation of photometric redshifts for quasars. The data set used for the experiment was obtained by merging four different surveys (SDSS, GALEX, UKIDSS and WISE), thus covering a wide range of wavelengths from the UV to the mid-infrared. The method is able i) to achieve a very high accuracy; ii) to drastically reduce the number of outliers and catastrophic objects; iii) to discriminate among parameters (or features) on the basis of their significance, so that the number of features used for training and analysis can be optimized in order to reduce both the computational demands and the effects of degeneracy. The best experiment, which makes use of a selected combination of parameters drawn from the four surveys, leads, in terms of DeltaZnorm (i.e. (zspec-zphot)/(1+zspec)), to an average of DeltaZnorm = 0.004, a standard deviation sigma = 0.069 and a Median Absolute Deviation MAD = 0.02 over the whole redshift range (i.e. zspec <= 3.6), defined by the 4-survey cross-matched spectroscopic sample. The fraction of catastrophic outliers, i.e. of objects with photo-z deviating more than 2sigma from the spectroscopic value is < 3%, leading to a sigma = 0.035 after their removal, over the same redshift range. The method is made available to the community through the DAMEWARE web application.

In the last decade a new generation of telescopes and sensors has allowed the production of a very large amount of data and astronomy has become, a data-rich science; this transition is often labeled as: "data revolution" and "data tsunami". The first locution puts emphasis on the expectations of the astronomers while the second stresses, instead, the dramatic problem arising from this large amount of data: which is no longer computable with traditional approaches to data storage, data reduction and data analysis. In a new, age new instruments are necessary, as it happened in the Bronze age when mankind left the old instruments made out of stone to adopt the new, better ones made with bronze. Everything changed, even the social structure. In a similar way, this new age of Astronomy calls for a new generation of tools and, for a new methodological approach to many problems, and for the acquisition of new skills. The attems to find a solution to this problems falls under the umbrella of a new discipline which originated by the intersection of astronomy, statistics and computer science: Astroinformatics, (Borne, 2009; Djorgovski et al., 2006).

We present a multi-purpose genetic algorithm, designed and implemented with GPGPU / CUDA parallel computing technology. The model was derived from our CPU serial implementation, named GAME (Genetic Algorithm Model Experiment). It was successfully tested and validated on the detection of candidate Globular Clusters in deep, wide-field, single band HST images. The GPU version of GAME will be made available to the community by integrating it into the web application DAMEWARE (DAta Mining Web Application REsource (http://dame.dsf.unina.it/beta_info.html), a public data mining service specialized on massive astrophysical data. Since genetic algorithms are inherently parallel, the GPGPU computing paradigm leads to a speedup of a factor of 200x in the training phase with respect to the CPU based version.

The scope of this paper is to compare the catalog extraction performances obtained using the new combination of SExtractor with PSFEx, against the more traditional and diffuse application of DAOPHOT with ALLSTAR; therefore, the paper may provide a guide for the selection of the most suitable catalog extraction software. Both software packages were tested on two kinds of simulated images having, respectively, a uniform spatial distribution of sources and an overdensity in the center. In both cases, SExtractor is able to generate a deeper catalog than DAOPHOT. Moreover, the use of neural networks for object classification plus the novel SPREAD\_MODEL parameter push down to the limiting magnitude the possibility of star/galaxy separation. DAOPHOT and ALLSTAR provide an optimal solution for point-source photometry in stellar fields and very accurate and reliable PSF photometry, with robust star-galaxy separation. However, they are not useful for galaxy characterization, and do not generate catalogs that are very complete for faint sources. On the other hand, SExtractor, along with the new capability to derive PSF photometry, turns to be competitive and returns accurate photometry also for galaxies. We can assess that the new version of SExtractor, used in conjunction with PSFEx, represents a very powerful software package for source extraction with performances comparable to those of DAOPHOT. Finally, by comparing the results obtained in the case of a uniform and of an overdense spatial distribution of stars, we notice, for both software packages, a decline for the latter case in the quality of the results produced in terms of magnitudes and centroids.

We present a multi-purpose genetic algorithm, designed and implemented with GPGPU / CUDA parallel computing technology. The model was derived from a multi-core CPU serial implementation, named GAME, already scientifically successfully tested and validated on astrophysical massive data classification problems, through a web application resource (DAMEWARE), specialized in data mining based on Machine Learning paradigms. Since genetic algorithms are inherently parallel, the GPGPU computing paradigm has provided an exploit of the internal training features of the model, permitting a strong optimization in terms of processing performances and scalability.

Context. Since the advent of modern multiband digital sky surveys, photometric redshifts (photo-z's) have become relevant if not crucial to many fields of observational cosmology, from the characterization of cosmic structures, to weak and strong lensing. Aims. We describe an application to an astrophysical context, namely the evaluation of photometric redshifts, of MLPQNA, a machine learning method based on Quasi Newton Algorithm. Methods. Theoretical methods for photo-z's evaluation are based on the interpolation of a priori knowledge (spectroscopic redshifts or SED templates) and represent an ideal comparison ground for neural networks based methods. The MultiLayer Perceptron with Quasi Newton learning rule (MLPQNA) described here is a computing effective implementation of Neural Networks for the first time exploited to solve regression problems in the astrophysical context and is offered to the community through the DAMEWARE (DAta Mining & ExplorationWeb Application REsource) infrastructure. Results. The PHAT contest (Hildebrandt et al. 2010) provides a standard dataset to test old and new methods for photometric redshift evaluation and with a set of statistical indicators which allow a straightforward comparison among different methods. The MLPQNA model has been applied on the whole PHAT1 dataset of 1984 objects after an optimization of the model performed by using as training set the 515 available spectroscopic redshifts. When applied to the PHAT1 dataset, MLPQNA obtains the best bias accuracy (0.0006) and very competitive accuracies in terms of scatter (0.056) and outlier percentage (16.3%), scoring as the second most effective empirical method among those which have so far participated to the contest. MLPQNA shows better generalization capabilities than most other empirical methods especially in presence of underpopulated regions of the Knowledge Base.

We present an application of self-adaptive supervised learning classifiers derived from the Machine Learning paradigm, to the identification of candidate Globular Clusters in deep, wide-field, single band HST images. Several methods provided by the DAME (Data Mining & Exploration) web application, were tested and compared on the NGC1399 HST data described in Paolillo 2011. The best results were obtained using a Multi Layer Perceptron with Quasi Newton learning rule which achieved a classification accuracy of 98.3%, with a completeness of 97.8% and 1.6% of contamination. An extensive set of experiments revealed that the use of accurate structural parameters (effective radius, central surface brightness) does improve the final result, but only by 5%. It is also shown that the method is capable to retrieve also extreme sources (for instance, very extended objects) which are missed by more traditional approaches.

Nowadays, many scientific areas share the same broad requirements of being able to deal with massive and distributed datasets while, when possible, being integrated with services and applications. In order to solve the growing gap between the incremental generation of data and our understanding of it, it is required to know how to access, retrieve, analyze, mine and integrate data from disparate sources. One of the fundamental aspects of any new generation of data mining software tool or package which really wants to become a service for the community is the possibility to use it within complex workflows which each user can fine tune in order to match the specific demands of his scientific goal. These workflows need often to access different resources (data, providers, computing facilities and packages) and require a strict interoperability on (at least) the client side. The project DAME (DAta Mining & Exploration) arises from these requirements by providing a distributed WEB-based data mining infrastructure specialized on Massive Data Sets exploration with Soft Computing methods. Originally designed to deal with astrophysical use cases, where first scientific application examples have demonstrated its effectiveness, the DAME Suite results as a multi-disciplinary platform-independent tool perfectly compliant with modern KDD (Knowledge Discovery in Databases) requirements and Information & Communication Technology trends.

Astronomical data are gathered through a very large number of heterogeneous techniques and stored in very diversified and often incompatible data repositories. Moreover in the e-science environment, it is needed to integrate services across distributed, heterogeneous, dynamic "virtual organizations" formed by different resources within a single enterprise and/or external resource sharing and service provider relationships. The DAME/VONeural project, run jointly by the University Federico II, INAF (National Institute of Astrophysics) Astronomical Observatories of Napoli and the California Institute of Technology, aims at creating a single, sustainable, distributed e-infrastructure for data mining and exploration in massive data sets, to be offered to the astronomical (but not only) community as a web application. The framework makes use of distributed computing environments (e.g. S.Co.P.E.) and matches the international IVOA standards and requirements. The integration process is technically challenging due to the need of achieving a specific quality of service when running on top of different native platforms. In these terms, the result of the DAME/VO-Neural project effort will be a service-oriented architecture, obtained by using appropriate standards and incorporating Grid paradigms and restful Web services frameworks where needed, that will have as main target the integration of interdisciplinary distributed systems within and across organizational domains.

The exponential growth of astronomical data collected by both ground based and space borne instruments has fostered the growth of Astroinformatics: a new discipline laying at the intersection between astronomy, applied computer science, and information and computation (ICT) technologies. At the very heart of Astroinformatics is a complex set of methodologies usually called Data Mining (DM) or Knowledge Discovery in Data Bases (KDD). In the astronomical domain, DM/KDD are still in a very early usage stage, even though new methods and tools are being continuously deployed in order to cope with the Massive Data Sets (MDS) that can only grow in the future. In this paper, we briefly outline some general problems encountered when applying DM/KDD methods to astrophysical problems, and describe the DAME (DAta Mining & Exploration) web application. While specifically tailored to work on MDS, DAME can be effectively applied also to smaller data sets. As an illustration, we describe two application of DAME to two different problems: the identification of candidate globular clusters in external galaxies, and the classification of active galactic nuclei (AGN). We believe that tools and services of this nature will become increasingly necessary for the data-intensive astronomy (and indeed all sciences) in the 21st century.

Nowadays, many scientific areas share the same need of being able to deal with massive and distributed datasets and to perform on them complex knowledge extraction tasks. This simple consideration is behind the international efforts to build virtual organizations such as, for instance, the Virtual Observatory (VObs). DAME (DAta Mining & Exploration) is an innovative, general purpose, Web-based, VObs compliant, distributed data mining infrastructure specialized in Massive Data Sets exploration with machine learning methods. Initially fine tuned to deal with astronomical data only, DAME has evolved in a general purpose platform which has found applications also in other domains of human endeavor. We present the products and a short outline of a science case, together with a detailed description of main features available in the beta release of the web application now released.

S.Co.P.E. is one of the four projects funded by the Italian Government in order to provide Southern Italy with a distributed computing infrastructure for fundamental science. Beside being aimed at building the infrastructure, S.Co.P.E. is also actively pursuing research in several areas among which astrophysics and observational cosmology. We shortly summarize the most significant results obtained in the first two years of the project and related to the development of middleware and Data Mining tools for the Virtual Observatory.

GRID-launcher-1.0 was built within the VO-Tech framework, as a software interface between the UK-ASTROGRID and a generic GRID infrastructures in order to allow any ASTROGRID user to launch on the GRID computing intensive tasks from the ASTROGRID Workbench or Desktop. Even though of general application, so far the Grid-Launcher has been tested on a few selected softwares (VONeural-MLP, VONeural-SVM, Sextractor and SWARP) and on the SCOPE-GRID.

VO-Neural is the natural evolution of the Astroneural project which was started in 1994 with the aim to implement a suite of neural tools for data mining in astronomical massive data sets. At a difference with its ancestor, which was implemented under Matlab, VO-Neural is written in C++, object oriented, and it is specifically tailored to work in distributed computing architectures. We discuss the current status of implementation of VO-Neural, present an application to the classification of Active Galactic Nuclei, and outline the ongoing work to improve the functionalities of the package.