In recent times, neural networks have become a powerful tool for the analysis of complex and abstract data models. However, their introduction intrinsically increases our uncertainty about which features of the analysis are model-related and which are due to the neural network. This means that predictions by neural networks have biases which cannot be trivially distinguished from being due to the true nature of the creation and observation of data or not. In order to attempt to address such issues we discuss Bayesian neural networks: neural networks where the uncertainty due to the network can be characterised. In particular, we present the Bayesian statistical framework which allows us to categorise uncertainty in terms of the ingrained randomness of observing certain data and the uncertainty from our lack of knowledge about how data can be created and observed. In presenting such techniques we show how errors in prediction by neural networks can be obtained in principle, and provide the two favoured methods for characterising these errors. We will also describe how both of these methods have substantial pitfalls when put into practice, highlighting the need for other statistical techniques to truly be able to do inference when using neural networks.

We present an extension of our recently developed Wasserstein optimized model to emulate accurate high-resolution features from computationally cheaper low-resolution cosmological simulations. Our deep physical modelling technique relies on restricted neural networks to perform a mapping of the distribution of the low-resolution cosmic density field to the space of the high-resolution small-scale structures. We constrain our network using a single triplet of high-resolution initial conditions and the corresponding low- and high-resolution evolved dark matter simulations from the Quijote suite of simulations. We exploit the information content of the high-resolution initial conditions as a well constructed prior distribution from which the network emulates the small-scale structures. Once fitted, our physical model yields emulated high-resolution simulations at low computational cost, while also providing some insights about how the large-scale modes affect the small-scale structure in real space.

With the advent of future big-data surveys, automated tools for unsupervised discovery are becoming ever more necessary. In this work, we explore the ability of deep generative networks for detecting outliers in astronomical imaging datasets. The main advantage of such generative models is that they are able to learn complex representations directly from the pixel space. Therefore, these methods enable us to look for subtle morphological deviations which are typically missed by more traditional moment-based approaches. We use a generative model to learn a representation of expected data defined by the training set and then look for deviations from the learned representation by looking for the best reconstruction of a given object. In this first proof-of-concept work, we apply our method to two different test cases. We first show that from a set of simulated galaxies, we are able to detect $\sim90\%$ of merging galaxies if we train our network only with a sample of isolated ones. We then explore how the presented approach can be used to compare observations and hydrodynamic simulations by identifying observed galaxies not well represented in the models.

An ambitious goal in cosmology is to forward-model the observed distribution of galaxies in the nearby Universe today from the initial conditions of large-scale structures. For practical reasons, the spatial resolution at which this can be done is necessarily limited. Consequently, one needs a mapping between the density of dark matter averaged over ~Mpc scales, and the distribution of dark matter halos (used as a proxy for galaxies) in the same region. Here we demonstrate a method for determining the halo mass distribution function by learning the tracer bias between density fields and halo catalogues using a neural bias model. The method is based on the Bayesian analysis of simple, physically motivated, neural network-like architectures, which we denote as neural physical engines, and neural density estimation. As a result, we are able to sample the initial phases of the dark matter density field whilst inferring the parameters describing the halo mass distribution function, providing a fully Bayesian interpretation of both the initial dark matter density distribution and the neural bias model. We successfully run an upgraded BORG inference using our new likelihood and neural bias model with halo catalogues derived from full N-body simulations. We notice orders of magnitude improvement in modelling compared to classical biasing techniques.

The Quijote simulations are a set of 43100 full N-body simulations spanning more than 7000 cosmological models in the $\{\Omega_{\rm m}, \Omega_{\rm b}, h, n_s, \sigma_8, M_\nu, w \}$ hyperplane. At a single redshift the simulations contain more than 8.5 trillions of particles over a combined volume of 43100 $(h^{-1}{\rm Gpc})^3$. Billions of dark matter halos and cosmic voids have been identified in the simulations, whose runs required more than 35 million core hours. The Quijote simulations have been designed for two main purposes: 1) to quantify the information content on cosmological observables, and 2) to provide enough data to train machine learning algorithms. In this paper we describe the simulations and show a few of their applications. We also release the Petabyte of data generated, comprising hundreds of thousands of simulation snapshots at multiple redshifts, halo and void catalogs, together with millions of summary statistics such as power spectra, bispectra, correlation functions, marked power spectra, and estimated probability density functions.

We present a novel halo painting network that learns to map approximate 3D dark matter fields to realistic halo distributions. This map is provided via a physically motivated network with which we can learn the non-trivial local relation between dark matter density field and halo distributions without relying on a physical model. Unlike other generative or regressive models, a well motivated prior and simple physical principles allow us to train the mapping network quickly and with relatively little data. In learning to paint halo distributions from computationally cheap, analytical and non-linear density fields, we bypass the need for full particle mesh simulations and halo finding algorithms. Furthermore, by design, our halo painting network needs only local patches of dark matter density to predict the halos, and as such, it can predict the 3D halo distribution for any arbitrary simulation box size. Our neural network can be trained using small simulations and used to predict large halo distributions, as long as the resolutions are equivalent. We evaluate our model's ability to generate 3D halo count distributions which reproduce, to a high degree, summary statistics such as the power spectrum and bispectrum, of the input or reference realizations.

Likelihood-free inference provides a framework for performing rigorous Bayesian inference using only forward simulations, properly accounting for all physical and observational effects that can be successfully included in the simulations. The key challenge for likelihood-free applications in cosmology, where simulation is typically expensive, is developing methods that can achieve high-fidelity posterior inference with as few simulations as possible. Density-estimation likelihood-free inference (DELFI) methods turn inference into a density estimation task on a set of simulated data-parameter pairs, and give orders of magnitude improvements over traditional Approximate Bayesian Computation approaches to likelihood-free inference. In this paper we use neural density estimators (NDEs) to learn the likelihood function from a set of simulated datasets, with active learning to adaptively acquire simulations in the most relevant regions of parameter space on-the-fly. We demonstrate the approach on a number of cosmological case studies, showing that for typical problems high-fidelity posterior inference can be achieved with just $\mathcal{O}(10^3)$ simulations or fewer. In addition to enabling efficient simulation-based inference, for simple problems where the form of the likelihood is known, DELFI offers a fast alternative to MCMC sampling, giving orders of magnitude speed-up in some cases. Finally, we introduce \textsc{pydelfi} -- a flexible public implementation of DELFI with NDEs and active learning -- available at \url{https://github.com/justinalsing/pydelfi}.

The detection of air-shower events via radio signals requires to develop a trigger algorithm for a clean discrimination between signal and background events in order to reduce the data stream coming from false triggers. In this contribution we will describe an approach to trigger air-shower events on a single-antenna level as well as performing an online reconstruction of the shower parameters using neural networks.

Compressing large data sets to a manageable number of summaries that are informative about the underlying parameters vastly simplifies both frequentist and Bayesian inference. When only simulations are available, these summaries are typically chosen heuristically, so they may inadvertently miss important information. We introduce a simulation-based machine learning technique that trains artificial neural networks to find non-linear functionals of data that maximise Fisher information: information maximising neural networks (IMNNs). In test cases where the posterior can be derived exactly, likelihood-free inference based on automatically derived IMNN summaries produces nearly exact posteriors, showing that these summaries are good approximations to sufficient statistics. In a series of numerical examples of increasing complexity and astrophysical relevance we show that IMNNs are robustly capable of automatically finding optimal, non-linear summaries of the data even in cases where linear compression fails: inferring the variance of Gaussian signal in the presence of noise; inferring cosmological parameters from mock simulations of the Lyman-{\alpha} forest in quasar spectra; and inferring frequency-domain parameters from LISA-like detections of gravitational waveforms. In this final case, the IMNN summary outperforms linear data compression by avoiding the introduction of spurious likelihood maxima. We anticipate that the automatic physical inference method described in this paper will be essential to obtain both accurate and precise cosmological parameter estimates from complex and large astronomical data sets, including those from LSST and Euclid.

We apply deep recurrent neural networks, which are capable of learning complex sequential information, to classify supernovae\footnote{Code available at \href{https://github.com/adammoss/supernovae}{https://github.com/adammoss/supernovae}}. The observational time and filter fluxes are used as inputs to the network, but since the inputs are agnostic additional data such as host galaxy information can also be included. Using the Supernovae Photometric Classification Challenge (SPCC) data, we find that deep networks are capable of learning about light curves, however the performance of the network is highly sensitive to the amount of training data. For a training size of 50\% of the representational SPCC dataset (around $10^4$ supernovae) we obtain a type-Ia vs. non-type-Ia classification accuracy of 94.7\%, an area under the Receiver Operating Characteristic curve AUC of 0.986 and a SPCC figure-of-merit $F_1=0.64$. When using only the data for the early-epoch challenge defined by the SPCC we achieve a classification accuracy of 93.1\%, AUC of 0.977 and $F_1=0.58$, results almost as good as with the whole light-curve. By employing bidirectional neural networks we can acquire impressive classification results between supernovae types -I,~-II and~-III at an accuracy of 90.4\% and AUC of 0.974. We also apply a pre-trained model to obtain classification probabilities as a function of time, and show it can give early indications of supernovae type. Our method is competitive with existing algorithms and has applications for future large-scale photometric surveys.

Discordance in the $\Lambda$CDM cosmological model can be seen by comparing parameters constrained by CMB measurements to those inferred by probes of large scale structure. Recent improvements in observations, including final data releases from both Planck and SDSS-III BOSS, as well as improved astrophysical uncertainty analysis of CFHTLenS, allows for an update in the quantification of any tension between large and small scales. This paper is intended, primarily, as a discussion on the quantifications of discordance when comparing the parameter constraints of a model when given two different data sets. We consider KL-divergence, comparison of Bayesian evidences and other statistics which are sensitive to the mean, variance and shape of the distributions. However, as a by-product, we present an update to the similar analysis in (Battye, Charnock and Moss; 2015) where we find that, considering new data and treatment of priors, the constraints from the CMB and from a combination of LSS probes are in greater agreement and any tension only persists to a minor degree. In particular, we find the parameter constraints from the combination of LSS probes which are most discrepant with the Planck2015+Pol+BAO parameter distributions can be quantified at a 2.55$\sigma$ tension using the method introduced in (Battye, Charnock and Moss; 2015). If instead we use the distributions constrained by the combination of LSS probes which are in greatest agreement with those from Planck2015+Pol+BAO this tension is only 0.76$\sigma$.

We present the first complete Markov chain Monte Carlo analysis of cosmological models with evolving cosmic (super)string networks, using the unconnected segment model in the unequal-time correlator formalism. For ordinary cosmic string networks, we derive joint constraints on Lambda cold dark matter (CDM) and string network parameters, namely the string tension Gmu, the loop-chopping efficiency c_r and the string wiggliness \alpha. For cosmic superstrings, we obtain joint constraints on the fundamental string tension Gmu_F, the string coupling g_s, the self-interaction coefficient c_s, and the volume of compact extra dimensions w. This constitutes the most comprehensive CMB analysis of LambdaCDM cosmology + strings to date. For ordinary cosmic string networks our updated constraint on the string tension is, in relativistic units, Gmu<1.1x10^-7, while for cosmic superstrings our constraint on the fundamental string tension is Gmu_F<2.8x10^-8, both obtained using Planck2015 temperature and polarisation data.

There is a tension between measurements of the amplitude of the power spectrum of density perturbations inferred using the Cosmic Microwave Background (CMB) and directly measured by Large-Scale Structure (LSS) on smaller scales. We show that this tension exists, and is robust, for a range of LSS indicators including clusters, lensing and redshift space distortions and using CMB data from either $Planck$ or WMAP+SPT/ACT. One obvious way to try to reconcile this is the inclusion of a massive neutrino which could be either active or sterile. Using $Planck$ and a combination of all the LSS data we find that (i) for an active neutrino $\sum m_{\nu}= (0.357\pm0.099)\,{\rm eV}$ and (ii) for a sterile neutrino $m_{\rm sterile}^{\rm eff}= (0.67\pm0.18)\,{\rm eV}$ and $\Delta N_{\rm eff}= 0.32\pm0.20$. This is, however, at the expense of a degraded fit to $Planck$ temperature data, and we quantify the residual tension at $2.5\sigma$ and $1.6 \sigma$ for massive and sterile neutrinos respectively. We also consider alternative explanations including a lower redshift for reionization that would be in conflict with polarisation measurements made by WMAP and $ad$-$hoc$ modifications to primordial power spectrum.