Hydrodynamical simulations of galaxy formation and evolution attempt to fully model the physics that shapes galaxies. The agreement between the morphology of simulated and real galaxies, and the way the morphological types are distributed across galaxy scaling relations are important probes of our knowledge of galaxy formation physics. Here we propose an unsupervised deep learning approach to perform a stringent test of the fine morphological structure of galaxies coming from the Illustris and IllustrisTNG (TNG100 and TNG50) simulations against observations from a subsample of the Sloan Digital Sky Survey. Our framework is based on PixelCNN, an autoregressive model for image generation with an explicit likelihood. We adopt a strategy that combines the output of two PixelCNN networks in a metric that isolates the fine morphological details of galaxies from the sky background. We are able to \emph{quantitatively} identify the improvements of IllustrisTNG, particularly in the high-resolution TNG50 run, over the original Illustris. However, we find that the fine details of galaxy structure are still different between observed and simulated galaxies. This difference is driven by small, more spheroidal, and quenched galaxies which are globally less accurate regardless of resolution and which have experienced little improvement between the three simulations explored. We speculate that this disagreement, that is less severe for quenched disky galaxies, may stem from a still too coarse numerical resolution, which struggles to properly capture the inner, dense regions of quenched spheroidal galaxies.

In recent times, neural networks have become a powerful tool for the analysis of complex and abstract data models. However, their introduction intrinsically increases our uncertainty about which features of the analysis are model-related and which are due to the neural network. This means that predictions by neural networks have biases which cannot be trivially distinguished from being due to the true nature of the creation and observation of data or not. In order to attempt to address such issues we discuss Bayesian neural networks: neural networks where the uncertainty due to the network can be characterised. In particular, we present the Bayesian statistical framework which allows us to categorise uncertainty in terms of the ingrained randomness of observing certain data and the uncertainty from our lack of knowledge about how data can be created and observed. In presenting such techniques we show how errors in prediction by neural networks can be obtained in principle, and provide the two favoured methods for characterising these errors. We will also describe how both of these methods have substantial pitfalls when put into practice, highlighting the need for other statistical techniques to truly be able to do inference when using neural networks.

Machine learning lies at the heart of new possibilities for scientific discovery, knowledge generation, and artificial intelligence. Its potential benefits to these fields requires going beyond predictive accuracy and focusing on interpretability. In particular, many scientific problems require interpretations in a domain-specific interpretable feature space (e.g. the frequency domain) whereas attributions to the raw features (e.g. the pixel space) may be unintelligible or even misleading. To address this challenge, we propose TRIM (TRansformation IMportance), a novel approach which attributes importances to features in a transformed space and can be applied post-hoc to a fully trained model. TRIM is motivated by a cosmological parameter estimation problem using deep neural networks (DNNs) on simulated data, but it is generally applicable across domains/models and can be combined with any local interpretation method. In our cosmology example, combining TRIM with contextual decomposition shows promising results for identifying which frequencies a DNN uses, helping cosmologists to understand and validate that the model learns appropriate physical features rather than simulation artifacts.

We present the first reconstruction of dark matter maps from weak lensing observational data using deep learning. We train a convolution neural network (CNN) with a Unet based architecture on over $3.6\times10^5$ simulated data realizations with non-Gaussian shape noise and with cosmological parameters varying over a broad prior distribution. We interpret our newly created DES SV map as an approximation of the posterior mean $P(\kappa | \gamma)$ of the convergence given observed shear. Our DeepMass method is substantially more accurate than existing mass-mapping methods. With a validation set of 8000 simulated DES SV data realizations, compared to Wiener filtering with a fixed power spectrum, the DeepMass method improved the mean-square-error (MSE) by 11 per cent. With N-body simulated MICE mock data, we show that Wiener filtering with the optimal known power spectrum still gives a worse MSE than our generalized method with no input cosmological parameters; we show that the improvement is driven by the non-linear structures in the convergence. With higher galaxy density in future weak lensing data unveiling more non-linear scales, it is likely that deep learning will be a leading approach for mass mapping with Euclid and LSST.

We present a Bayesian machine learning architecture that combines a physically motivated parametrization and an analytic error model for the likelihood with a deep generative model providing a powerful data-driven prior for complex signals. This combination yields an interpretable and differentiable generative model, allows the incorporation of prior knowledge, and can be utilized for observations with different data quality without having to retrain the deep network. We demonstrate our approach with an example of astronomical source separation in current imaging data, yielding a physical and interpretable model of astronomical scenes.

We develop a generative model-based approach to Bayesian inverse problems, such as image reconstruction from noisy and incomplete images. Our framework addresses two common challenges of Bayesian reconstructions: 1) It makes use of complex, data-driven priors that comprise all available information about the uncorrupted data distribution. 2) It enables computationally tractable uncertainty quantification in the form of posterior analysis in latent and data space. The method is very efficient in that the generative model only has to be trained once on an uncorrupted data set, after that, the procedure can be used for arbitrary corruption types.

This paper introduces cosmoDC2, a large synthetic galaxy catalog designed to support precision dark energy science with the Large Synoptic Survey Telescope (LSST). CosmoDC2 is the starting point for the second data challenge (DC2) carried out by the LSST Dark Energy Science Collaboration (LSST DESC). The catalog is based on a trillion-particle, 4.225 Gpc^3 box cosmological N-body simulation, the `Outer Rim' run. It covers 440 deg^2 of sky area to a redshift of z=3 and is complete to a magnitude depth of 28 in the r-band. Each galaxy is characterized by a multitude of properties including stellar mass, morphology, spectral energy distributions, broadband filter magnitudes, host halo information and weak lensing shear. The size and complexity of cosmoDC2 requires an efficient catalog generation methodology; our approach is based on a new hybrid technique that combines data-driven empirical approaches with semi-analytic galaxy modeling. A wide range of observation-based validation tests has been implemented to ensure that cosmoDC2 enables the science goals of the planned LSST DESC DC2 analyses. This paper also represents the official release of the cosmoDC2 data set, including an efficient reader that facilitates interaction with the data.

The Core Cosmology Library (CCL) provides routines to compute basic cosmological observables to a high degree of accuracy, which have been verified with an extensive suite of validation tests. Predictions are provided for many cosmological quantities, including distances, angular power spectra, correlation functions, halo bias and the halo mass function through state-of-the-art modeling prescriptions available in the literature. Fiducial specifications for the expected galaxy distributions for the Large Synoptic Survey Telescope (LSST) are also included, together with the capability of computing redshift distributions for a user-defined photometric redshift model. A rigorous validation procedure, based on comparisons between CCL and independent software packages, allows us to establish a well-defined numerical accuracy for each predicted quantity. As a result, predictions for correlation functions of galaxy clustering, galaxy-galaxy lensing and cosmic shear are demonstrated to be within a fraction of the expected statistical uncertainty of the observables for the models and in the range of scales of interest to LSST. CCL is an open source software package written in C, with a python interface and publicly available at https://github.com/LSSTDESC/CCL.

Large scale imaging surveys will increase the number of galaxy-scale strong lensing candidates by maybe three orders of magnitudes beyond the number known today. Finding these rare objects will require picking them out of at least tens of millions of images and deriving scientific results from them will require quantifying the efficiency and bias of any search method. To achieve these objectives automated methods must be developed. Because gravitational lenses are rare objects reducing false positives will be particularly important. We present a description and results of an open gravitational lens finding challenge. Participants were asked to classify 100,000 candidate objects as to whether they were gravitational lenses or not with the goal of developing better automated methods for finding lenses in large data sets. A variety of methods were used including visual inspection, arc and ring finders, support vector machines (SVM) and convolutional neural networks (CNN). We find that many of the methods will be easily fast enough to analyse the anticipated data flow. In test data, several methods are able to identify upwards of half the lenses after applying some thresholds on the lens characteristics such as lensed image brightness, size or contrast with the lens galaxy without making a single false-positive identification. This is significantly better than direct inspection by humans was able to do. (abridged)

In recent years, machine learning (ML) methods have remarkably improved how cosmologists can interpret data. The next decade will bring new opportunities for data-driven cosmological discovery, but will also present new challenges for adopting ML methodologies and understanding the results. ML could transform our field, but this transformation will require the astronomy community to both foster and promote interdisciplinary research endeavors.

We measure cosmic weak lensing shear power spectra with the Subaru Hyper Suprime-Cam (HSC) survey first-year shear catalog covering 137deg$^2$ of the sky. Thanks to the high effective galaxy number density of $\sim$17 arcmin$^{-2}$ even after conservative cuts such as magnitude cut of $i<24.5$ and photometric redshift cut of $0.3\leq z \leq 1.5$, we obtain a high significance measurement of the cosmic shear power spectra in 4 tomographic redshift bins, achieving a total signal-to-noise ratio of 16 in the multipole range $300 \leq \ell \leq 1900$. We carefully account for various uncertainties in our analysis including the intrinsic alignment of galaxies, scatters and biases in photometric redshifts, residual uncertainties in the shear measurement, and modeling of the matter power spectrum. The accuracy of our power spectrum measurement method as well as our analytic model of the covariance matrix are tested against realistic mock shear catalogs. For a flat $\Lambda$ cold dark matter ($\Lambda$CDM) model, we find $S_8\equiv \sigma_8(\Omega_{\rm m}/0.3)^\alpha=0.800^{+0.029}_{-0.028}$ for $\alpha=0.45$ ($S_8=0.780^{+0.030}_{-0.033}$ for $\alpha=0.5$) from our HSC tomographic cosmic shear analysis alone. In comparison with Planck cosmic microwave background constraints, our results prefer slightly lower values of $S_8$, although metrics such as the Bayesian evidence ratio test do not show significant evidence for discordance between these results. We study the effect of possible additional systematic errors that are unaccounted in our fiducial cosmic shear analysis, and find that they can shift the best-fit values of $S_8$ by up to $\sim 0.6\sigma$ in both directions. The full HSC survey data will contain several times more area, and will lead to significantly improved cosmological constraints.

Mapping the underlying density field, including non-visible dark matter, using weak gravitational lensing measurements is now a standard tool in cosmology. Due to its importance to the science results of current and upcoming surveys, the quality of the convergence reconstruction methods should be well understood. We compare three methods: Kaiser-Squires (KS), Wiener filter, and GLIMPSE. KS is a direct inversion, not accounting for survey masks or noise. The Wiener filter is well-motivated for Gaussian density fields in a Bayesian framework. GLIMPSE uses sparsity, aiming to reconstruct non-linearities in the density field. We compare these methods with several tests using public Dark Energy Survey (DES) Science Verification (SV) data and realistic DES simulations. The Wiener filter and GLIMPSE offer substantial improvements over smoothed KS with a range of metrics. Both the Wiener filter and GLIMPSE convergence reconstructions show a 12 per cent improvement in Pearson correlation with the underlying truth from simulations. To compare the mapping methods' abilities to find mass peaks, we measure the difference between peak counts from simulated {\Lambda}CDM shear catalogues and catalogues with no mass fluctuations (a standard data vector when inferring cosmology from peak statistics); the maximum signal-to-noise of these peak statistics is increased by a factor of 3.5 for the Wiener filter and 9 for GLIMPSE. With simulations we measure the reconstruction of the harmonic phases; the phase residuals' concentration is improved 17 per cent by GLIMPSE and 18 per cent by the Wiener filter. The correlation between reconstructions from data and foreground redMaPPer clusters is increased 18 per cent by the Wiener filter and 32 per cent by GLIMPSE.

We present results from a set of simulations designed to constrain the weak lensing shear calibration for the Hyper Suprime-Cam (HSC) survey. These simulations include HSC observing conditions and galaxy images from the Hubble Space Telescope (HST), with fully realistic galaxy morphologies and the impact of nearby galaxies included. We find that the inclusion of nearby galaxies in the images is critical to reproducing the observed distributions of galaxy sizes and magnitudes, due to the non-negligible fraction of unrecognized blends in ground-based data, even with the excellent typical seeing of the HSC survey (0.58" in the $i$-band). Using these simulations, we detect and remove the impact of selection biases due to the correlation of weights and the quantities used to define the sample (S/N and apparent size) with the lensing shear. We quantify and remove galaxy property-dependent multiplicative and additive shear biases that are intrinsic to our shear estimation method, including a $\sim 10$ per cent-level multiplicative bias due to the impact of nearby galaxies and unrecognized blends. Finally, we check the sensitivity of our shear calibration estimates to other cuts made on the simulated samples, and find that the changes in shear calibration are well within the requirements for HSC weak lensing analysis. Overall, the simulations suggest that the weak lensing multiplicative biases in the first-year HSC shear catalog are controlled at the 1 per cent level.

The use of high-quality simulated sky catalogs is essential for the success of cosmological surveys. The catalogs have diverse applications, such as investigating signatures of fundamental physics in cosmological observables, understanding the effect of systematic uncertainties on measured signals and testing mitigation strategies for reducing these uncertainties, aiding analysis pipeline development and testing, and survey strategy optimization. The list of applications is growing with improvements in the quality of the catalogs and the details that they can provide. Given the importance of simulated catalogs, it is critical to provide rigorous validation protocols that enable both catalog providers and users to assess the quality of the catalogs in a straightforward and comprehensive way. For this purpose, we have developed the DESCQA framework for the Large Synoptic Survey Telescope Dark Energy Science Collaboration as well as for the broader community. The goal of DESCQA is to enable the inspection, validation, and comparison of an inhomogeneous set of synthetic catalogs via the provision of a common interface within an automated framework. In this paper, we present the design concept and first implementation of DESCQA. In order to establish and demonstrate its full functionality we use a set of interim catalogs and validation tests. We highlight several important aspects, both technical and scientific, that require thoughtful consideration when designing a validation framework, including validation metrics and how these metrics impose requirements on the synthetic sky catalogs.

We study the clustering of the highest-z galaxies (from ~ $0.1$ to a few tens Mpc scales) using the BLUETIDES simulation and compare it to current observational constraints from Hubble legacy and Hyper Suprime Cam (HSC) fields (at $z=6-7.2$). With a box length of $400$ $Mpc/h$ on each side and $0.7$ trillion particles, BLUETIDES is the largest high resolution cosmological hydrodynamic simulation to date ideally suited for studies of high-z galaxies. We find that galaxies with magnitude $m_{UV}<27.7$ have a bias ($b_g$) of $8.1\pm 1.2$ at $z=8$, and typical halo masses $M_H \gtrsim 6\times10^{10} M_{\odot}$. Given the redshift evolution between $z=8$ to $z=10$ ($b_g\propto(1+z)^{1.6}$), our inferred values of the bias and halo masses are consistent with measured angular clustering at $z \sim 6.8$ from these brighter samples. The bias of fainter galaxies (in the Hubble legacy field at $H_{160} \lesssim29.5$) is $5.9\pm0.9$ at $z=8$ corresponding to halo masses $M_H \gtrsim 10^{10} M_{\odot}$. We investigate directly the 1-halo term inthe clustering and show that it dominates on scales $r \lesssim 0.1$ Mpc/$h$ ($\Theta \lesssim 3"$) with non-linear effect at transition scales between the 1-halo and 2-halo term affecting scales 0.1 $\lesssim r \lesssim $ 20 Mpc/$h$ ($3"\lesssim \Theta \lesssim 90"$). Current clustering measurements probe down to the scales in the transition between 1-halo to 2-halo regime where non-linear effects are important. The amplitude of the 1-halo term implies that occupation numbers for satellites in \texttt{BLUETIDES} are somewhat higher than standard HODs adopted in these analyses (which predict amplitudes in the 1-halo regime suppressed by a factor 2-3). That possibly implies a higher number of galaxies detected by JWST (at small scales and even fainter magnitudes) observing these fields.

We present and characterize the catalog of galaxy shape measurements that will be used for cosmological weak lensing measurements in the Wide layer of the first year of the Hyper Suprime-Cam (HSC) survey. The catalog covers an area of 136.9 deg$^2$ split into six fields, with a mean $i$-band seeing of $0.58$ arcsec and $5\sigma$ point-source depth of $i\sim 26$. Given conservative galaxy selection criteria for first year science, the depth and excellent image quality results in unweighted and weighted source number densities of 24.6 and 21.8 arcmin$^{-2}$, respectively. We define the requirements for cosmological weak lensing science with this catalog, then focus on characterizing potential systematics in the catalog using a series of internal null tests for problems with point-spread function (PSF) modeling, shear estimation, and other aspects of the image processing. We find that the PSF models narrowly meet requirements for weak lensing science with this catalog, with fractional PSF model size residuals of approximately $0.003$ (requirement: 0.004) and the PSF model shape correlation function $\rho_1<3\times 10^{-7}$ (requirement: $4\times 10^{-7}$) at 0.5$^\circ$ scales. A variety of galaxy shape-related null tests are statistically consistent with zero, but star-galaxy shape correlations reveal additive systematics on $>1^\circ$ scales that are sufficiently large as to require mitigation in cosmic shear measurements. Finally, we discuss the dominant systematics and the planned algorithmic changes to reduce them in future data reductions.

Merging galaxy clusters present a unique opportunity to study the properties of dark matter in an astrophysical context. These are rare and extreme cosmic events in which the bulk of the baryonic matter becomes displaced from the dark matter halos of the colliding subclusters. Since all mass bends light, weak gravitational lensing is a primary tool to study the total mass distribution in such systems. Combined with X-ray and optical analyses, mass maps of cluster mergers reconstructed from weak-lensing observations have been used to constrain the self-interaction cross-section of dark matter. The dynamically complex Abell 520 (A520) cluster is an exceptional case, even among merging systems: multi-wavelength observations have revealed a surprising high mass-to-light concentration of dark mass, the interpretation of which is difficult under the standard assumption of effectively collisionless dark matter. We revisit A520 using a new sparsity-based mass-mapping algorithm to independently assess the presence of the puzzling dark core. We obtain high-resolution mass reconstructions from two separate galaxy shape catalogs derived from Hubble Space Telescope observations of the system. Our mass maps agree well overall with the results of previous studies, but we find important differences. In particular, although we are able to identify the dark core at a certain level in both data sets, it is at much lower significance than has been reported before using the same data. As we cannot confirm the detection in our analysis, we do not consider A520 as posing a significant challenge to the collisionless dark matter scenario.

Galaxy-scale strong gravitational lensing is not only a valuable probe of the dark matter distribution of massive galaxies, but can also provide valuable cosmological constraints, either by studying the population of strong lenses or by measuring time delays in lensed quasars. Due to the rarity of galaxy-scale strongly lensed systems, fast and reliable automated lens finding methods will be essential in the era of large surveys such as LSST, Euclid, and WFIRST. To tackle this challenge, we introduce CMU DeepLens, a new fully automated galaxy-galaxy lens finding method based on Deep Learning. This supervised machine learning approach does not require any tuning after the training step which only requires realistic image simulations of strongly lensed systems. We train and validate our model on a set of 20,000 LSST-like mock observations including a range of lensed systems of various sizes and signal-to-noise ratios (S/N). We find on our simulated data set that for a rejection rate of non-lenses of 99%, a completeness of 90% can be achieved for lenses with Einstein radii larger than 1.4" and S/N larger than 20 on individual $g$-band LSST exposures. Finally, we emphasize the importance of realistically complex simulations for training such machine learning methods by demonstrating that the performance of models of significantly different complexities cannot be distinguished on simpler simulations. We make our code publicly available at https://github.com/McWilliamsCenter/CMUDeepLens .

Peak statistics in weak lensing maps access the non-Gaussian information contained in the large-scale distribution of matter in the Universe. They are therefore a promising complement to two-point and higher-order statistics to constrain our cosmological models. To prepare for the high-precision data of next-generation surveys, we assess the constraining power of peak counts in a simulated Euclid-like survey on the cosmological parameters $\Omega_\mathrm{m}$, $\sigma_8$, and $w_0^\mathrm{de}$. In particular, we study how the Camelus model--a fast stochastic algorithm for predicting peaks--can be applied to such large surveys. We measure the peak count abundance in a mock shear catalogue of ~5,000 sq. deg. using a multiscale mass map filtering technique. We then constrain the parameters of the mock survey using Camelus combined with approximate Bayesian computation (ABC). We find that peak statistics yield a tight but significantly biased constraint in the $\sigma_8$-$\Omega_\mathrm{m}$ plane, indicating the need to better understand and control the model's systematics. We calibrate the model to remove the bias and compare results to those from the two-point correlation functions (2PCF) measured on the same field. In this case, we find the derived parameter $\Sigma_8=\sigma_8(\Omega_\mathrm{m}/0.27)^\alpha=0.76_{-0.03}^{+0.02}$ with $\alpha=0.65$ for peaks, while for 2PCF the value is $\Sigma_8=0.76_{-0.01}^{+0.02}$ with $\alpha=0.70$. We therefore see comparable constraining power between the two probes, and the offset of their $\sigma_8$-$\Omega_\mathrm{m}$ degeneracy directions suggests that a combined analysis would yield tighter constraints than either measure alone. As expected, $w_0^\mathrm{de}$ cannot be well constrained without a tomographic analysis, but its degeneracy directions with the other two varied parameters are still clear for both peaks and 2PCF. (abridged)

Understanding the nature of dark energy, the mysterious force driving the accelerated expansion of the Universe, is a major challenge of modern cosmology. The next generation of cosmological surveys, specifically designed to address this issue, rely on accurate measurements of the apparent shapes of distant galaxies. However, shape measurement methods suffer from various unavoidable biases and therefore will rely on a precise calibration to meet the accuracy requirements of the science analysis. This calibration process remains an open challenge as it requires large sets of high quality galaxy images. To this end, we study the application of deep conditional generative models in generating realistic galaxy images. In particular we consider variations on conditional variational autoencoder and introduce a new adversarial objective for training of conditional generative networks. Our results suggest a reliable alternative to the acquisition of expensive high quality observations for generating the calibration data needed by the next generation of cosmological surveys.

We propose a new mass-mapping algorithm, specifically designed to recover small-scale information from a combination of gravitational shear and flexion. Including flexion allows us to supplement the shear on small scales in order to increase the sensitivity to substructures and the overall resolution of the convergence map without relying on strong lensing constraints. In order to preserve all available small scale information, we avoid any binning of the irregularly sampled input shear and flexion fields and treat the mass-mapping problem as a general ill-posed inverse problem, regularised using a robust multi-scale wavelet sparsity prior. The resulting algorithm incorporates redshift, reduced shear, and reduced flexion measurements for individual galaxies and is made highly efficient by the use of fast Fourier estimators. We test our reconstruction method on a set of realistic weak lensing simulations corresponding to typical HST/ACS cluster observations and demonstrate our ability to recover substructures with the inclusion of flexion which are lost if only shear information is used. In particular, we can detect substructures at the 15$^{\prime \prime}$ scale well outside of the critical region of the clusters. In addition, flexion also helps to constrain the shape of the central regions of the main dark matter halos. Our mass-mapping software, called Glimpse2D, is made freely available at http://www.cosmostat.org/software/glimpse .

Detection of supernovae and, more generally, of transient events in large surveys can provide numerous false detections.In the case of a deferred processing of survey images, this implies reconstructing complete light curves for all detections, requiring sizable processing time and resources.Optimizing the detection of transient events is thus an important issue for both present and future surveys.We present here the optimization done in the SuperNova Legacy Survey (SNLS) for the 5-year data deferred photometric analysis. In this analysis, detections are derived from stacks of subtracted images with one stack per lunation.The 3-year analysis provided 300,000 detections dominated by signals of bright objects that were not perfectly subtracted.Allowing these artifacts to be detected leads not only to a waste of resources but also to possible signal coordinate contamination.We developed a subtracted image stack treatment to reduce the number of non SN-like events using morphological component analysis.This technique exploits the morphological diversity of objects to be detected to extract the signal of interest.At the level of our subtraction stacks, SN-like events are rather circular objects while most spurious detections exhibit different shapes.A two-step procedure was necessary to have a proper evaluation of the noise in the subtracted image stacks and thus a reliable signal extraction.We also set up a new detection strategy to obtain coordinates with good resolution for the extracted signal.SNIa MC generated images were used to study detection efficiency and coordinate resolution.When tested on SNLS 3 data this procedure decreases the number of detections by a factor of two, while losing only 10% of SN-like events, almost all faint.MC results show that SNIa detection efficiency is equivalent to that of the original method for bright events, while the coordinate resolution is improved.

Upcoming spectroscopic galaxy surveys are extremely promising to help in addressing the major challenges of cosmology, in particular in understanding the nature of the dark universe. The strength of these surveys comes from their unprecedented depth and width. Optimal extraction of their three-dimensional information is of utmost importance to best constrain the properties of the dark universe. Although there is theoretical motivation and novel tools to explore these surveys using the 3D spherical Fourier-Bessel (SFB) power spectrum of galaxy number counts $C_\ell(k,k^\prime)$, most survey optimisations and forecasts are based on the tomographic spherical harmonics power spectrum $C^{(ij)}_\ell$. We performed a new investigation of the information that can be extracted from the tomographic and 3D SFB techniques by comparing the forecast cosmological parameter constraints obtained from a Fisher analysis in the context of planned stage IV wide-field galaxy surveys. The comparison was made possible by careful and coherent treatment of non-linear scales in the two analyses. Nuisance parameters related to a scale- and redshift-dependent galaxy bias were also included for the first time in the computation of both the 3D SFB and tomographic power spectra. Tomographic and 3D SFB methods can recover similar constraints in the absence of systematics. However, constraints from the 3D SFB analysis are less sensitive to unavoidable systematics stemming from a redshift- and scale-dependent galaxy bias. Even for surveys that are optimised with tomography in mind, a 3D SFB analysis is more powerful. In addition, for survey optimisation, the figure of merit for the 3D SFB method increases more rapidly with redshift, especially at higher redshifts, suggesting that the 3D SFB method should be preferred for designing and analysing future wide-field spectroscopic surveys.

We compare the efficiency with which 2D and 3D weak lensing mass mapping techniques are able to detect clusters of galaxies using two state-of-the-art mass reconstruction techniques: MRLens in 2D and GLIMPSE in 3D. We simulate otherwise-empty cluster fields for 96 different virial mass-redshift combinations spanning the ranges $3\times10^{13}h^{-1}M_\odot \le M_{vir}\le 10^{15}h^{-1}M_\odot$ and $0.05 \le z_{\rm cl} \le 0.75$, and for each generate 1000 realisations of noisy shear data in 2D and 3D. For each field, we then compute the cluster (false) detection rate as the mean number of cluster (false) detections per reconstruction over the sample of 1000 reconstructions. We show that both MRLens and GLIMPSE are effective tools for the detection of clusters from weak lensing measurements, and provide comparable quality reconstructions at low redshift. At high redshift, GLIMPSE reconstructions offer increased sensitivity in the detection of clusters, yielding cluster detection rates up to a factor of $\sim 10\times$ that seen in 2D reconstructions using MRLens. We conclude that 3D mass mapping techniques are more efficient for the detection of clusters of galaxies in weak lensing surveys than 2D methods, particularly since 3D reconstructions yield unbiased estimators of both the mass and redshift of the detected clusters directly.

The primordial power spectrum describes the initial perturbations that seeded the large-scale structure we observe today. It provides an indirect probe of inflation or other structure-formation mechanisms. In this letter, we recover the primordial power spectrum from the Planck PR1 dataset, using our recently published algorithm PRISM. PRISM is a sparsity-based inversion method, that aims at recovering features in the primordial power spectrum from the empirical power spectrum of the cosmic microwave background (CMB). This ill-posed inverse problem is regularised using a sparsity prior on features in the primordial power spectrum in a wavelet dictionary. Although this non-parametric method does not assume a strong prior on the shape of the primordial power spectrum, it is able to recover both its general shape and localised features. As a results, this approach presents a reliable way of detecting deviations from the currently favoured scale-invariant spectrum. We applied PRISM to 100 simulated Planck data to investigate its performance on Planck-like data. We also tested the algorithm's ability to recover a small localised feature at $k \sim 0.125$ Mpc$^{-1}$, which caused a large dip at $\ell \sim 1800$ in the angular power spectrum. We then applied PRISM to the Planck PR1 power spectrum to recover the primordial power spectrum. We find no significant departures from the fiducial Planck PR1 near scale-invariant primordial power spectrum with $A_s=2.215\times10^{-9}$ and $n_s = 0.9624$.

The primordial power spectrum is an indirect probe of inflation or other structure-formation mechanisms. We introduce a new method, named \textbf{PRISM}, to estimate this spectrum from the empirical cosmic microwave background (CMB) power spectrum. This is a sparsity-based inversion method, which leverages a sparsity prior on features in the primordial spectrum in a wavelet dictionary to regularise the inverse problem. This non-parametric approach is able to reconstruct the global shape as well as localised features of spectrum accurately and proves to be robust for detecting deviations from the currently favoured scale-invariant spectrum. We investigate the strength of this method on a set of WMAP nine-year simulated data for three types of primordial spectra and then process the WMAP nine-year data as well as the Planck PR1 data. We find no significant departures from a near scale-invariant spectrum.

The primordial power spectrum describes the initial perturbations in the Universe which eventually grew into the large-scale structure we observe today, and thereby provides an indirect probe of inflation or other structure-formation mechanisms. Here, we introduce a new method to estimate this spectrum from the empirical power spectrum of cosmic microwave background (CMB) maps. A sparsity-based linear inversion method, coined \textbf{PRISM}, is presented. This technique leverages a sparsity prior on features in the primordial power spectrum in a wavelet basis to regularise the inverse problem. This non-parametric approach does not assume a strong prior on the shape of the primordial power spectrum, yet is able to correctly reconstruct its global shape as well as localised features. These advantages make this method robust for detecting deviations from the currently favoured scale-invariant spectrum. We investigate the strength of this method on a set of WMAP 9-year simulated data for three types of primordial power spectra: a nearly scale-invariant spectrum, a spectrum with a small running of the spectral index, and a spectrum with a localised feature. This technique proves to easily detect deviations from a pure scale-invariant power spectrum and is suitable for distinguishing between simple models of the inflation. We process the WMAP 9-year data and find no significant departure from a nearly scale-invariant power spectrum with the spectral index $n_s = 0.972$. A high resolution primordial power spectrum can be reconstructed with this technique, where any strong local deviations or small global deviations from a pure scale-invariant spectrum can easily be detected.

We present GLIMPSE - Gravitational Lensing Inversion and MaPping with Sparse Estimators - a new algorithm to generate density reconstructions in three dimensions from photometric weak lensing measurements. This is an extension of earlier work in one dimension aimed at applying compressive sensing theory to the inversion of gravitational lensing measurements to recover 3D density maps. Using the assumption that the density can be represented sparsely in our chosen basis - 2D transverse wavelets and 1D line of sight dirac functions - we show that clusters of galaxies can be identified and accurately localised and characterised using this method. Throughout, we use simulated data consistent with the quality currently attainable in large surveys. We present a thorough statistical analysis of the errors and biases in both the redshifts of detected structures and their amplitudes. The GLIMPSE method is able to produce reconstructions at significantly higher resolution than the input data; in this paper we show reconstructions with 6x finer redshift resolution than the shear data. Considering cluster simulations with 0.05 <= z <= 0.75 and 3e13/h Msun <= Mvir <= 1e15/h Msun, we show that the redshift extent of detected peaks is typically 1-2 pixels, or Dz <~ 0.07, and that we are able to recover an unbiased estimator of the redshift of a detected cluster by considering many realisations of the noise. We also recover an accurate estimator of the mass, that is largely unbiased when the redshift is known, and whose bias is constrained to <~ 5% in the majority of our simulations when the estimated redshift is taken to be the true redshift. This shows a substantial improvement over earlier 3D inversion methods, which showed redshift smearing with a typical standard deviation of 0.2-0.3, a significant damping of the amplitude of the peaks detected, and a bias in the detected redshift.

Future cosmological surveys will provide 3D large scale structure maps with large sky coverage, for which a 3D Spherical Fourier-Bessel (SFB) analysis in spherical coordinates is natural. Wavelets are particularly well-suited to the analysis and denoising of cosmological data, but a spherical 3D isotropic wavelet transform does not currently exist to analyse spherical 3D data. The aim of this paper is to present a new formalism for a spherical 3D isotropic wavelet, i.e. one based on the SFB decomposition of a 3D field and accompany the formalism with a public code to perform wavelet transforms. We describe a new 3D isotropic spherical wavelet decomposition based on the undecimated wavelet transform (UWT) described in Starck et al. 2006. We also present a new fast Discrete Spherical Fourier-Bessel Transform (DSFBT) based on both a discrete Bessel Transform and the HEALPIX angular pixelisation scheme. We test the 3D wavelet transform and as a toy-application, apply a denoising algorithm in wavelet space to the Virgo large box cosmological simulations and find we can successfully remove noise without much loss to the large scale structure. We have described a new spherical 3D isotropic wavelet transform, ideally suited to analyse and denoise future 3D spherical cosmological surveys, which uses a novel Discrete Spherical Fourier-Bessel Transform. We illustrate its potential use for denoising using a toy model. All the algorithms presented in this paper are available for download as a public code called MRS3D at http://jstarck.free.fr/mrs3d.html