We present the GEOMAX algorithm and its Python implementation for a two-step compression of bispectrum measurements. The first step groups bispectra by the geometric properties of their arguments; the second step then maximises the Fisher information with respect to a chosen set of model parameters in each group. The algorithm only requires the derivatives of the data vector with respect to the parameters and a small number of mock data, producing an effective, non-linear compression. By applying GEOMAX to bispectrum monopole measurements from BOSS DR12 CMASS redshift-space galaxy clustering data, we reduce the $68\%$ credible intervals for the inferred parameters $\left(b_1,b_2,f,\sigma_8\right)$ by $\left(50.4\%,56.1\%,33.2\%,38.3\%\right)$ with respect to standard MCMC on the full data vector. We run the analysis and comparison between compression methods over one hundred galaxy mocks to test the statistical significance of the improvements. On average GEOMAX performs $\sim15\%$ better than geometrical or maximal linear compression alone and is consistent with being lossless. Given its flexibility, the GEOMAX approach has the potential to optimally exploit three-point statistics of various cosmological probes like weak lensing or line-intensity maps from current and future cosmological data-sets such as DESI, Euclid, PFS and SKA.

Measuring cosmic shear in wide-field imaging surveys requires accurate knowledge of the redshift distribution of all sources. The clustering-redshift technique exploits the angular cross-correlation of a target galaxy sample with unknown redshifts and a reference sample with known redshifts, and is an attractive alternative to colour-based methods of redshift calibration. We test the performance of such clustering redshift measurements using mock catalogues that resemble the Kilo-Degree Survey (KiDS). These mocks are created from the MICE simulation and closely mimic the properties of the KiDS source sample and the overlapping spectroscopic reference samples. We quantify the performance of the clustering redshifts by comparing the cross-correlation results with the true redshift distributions in each of the five KiDS photometric redshift bins. Such a comparison to an informative model is necessary due to the incompleteness of the reference samples at high redshifts. Clustering mean redshifts are unbiased at $|\Delta z|<0.006$ under these conditions. The redshift evolution of the galaxy bias can be reliably mitigated at this level of precision using auto-correlation measurements and self-consistency relations, and will not become a dominant source of systematic error until the arrival of Stage-IV cosmic shear surveys. Using redshift distributions from a direct colour-based estimate instead of the true redshift distributions as a model for comparison with the clustering redshifts increases the biases in the mean to up to $|\Delta z|\sim0.04$. This indicates that the interpretation of clustering redshifts in real-world applications will require more sophisticated (parameterised) models of the redshift distribution in the future. If such better models are available, the clustering-redshift technique promises to be a highly complementary alternative to other methods of redshift calibration.

We present weak lensing shear catalogues from the fourth data release of the Kilo-Degree Survey, KiDS-1000, spanning 1006 square degrees of deep and high-resolution imaging. Our `gold-sample' of galaxies, with well calibrated photometric redshift distributions, consists of 21 million galaxies with an effective number density of $\sim 6.2$ galaxies per square arcminute. We quantify the accuracy of the spatial, temporal and flux-dependent point-spread function (PSF) model, verifying that the model meets our requirements to induce less than a $0.1\sigma$ change in the inferred cosmic shear constraints on the clustering cosmological parameter $S_8 = \sigma_8\sqrt{\Omega_{\rm m}/0.3}$. Through a series of two-point null-tests we validate the shear estimates, finding no evidence for significant non-lensing B-mode distortions in the data. PSF residuals are detected in the highest-redshift bins, originating from object selection and/or weight bias. The amplitude is however shown to be sufficiently low and within our stringent requirements. With a shear-ratio null-test we verify the expected redshift scaling of the galaxy-galaxy lensing signal around luminous red galaxies. We conclude that the joint KiDS-1000 shear and photometric redshift calibration is sufficiently robust for combined-probe gravitational lensing and spectroscopic clustering analyses.

We present the methodology for a joint cosmological analysis of weak gravitational lensing from the fourth data release of the ESO Kilo-Degree Survey (KiDS-1000) and galaxy clustering from the partially overlapping BOSS and 2dFLenS surveys. Cross-correlations between galaxy positions and ellipticities are incorporated into the analysis, developing a hybrid model of non-linear scales that blends perturbative and non-perturbative approaches, and assessing signal contributions by astrophysical effects. All weak lensing signals are measured consistently via Fourier-space statistics that are insensitive to the survey mask and display low levels of mode mixing. The calibration of photometric redshift distributions and multiplicative gravitational shear bias is updated, and a more complete tally of residual calibration uncertainties is propagated into the likelihood. A dedicated suite of more than 20000 mocks is used to assess the performance of covariance models and to quantify the impact of survey geometry and spatial variations of survey depth on signals and their errors. The sampling distributions for the likelihood and the $\chi^2$ goodness-of-fit statistic are validated, with proposed changes to calculating the effective number of degrees of freedom. Standard weak lensing point estimates on $S_8=\sigma_8\,(\Omega_{\rm m}/0.3)^{1/2}$ derived from its marginal posterior are easily misinterpreted to be biased low, and an alternative estimator and associated credible interval are proposed. Known systematic effects pertaining to weak lensing modelling and inference are shown to bias $S_8$ by no more than 0.1 standard deviations, with the caveat that no conclusive validation data exist for models of intrinsic galaxy alignments. Compared to the previous KiDS analyses, $S_8$ constraints are expected to improve by 20% for weak lensing alone and by 29% for the joint analysis. [abridged]

The physics of gravity on cosmological scales affects both the rate of assembly of large-scale structure, and the gravitational lensing of background light through this cosmic web. By comparing the amplitude of these different observational signatures, we can construct tests that can distinguish General Relativity from its potential modifications. We use the latest weak gravitational lensing dataset from the Kilo-Degree Survey, KiDS-1000, in conjunction with overlapping galaxy spectroscopic redshift surveys BOSS and 2dFLenS, to perform the most precise existing amplitude-ratio test. We measure the associated E_G statistic with 15-20% errors, in five dz = 0.1 tomographic redshift bins in the range 0.2 < z < 0.7, on projected scales up to 100 Mpc/h. The scale-independence and redshift-dependence of these measurements are consistent with the theoretical expectation of General Relativity in a Universe with matter density Omega_m = 0.27 +/- 0.04. We demonstrate that our results are robust against different analysis choices, including schemes for correcting the effects of source photometric redshift errors, and compare the performance of angular and projected galaxy-galaxy lensing statistics.

We present updated cosmological constraints for the KiDS+VIKING-450 cosmic shear dataset (KV450), estimated using redshift distributions and photometric samples defined using self organising maps (SOMs). Our fiducial analysis finds marginal posterior constraints of $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.716^{+0.043}_{-0.038}$; smaller than, but fully consistent with, previous work using this dataset ($|\Delta S_8| = 0.023$). We analyse additional samples and redshift distributions constructed in three ways: excluding certain spectroscopic surveys during redshift calibration, excluding lower-confidence spectroscopic redshifts in redshift calibration, and considering only photometric sources which are jointly calibrated by at least three spectroscopic surveys. In all cases, the method utilised here proves robust: we find a maximal deviation from our fiducial analysis of $|\Delta S_8| \leq 0.009$ for all samples defined and analysed using our SOM. Our largest shift in $S_8$ is found when calibrating redshift distributions without the DEEP2 spectroscopic subset, where we find $S_8=0.707_{-0.042}^{+0.046}$. This difference with respect to the fiducial is both significantly smaller than, and in the opposite direction to, the equivalent shift from previous work. No sample analysed in this work results in a meaningful positive shift in $S_8$ with respect to our fiducial constraints. These results suggest that our improved cosmological parameter estimates are insensitive to pathological misrepresentation of photometric sources by the spectroscopy used for direct redshift calibration, and therefore that this systematic effect cannot be responsible for the observed difference between $S_8$ estimates made with KV450 and Planck CMB probes.

We present a combined tomographic weak gravitational lensing analysis of the Kilo Degree Survey (KV450) and the Dark Energy Survey (DES-Y1). We homogenize the analysis of these two public cosmic shear datasets by adopting consistent priors and modeling of nonlinear scales, and determine new redshift distributions for DES-Y1 based on deep public spectroscopic surveys. Adopting these revised redshifts results in a $0.8\sigma$ reduction in the DES-inferred value for $S_8$, which decreases to a $0.5\sigma$ reduction when including a systematic redshift calibration error model from mock DES data based on the MICE2 simulation. The combined KV450 + DES-Y1 constraint on $S_8 = 0.762^{+0.025}_{-0.024}$ is in tension with the Planck 2018 constraint from the cosmic microwave background at the level of $2.5\sigma$. This result highlights the importance of developing methods to provide accurate redshift calibration for current and future weak lensing surveys.

We present a mitigation strategy to reduce the impact of non-linear galaxy bias on the joint `$3 \times 2 $pt' cosmological analysis of weak lensing and galaxy surveys. The $\Psi$-statistics that we adopt are based on Complete Orthogonal Sets of E/B Integrals (COSEBIs). As such they are designed to minimise the contributions to the observable from the smallest physical scales where models are highly uncertain. We demonstrate that $\Psi$-statistics carry the same constraining power as the standard two-point galaxy clustering and galaxy-galaxy lensing statistics, but are significantly less sensitive to scale-dependent galaxy bias. Using two galaxy bias models, motivated by halo-model fits to data and simulations, we quantify the error in a standard $3 \times 2$pt analysis where constant galaxy bias is assumed. Even when adopting conservative angular scale cuts, that degrade the overall cosmological parameter constraints, we find of order $1 \sigma$ biases for Stage III surveys on the cosmological parameter $S_8 = \sigma_8(\Omega_{\rm m}/0.3)^{\alpha}$. This arises from a leakage of the smallest physical scales to all angular scales in the standard two-point correlation functions. In contrast, when analysing $\Psi$-statistics under the same approximation of constant galaxy bias, we show that the bias on the recovered value for $S_8$ can be decreased by a factor of $\sim 2$, with less conservative scale cuts. Given the challenges in determining accurate galaxy bias models in the highly non-linear regime, we argue that $3 \times 2$pt analyses should move towards new statistics that are less sensitive to the smallest physical scales.

In this paper we introduce the \textsc{Deepz} deep learning photometric redshift (photo-$z$) code. As a test case, we apply the code to the PAU survey (PAUS) data in the COSMOS field. \textsc{Deepz} reduces the $\sigma_{68}$ scatter statistic by 50\% at $i_{\rm AB}=22.5$ compared to existing algorithms. This improvement is achieved through various methods, including transfer learning from simulations where the training set consists of simulations as well as observations, which reduces the need for training data. The redshift probability distribution is estimated with a mixture density network (MDN), which produces accurate redshift distributions. Our code includes an autoencoder to reduce noise and extract features from the galaxy SEDs. It also benefits from combining multiple networks, which lowers the photo-$z$ scatter by 10 percent. Furthermore, training with randomly constructed coadded fluxes adds information about individual exposures, reducing the impact of photometric outliers. In addition to opening up the route for higher redshift precision with narrow bands, these machine learning techniques can also be valuable for broad-band surveys.

We study and compare fitting methods for the Lyman-$\alpha$ (Ly$\alpha$) forest 3D correlation function. We use the nested sampler PolyChord and the community code picca to perform a Bayesian analysis which we compare with previous frequentist analyses. By studying synthetic correlation functions, we find that the frequentist profile likelihood produces results in good agreement with a full Bayesian analysis. On the other hand, Maximum Likelihood Estimation with the Gaussian approximation for the uncertainties is inadequate for current data sets. We compute for the first time the full posterior distribution from the Ly$\alpha$ forest correlation functions measured by the extended Baryon Oscillation Spectroscopic Survey (eBOSS). We highlight the benefits of sampling the full posterior distribution by expanding the baseline analysis to better understand the contamination by Damped Ly$\alpha$ systems (DLAs). We make our improvements and results publicly available as part of the picca package.

Recent works combining neutrino oscillation and cosmological data to determine the neutrino hierarchy found a range of odds in favour of the normal hierarchy. These results arise from differing approaches to incorporating prior knowledge about neutrinos. We develop a hierarchy-agnostic prior and show that the hierarchy cannot be conclusively determined with current data. The determination of the hierarchy is limited by the neutrino mass scale $\Sigma_{\nu}$ measurement. We obtain a target precision of $\sigma(\Sigma_{\nu}) = 0.014$ eV, necessary for conclusively establishing the normal hierarchy with future data.

Intrinsic alignments (IAs) of galaxies are an important contaminant for cosmic shear studies, but the modelling is complicated by the dependence of the signal on the source galaxy sample. In this paper, we use the halo model formalism to capture this diversity and examine its implications for a Stage III cosmic shear survey. We account for the different IA signatures at large and small scales as well for the different contribution from central/satellite and red/blue galaxies. We inform our model using the most recent observational findings: we include a luminosity dependence at both large and small scales and a radial dependence of the signal within the halo. We predict the impact of the total IA signal on the lensing angular power spectra, including the current uncertainties from the IA best-fits to illustrate the range of possible impact on the lensing signal: the lack of constraints for fainter galaxies is the main source of uncertainty for our predictions of the IA signal. We investigate how well the widely used non-linear alignment model can capture the complexity of the IA signal and find that while for Stage III surveys it is flexible enough, in the case of a Stage IV survey, this can lead to $1\sigma$ bias on $\Omega_m$.

We study the alignments of satellite galaxies, and their anisotropic distribution, with respect to location and orientation of their host central galaxy in MassiveBlack-II and IllustrisTNG simulations. We find that: the shape of the satellite system in halos of mass ($> 10^{13}h^{-1}M_{\odot}$) is well aligned with the shape of the central galaxy at $z=0.06$ with the mean alignment between the major axes being $\sim \Delta \theta = 12^{\circ}$ when compared to a uniform random distribution; that satellite galaxies tend to be anisotropically distributed along the major axis of the central galaxy with a stronger alignment in halos of higher mass or luminosity; and that the satellite distribution is more anisotropic for central galaxies with lower star formation rate, which are spheroidal, and for red central galaxies.Radially we find that satellites tend to be distributed along the major axis of the shape of the stellar component of central galaxies at smaller scales and the dark matter component on larger scales. We find that the dependence of satellite anisotropy on central galaxy properties and the radial distance is similar in both the simulations with a larger amplitude in MassiveBlack-II. The orientation of satellite galaxies tends to point toward the location of the central galaxy at small scales and this correlation decreases with increasing distance, and the amplitude of satellite alignment is higher in high mass halos. However, the projected ellipticities do not exhibit a scale-dependent radial alignment, as has been seen in some observational measurements.

We present cosmological constraints from a joint cosmic shear analysis of the Kilo-Degree Survey (KV450) and the Dark Energy Survey (DES-Y1), conducted using Complete Orthogonal Sets of E/B-Integrals (COSEBIs). With COSEBIs we isolate any B-modes which have a non-cosmic shear origin and demonstrate the robustness of our cosmological E-mode analysis as no significant B-modes are detected. We highlight how COSEBIs are fairly insensitive to the amplitude of the non-linear matter power spectrum at high $k$-scales, mitigating the uncertain impact of baryon feedback in our analysis. COSEBIs, therefore, allow us to utilise additional small-scale information, improving the DES-Y1 joint constraints on $S_8=\sigma_8(\Omega_{\rm m}/0.3)^{0.5}$ and $\Omega_{\rm m}$ by $20\%$. Adopting a flat $\Lambda$CDM model we find $S_8=0.755^{+0.019}_{-0.021}$, which is in $3.2\sigma$ tension with the Planck Legacy analysis of the cosmic microwave background.

We reanalyse the anisotropic galaxy clustering measurement from the Baryon Oscillation Spectroscopic Survey (BOSS), demonstrating that using the full shape information provides cosmological constraints that are comparable to other low-redshift probes. We find $\Omega_\mathrm{m} = 0.317^{+0.015}_{-0.019}$, $\sigma_8 = 0.710\pm 0.049$, and $h = 0.704\pm 0.024$ for flat $\Lambda$CDM cosmologies using uninformative priors on $\Omega_\mathrm{c}h^2$, $100\theta_\mathrm{MC}$, $\ln 10^{10} A_{s}$, and $n_{s}$, and a prior on $\Omega_\mathrm{b}h^2$ that is much wider than current constraints. We quantify the agreement between the Planck 2018 constraints from the cosmic microwave background and BOSS, finding the two data sets to be consistent within a flat $\Lambda$CDM cosmology using the Bayes factor as well as the prior-insensitive suspiciousness statistic. Combining two low-redshift probes, we jointly analyse the clustering of BOSS galaxies with weak lensing measurements from the Kilo-Degree Survey (KV450). The combination of BOSS and KV450 improves the measurement by up to 45%, constraining $\sigma_8 = 0.702\pm 0.029$ and $S_8 = \sigma_8\sqrt{\Omega_\mathrm{m}/0.3} = 0.728\pm 0.026$. Over the full 5D parameter space, the odds in favour of single cosmology describing galaxy clustering, lensing, and the cosmic microwave background are $7\pm2$. The suspiciousness statistic signals a $2.1\pm0.3\sigma$ tension between the combined low-redshift probes and measurements from the cosmic microwave background.

We present a semi-analytic model for the shear two-point correlation function of a cosmic shear survey with non-uniform depth. Ground-based surveys are subject to depth variations that primarily arise through varying atmospheric conditions. For a survey like the Kilo-Degree Survey (KiDS), we find that the measured depth variation increases the amplitude of the observed shear correlation function at the level of a few percent out to degree-scales, relative to the assumed uniform-depth case. The impact on the inferred cosmological parameters is shown to be insignificant for a KiDS-like survey. For next-generation cosmic shear experiments, however, we conclude that variable depth should be accounted for.

We present a tomographic cosmic shear analysis of the Kilo-Degree Survey (KiDS) combined with the VISTA Kilo-Degree Infrared Galaxy Survey (VIKING). This is the first time that a full optical to near-infrared data set has been used for a wide-field cosmological weak lensing experiment. This unprecedented data, spanning $450~$deg$^2$, allows us to improve significantly the estimation of photometric redshifts, such that we are able to include robustly higher-redshift sources for the lensing measurement, and - most importantly - solidify our knowledge of the redshift distributions of the sources. Based on a flat $\Lambda$CDM model we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.737_{-0.036}^{+0.040}$ in a blind analysis from cosmic shear alone. The tension between KiDS cosmic shear and the Planck-Legacy CMB measurements remains in this systematically more robust analysis, with $S_8$ differing by $2.3\sigma$. This result is insensitive to changes in the priors on nuisance parameters for intrinsic alignment, baryon feedback, and neutrino mass. KiDS shear measurements are calibrated with a new, more realistic set of image simulations and no significant B-modes are detected in the survey, indicating that systematic errors are under control. When calibrating our redshift distributions by assuming the 30-band COSMOS-2015 photometric redshifts are correct (following the Dark Energy Survey and the Hyper Suprime-Cam Survey), we find the tension with Planck is alleviated. The robust determination of source redshift distributions remains one of the most challenging aspects for future cosmic shear surveys.

We present constraints on Horndeski gravity from a combined analysis of cosmic shear, galaxy-galaxy lensing and galaxy clustering from $450\,\mathrm{deg}^2$ of the Kilo-Degree Survey (KiDS) and the Galaxy And Mass Assembly (GAMA) survey. The Horndeski class of dark energy/modified gravity models includes the majority of universally coupled extensions to $\Lambda$CDM with one scalar field in addition to the metric. We study the functions of time that fully describe the evolution of linear perturbations in Horndeski gravity. Our results are compatible throughout with a $\Lambda$CDM model. By imposing gravitational wave constraints, we fix the tensor speed excess to zero and consider a subset of models including e.g. quintessence and $f(R)$ theories. Assuming proportionality of the Horndeski functions $\alpha_B$ and $\alpha_M$ (kinetic braiding and the Planck mass run rate, respectively) to the dark energy density fraction $\Omega_{\mathrm{DE}}(a) = 1 - \Omega_{\mathrm{m}}(a)$, we find for the proportionality coefficients $\hat{\alpha}_B = 0.20_{-0.33}^{+0.20} \,$ and $\, \hat{\alpha}_M = 0.25_{-0.29}^{+0.19}$. Our value of $S_8 \equiv \sigma_8 \sqrt{\Omega_{\mathrm{m}}/0.3}$ is in better agreement with the $Planck$ estimate when measured in the enlarged Horndeski parameter space than in a pure $\Lambda$CDM scenario. In our joint three-probe analysis we report a downward shift of the $S_8$ best fit value from the $Planck$ measurement of $\Delta S_8 = 0.016_{-0.046}^{+0.048}$ in Horndeski gravity, compared to $\Delta S_8 = 0.059_{-0.039}^{+0.040}$ in $\Lambda$CDM. Our constraints are robust to the modelling uncertainty of the non-linear matter power spectrum in Horndeski gravity. Our likelihood code for multi-probe analysis in both $\Lambda$CDM and Horndeski gravity is publicly available at http://github.com/alessiospuriomancini/KiDSHorndeski .

We present here the cosmo-SLICS, a new suite of simulations specially designed for the analysis of current and upcoming weak lensing data beyond the standard two-point cosmic shear. We sample the $[\Omega_{\rm m}, \sigma_8, h, w_0]$ parameter space at 25 points organised in a Latin hyper-cube, spanning a range that contains most of the $2\sigma$ posterior distribution from ongoing lensing surveys. At each of these nodes we evolve a pair of $N$-body simulations in which the sampling variance is highly suppressed, and ray-trace the volumes 800 times to further increase the effective sky coverage. We extract a lensing covariance matrix from these pseudo-independent light-cones and show that it closely matches a brute-force construction based on an ensemble of 800 truly independent $N$-body runs. More precisely, a Fisher analysis reveals that both methods yield marginalized two-dimensional constraints that vary by less than 6% in area, a result that holds under different survey specifications and that matches to within 15% the area obtained from an analytical covariance calculation. Extending this comparison with our 25 $w$CDM models, we probe the cosmology dependence of the lensing covariance directly from numerical simulations, reproducing remarkably well the Fisher results from the analytical models at most cosmologies. We demonstrate that varying the cosmology at which the covariance matrix is evaluated in the first place might have an order of magnitude greater impact on the parameter constraints than varying the choice of covariance estimation technique. We present a test case in which we generate fast predictions for both the lensing signal and its associated variance with a flexible Gaussian process regression emulator, achieving an accuracy of a few percent on the former and 10% on the latter.

The PAU Survey (PAUS) is an innovative photometric survey with 40 narrow bands at the William Herschel Telescope (WHT). The narrow bands are spaced at 100\AA\ intervals covering the range 4500\AA\ to 8500\AA\ and, in combination with standard broad bands, enable excellent redshift precision. This paper describes the technique, galaxy templates and additional photometric calibration used to determine early photometric redshifts from PAUS. Using BCNz2, a new photometric redshift code developed for this purpose, we characterise the photometric redshift performance using PAUS data on the COSMOS field. Comparison to secure spectra from zCOSMOS DR3 shows that PAUS achieves $\sigma_{68} /(1+z) = 0.0037$ to $i_{\mathrm{AB}} < 22.5$ when selecting the best 50% of the sources based on a photometric redshift quality cut. Furthermore, a higher photo-z precision ($\sigma_{68}/(1+z) \sim 0.001$) is obtained for a bright and high quality selection, which is driven by the identification of emission lines. We conclude that PAUS meets its design goals, opening up a hitherto uncharted regime of deep, wide, and dense galaxy survey with precise redshifts that will provide unique insights into the formation, evolution and clustering of galaxies, as well as their intrinsic alignments.

We propose a principled Bayesian method for quantifying tension between correlated datasets with wide uninformative parameter priors. This is achieved by extending the Suspiciousness statistic, which is insensitive to priors. Our method uses global summary statistics, and as such it can be used as a diagnostic for internal consistency. We show how our approach can be combined with methods that use parameter space and data space to identify the existing internal discrepancies. As an example, we use it to test the internal consistency of the KiDS-450 data in 4 photometric redshift bins, and to recover controlled internal discrepancies in simulated KiDS data. We propose this as a diagnostic of internal consistency for present and future cosmological surveys, and as a tension metric for data sets that have non-negligible correlation, such as LSST and Euclid.

This paper introduces cosmoDC2, a large synthetic galaxy catalog designed to support precision dark energy science with the Large Synoptic Survey Telescope (LSST). CosmoDC2 is the starting point for the second data challenge (DC2) carried out by the LSST Dark Energy Science Collaboration (LSST DESC). The catalog is based on a trillion-particle, 4.225 Gpc^3 box cosmological N-body simulation, the `Outer Rim' run. It covers 440 deg^2 of sky area to a redshift of z=3 and is complete to a magnitude depth of 28 in the r-band. Each galaxy is characterized by a multitude of properties including stellar mass, morphology, spectral energy distributions, broadband filter magnitudes, host halo information and weak lensing shear. The size and complexity of cosmoDC2 requires an efficient catalog generation methodology; our approach is based on a new hybrid technique that combines data-driven empirical approaches with semi-analytic galaxy modeling. A wide range of observation-based validation tests has been implemented to ensure that cosmoDC2 enables the science goals of the planned LSST DESC DC2 analyses. This paper also represents the official release of the cosmoDC2 data set, including an efficient reader that facilitates interaction with the data.

Intrinsic galaxy alignments are a source of bias for weak lensing measurements as well as a tool for understanding galaxy formation and evolution. In this work, we measure the alignment of shapes of satellite galaxies, in galaxy groups, with respect to the brightest group galaxy (BGG), as well as alignments of the BGG shape with the satellite positions, using the highly complete Galaxy And Mass Assembly (GAMA) spectroscopic survey and deep imaging from the Kilo Degree Survey. We control systematic errors with dedicated image simulations and measure accurate shapes using the DEIMOS shape measurement method. We find a significant satellite radial alignment signal, which vanishes at large separations from the BGG. We do not identify any strong trends of the signal with galaxy absolute magnitude or group mass. The alignment signal is dominated by red satellites. We also find that the outer regions of galaxies are aligned more strongly than their inner regions, by varying the radial weight employed during the shape measurement process. This behaviour is evident for both red and blue satellites. BGGs are also found to be aligned with satellite positions, with this alignment being stronger when considering the innermost satellites, using red BGGs and the shape of the outer region of the BGG. Lastly, we measure the global intrinsic alignment signal in the GAMA sample for two different radial weight functions and find no significant difference.

We directly constrain the non-linear alignment (NLA) model of intrinsic galaxy alignments, analysing the most representative and complete flux-limited sample of spectroscopic galaxies available for cosmic shear surveys. We measure the projected galaxy position-intrinsic shear correlations and the projected galaxy clustering signal using high-resolution imaging from the Kilo Degree Survey (KiDS) overlapping with the GAMA spectroscopic survey, and data from the Sloan Digital Sky Survey. Separating samples by colour, we make no significant detection of blue galaxy alignments, constraining the blue galaxy NLA amplitude $A_{\textrm{IA}}^{\textrm{B}}=0.21^{+0.37}_{-0.36}$ to be consistent with zero. We make robust detections ($\sim9\sigma$) for red galaxies, with $A_{\textrm{IA}}^{\textrm{R}}=3.18^{+0.47}_{-0.46}$, corresponding to a net radial alignment with the galaxy density field, and we find no evidence for any scaling of alignments with galaxy luminosity. We provide informative priors for current and future weak lensing surveys, an improvement over de facto wide priors that allow for unrealistic levels of intrinsic alignment contamination. For a colour-split cosmic shear analysis of the final KiDS survey area, we forecast that our priors will improve the constraining power on $S_{8}$ and the dark energy equation of state $w_{0}$, by up to $62\%$ and $51\%$, respectively. Our results indicate, however, that the modelling of red/blue-split galaxy alignments may be insufficient to describe samples with variable central/satellite galaxy fractions.

We present three tiers of Bayesian consistency tests for the general case of $correlated$ datasets. Building on duplicates of the model parameters assigned to each dataset, these tests range from Bayesian evidence ratios as a global summary statistic, to posterior distributions of model parameter differences, to consistency tests in the data domain derived from posterior predictive distributions. For each test we motivate meaningful threshold criteria for the internal consistency of datasets. Without loss of generality we focus on mutually exclusive, correlated subsets of the same dataset in this work. As an application, we revisit the consistency analysis of the two-point weak lensing shear correlation functions measured from KiDS-450 data. We split this dataset according to large vs. small angular scales, tomographic redshift bin combinations, and estimator type. We do not find any evidence for significant internal tension in the KiDS-450 data, with significances below $3\, \sigma$ in all cases. Software and data used in this analysis can be found at http://kids.strw.leidenuniv.nl/sciencedata.php

The outer regions of galaxies are more susceptible to the tidal interactions that lead to intrinsic alignments of galaxies. The resulting alignment signal may therefore depend on the passband if the colours of galaxies vary spatially. To quantify this, we measured the shapes of galaxies with spectroscopic redshifts from the GAMA survey using deep gri imaging data from the KiloDegree Survey. The performance of the moment-based shape measurement algorithm DEIMOS was assessed using dedicated image simulations, which showed that the ellipticities could be determined with an accuracy better than 1% in all bands. Additional tests for potential systematic errors did not reveal any issues. We measure a significant difference of the alignment signal between the g,r and i-band observations. This difference exceeds the amplitude of the linear alignment model on scales below 2 Mpc/h. Separating the sample into central/satellite and red/blue galaxies, we find that that the difference is dominated by red satellite galaxies.

We present a novel method to compress galaxy clustering three-point statistics and apply it to redshift space galaxy bispectrum monopole measurements from BOSS DR12 CMASS data considering a $k$-space range of $0.03-0.12\,h/\mathrm{Mpc}$. The method consists in binning together bispectra evaluated at sets of wave-numbers forming closed triangles with similar geometrical properties: the area, the cosine of the largest angle and the ratio between the cosines of the remaining two angles. This enables us to increase the number of bispectrum measurements for example by a factor of $23$ over the standard binning (from 116 to 2734 triangles used), which is otherwise limited by the number of mock catalogues available to estimate the covariance matrix needed to derive parameter constraints. The $68\%$ credible intervals for the inferred parameters $\left(b_1,b_2,f,\sigma_8\right)$ are thus reduced by $\left(-39\%,-49\%,-29\%,-22\%\right)$, respectively. We find very good agreement with the posteriors recently obtained by alternative maximal compression methods. This new method does not require the a-priori computation of the data-vector covariance matrix and has the potential to be directly applicable to other three-point statistics (e.g. glaxy clustering, weak gravitational lensing, 21 cm emission line) measured from future surveys such as DESI, Euclid, PFS and SKA.

We apply two compression methods to the galaxy power spectrum monopole/quadrupole and bispectrum monopole measurements from the BOSS DR12 CMASS sample. Both methods reduce the dimension of the original data-vector to the number of cosmological parameters considered, using the Karhunen-Lo\`eve algorithm with an analytic covariance model. In the first case, we infer the posterior through MCMC sampling from the likelihood of the compressed data-vector (MC-KL). The second, faster option, works by first Gaussianising and then orthogonalising the parameter space before the compression; in this option (G-PCA) we only need to run a low-resolution preliminary MCMC sample for the Gaussianization to compute our posterior. Both compression methods accurately reproduce the posterior distributions obtained by standard MCMC sampling on the CMASS dataset for a $k$-space range of $0.03-0.12\,h/\mathrm{Mpc}$. The compression enables us to increase the number of bispectrum measurements by a factor of $\sim 23$ over the standard binning (from 116 to 2734 triangles used), which is otherwise limited by the number of mock catalogues available. This reduces the $68\%$ credible intervals for the parameters $\left(b_1,b_2,f,\sigma_8\right)$ by $\left(-24.8\%,-52.8\%,-26.4\%,-21\%\right)$, respectively. The best-fit values we obtain are $(b_1=2.31\pm0.17,b_2=0.77\pm0.19,$ $f(z_{\mathrm{CMASS}})=0.67\pm0.06,\sigma_8(z_{\mathrm{CMASS}})=0.51\pm0.03)$. Using these methods for future redshift surveys like DESI, Euclid and PFS will drastically reduce the number of simulations needed to compute accurate covariance matrices and will facilitate tighter constraints on cosmological parameters.

We present a new measurement of $E_{\rm G}$, which combines measurements of weak gravitational lensing, real-space galaxy clustering and redshift space distortions. This statistic was proposed as a consistency test of General Relativity (GR) that is insensitive to linear, deterministic galaxy bias and the matter clustering amplitude. We combine deep imaging data from KiDS with overlapping spectroscopy from 2dFLenS, BOSS DR12 and GAMA and find $E_{\rm G}(\overline{z}=0.267)=0.43 \pm 0.13$ (GAMA), $E_{\rm G}(\overline{z}=0.305)=0.27 \pm 0.08$ (LOWZ+2dFLOZ) and $E_{\rm G}(\overline{z}=0.554)=0.26 \pm 0.07$ (CMASS+2dFHIZ). We demonstrate that the existing tension in the value of the matter density parameter hinders the robustness of this statistic as solely a test of GR. We find that our $E_{\rm G}$ measurements, as well as existing ones in the literature, favour a lower matter density cosmology than the Cosmic Microwave Background. For a flat $\Lambda$CDM Universe and assuming GR, we find $\Omega_{\rm m}(z=0)=0.25\pm0.03$. With this paper we publicly release the 2dFLenS dataset at: \url{http://2dflens.swin.edu.au}.

We present a weak gravitational lensing analysis of 815 square degree of $i$-band imaging from the Kilo-Degree Survey (KiDS-$i$-800). In contrast to the deep $r$-band observations, which take priority during excellent seeing conditions and form the primary KiDS dataset (KiDS-$r$-450), the complementary yet shallower KiDS-$i$-800 spans a wide range of observing conditions. The overlapping KiDS-$i$-800 and KiDS-$r$-450 imaging therefore provides a unique opportunity to assess the robustness of weak lensing measurements. In our analysis, we introduce two new `null' tests. The `nulled' two-point shear correlation function uses a matched catalogue to show that the calibrated KiDS-$i$-800 and KiDS-$r$-450 shear measurements agree at the level of $1 \pm 4$\%. We use five galaxy lens samples to determine a `nulled' galaxy-galaxy lensing signal from the full KiDS-$i$-800 and KiDS-$r$-450 surveys and find that the measurements agree to $7 \pm 5$\% when the KiDS-$i$-800 source redshift distribution is calibrated using either spectroscopic redshifts, or the 30-band photometric redshifts from the COSMOS survey.

We investigate the use of estimators of weak lensing power spectra based on a flat-sky implementation of the Pseudo-Cl (PCl) technique, where the masked shear field is transformed without regard for masked regions of sky. This masking mixes power, and E-convergence and B-modes. To study the accuracy of forward-modelling and full-sky power spectrum recovery we consider both large-area survey geometries, and small-scale masking due to stars and a checkerboard model for field-of-view gaps. The power spectrum for the large-area survey geometry is sparsely-sampled and highly oscillatory, which makes modelling problematic. Instead, we derive an overall calibration for large-area mask bias using simulated fields. The effects of small-area star masks can be accurately corrected for, while the checkerboard mask has oscillatory and spiky behaviour which leads to percent biases. Apodisation of the masked fields leads to increased biases and a loss of information. We find that we can construct an unbiased forward-model of the raw PCls, and recover the full-sky convergence power to within a few percent accuracy for both Gaussian and lognormal-distributed shear fields. Propagating this through to cosmological parameters using a Fisher-Matrix formalism, we find we can make unbiased estimates of parameters for surveys up to 1,200 deg$^2$ with 30 galaxies per arcmin$^2$, beyond which the percent biases become larger than the statistical accuracy. This implies a flat-sky PCl analysis is accurate for current surveys but a Euclid-like survey will require higher accuracy.

We present the first "clipped" cosmic shear measurement using data from the Kilo-Degree Survey (KiDS-450). "Clipping" transformations suppress the signal from the highest density, non-linear regions of cosmological fields. We demonstrate that these transformations improve constraints on $S_8=\sigma_8(\Omega_{\rm{m}}/0.3)^{0.5}$ when used in combination with conventional two-point statistics. For the KiDS-450 data, we find that the combined measurements improve the constraints on $S_8$ by 17%, compared to shear correlation functions alone. We determine the expectation value of the clipped shear correlation function using a suite of numerical simulations, and develop methodology to mitigate the impact of masking and shot noise. Future improvements in numerical simulations and mass reconstruction methodology will permit the precise calibration of clipped cosmic shear statistics such that clipping can become a standard tool in weak lensing analyses.

We present cosmological parameter constraints from a joint analysis of three cosmological probes: the tomographic cosmic shear signal in $\sim$450 deg$^2$ of data from the Kilo Degree Survey (KiDS), the galaxy-matter cross-correlation signal of galaxies from the Galaxies And Mass Assembly (GAMA) survey determined with KiDS weak lensing, and the angular correlation function of the same GAMA galaxies. We use fast power spectrum estimators that are based on simple integrals over the real-space correlation functions, and show that they are practically unbiased over relevant angular frequency ranges. We test our full pipeline on numerical simulations that are tailored to KiDS and retrieve the input cosmology. By fitting different combinations of power spectra, we demonstrate that the three probes are internally consistent. For all probes combined, we obtain $S_8\equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3}=0.800_{-0.027}^{+0.029}$, consistent with Planck and the fiducial KiDS-450 cosmic shear correlation function results. Marginalising over wide priors on the mean of the tomographic redshift distributions yields consistent results for $S_8$ with an increase of $28\%$ in the error. The combination of probes results in a $26\%$ reduction in uncertainties of $S_8$ over using the cosmic shear power spectra alone. The main gain from these additional probes comes through their constraining power on nuisance parameters, such as the galaxy intrinsic alignment amplitude or potential shifts in the redshift distributions, which are up to a factor of two better constrained compared to using cosmic shear alone, demonstrating the value of large-scale structure probe combination.

We conduct a comprehensive study of the effects of incorporating galaxy morphology information in photometric redshift estimation. Using machine learning methods, we assess the changes in the scatter and catastrophic outlier fraction of photometric redshifts when galaxy size, ellipticity, S\'{e}rsic index and surface brightness are included in training on galaxy samples from the SDSS and the CFHT Stripe-82 Survey (CS82). We show that by adding galaxy morphological parameters to full $ugriz$ photometry, only mild improvements are obtained, while the gains are substantial in cases where fewer passbands are available. For instance, the combination of $grz$ photometry and morphological parameters almost fully recovers the metrics of $5$-band photometric redshifts. We demonstrate that with morphology it is possible to determine useful redshift distribution $N(z)$ of galaxy samples without any colour information. We also find that the inclusion of quasar redshifts and associated object sizes in training improves the quality of photometric redshift catalogues, compensating for the lack of a good star-galaxy separator. We further show that morphological information can mitigate biases and scatter due to bad photometry. As an application, we derive both point estimates and posterior distributions of redshifts for the official CS82 catalogue, training on morphology and SDSS Stripe-82 $ugriz$ bands when available. Our redshifts yield a 68th percentile error of $0.058(1+z)$, and a catastrophic outlier fraction of $5.2$ per cent. We further include a deep extension trained on morphology and single $i$-band CS82 photometry.

We explore two methods of compressing the redshift space galaxy power spectrum and bispectrum with respect to a chosen set of cosmological parameters. Both methods involve reducing the dimension of the original data-vector ( e.g. 1000 elements ) to the number of cosmological parameters considered ( e.g. seven ) using the Karhunen-Lo\`eve algorithm. In the first case, we run MCMC sampling on the compressed data-vector in order to recover the one-dimensional (1D) and two-dimensional (2D) posterior distributions. The second option, approximately 2000 times faster, works by orthogonalising the parameter space through diagonalisation of the Fisher information matrix before the compression, obtaining the posterior distributions without the need of MCMC sampling. Using these methods for future spectroscopic redshift surveys like DESI, EUCLID and PFS would drastically reduce the number of simulations needed to compute accurate covariance matrices with minimal loss of constraining power. We consider a redshift bin of a DESI-like experiment. Using the power spectrum combined with the bispectrum as a data-vector, both compression methods on average recover the 68% credible regions to within 0.7% and 2% of those resulting from standard MCMC sampling respectively. These confidence intervals are also smaller than the ones obtained using only the power spectrum by (81%, 80%, 82%) respectively for the bias parameter b_1, the growth rate f and the scalar amplitude parameter A_s.

Weak gravitational lensing alters the apparent separations between observed sources, potentially affecting clustering statistics. We derive a general expression for the lensing deflection which is valid for any three-point statistic, and investigate its effect on the three-point clustering correlation function. We find that deflection of the clustering correlation function is greatest at around $z=2$. It is most prominent in regions where the correlation function varies rapidly, in particular at the baryon acoustic oscillation scale where it smooths out the peaks and troughs, reducing the peak-to-trough difference by about 0.1 percent at $z=1$ and around 2.3 percent at $z=10$. The modification due to lensing deflection is typically at the per cent level of the expected errors in a Euclid-like survey and therefore undetectable.

Tidal gravitational forces can modify the shape of galaxies and clusters of galaxies, thus correlating their orientation with the surrounding matter density field. We study the dependence of this phenomenon, known as intrinsic alignment (IA), on the mass of the dark matter haloes that host these bright structures, analysing the Millennium and Millennium-XXL $N$-body simulations. We closely follow the observational approach, measuring the halo position-halo shape alignment and subsequently dividing out the dependence on halo bias. We derive a theoretical scaling of the IA amplitude with mass in a dark matter universe, and predict a power-law with slope $\beta_{\mathrm{M}}$ in the range $1/3$ to $1/2$, depending on mass scale. We find that the simulation data agree with each other and with the theoretical prediction remarkably well over three orders of magnitude in mass, with the joint analysis yielding an estimate of $\beta_{\mathrm{M}} = 0.36^{+0.01}_{-0.01}$. This result does not depend on redshift or on the details of the halo shape measurement. The analysis is repeated on observational data, obtaining a significantly higher value, $\beta_{\mathrm{M}} = 0.56^{+0.05}_{-0.05}$. There are also small but significant deviations from our simple model in the simulation signals at both the high- and low-mass end. We discuss possible reasons for these discrepancies, and argue that they can be attributed to physical processes not captured in the model or in the dark matter-only simulations.

We present measurements of the weak gravitational lensing shear power spectrum based on $450$ sq. deg. of imaging data from the Kilo Degree Survey. We employ a quadratic estimator in two and three redshift bins and extract band powers of redshift auto-correlation and cross-correlation spectra in the multipole range $76 \leq \ell \leq 1310$. The cosmological interpretation of the measured shear power spectra is performed in a Bayesian framework assuming a $\Lambda$CDM model with spatially flat geometry, while accounting for small residual uncertainties in the shear calibration and redshift distributions as well as marginalising over intrinsic alignments, baryon feedback and an excess-noise power model. Moreover, massive neutrinos are included in the modelling. The cosmological main result is expressed in terms of the parameter combination $S_8 \equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3}$ yielding $S_8 = \ 0.651 \pm 0.058$ (3 z-bins), confirming the recently reported tension in this parameter with constraints from Planck at $3.2\sigma$ (3 z-bins). We cross-check the results of the 3 z-bin analysis with the weaker constraints from the 2 z-bin analysis and find them to be consistent. The high-level data products of this analysis, such as the band power measurements, covariance matrices, redshift distributions, and likelihood evaluation chains are available at http://kids.strw.leidenuniv.nl/

It is common practice in cosmology to model large-scale structure observables as lognormal random fields, and this approach has been successfully applied in the past to the matter density and weak lensing convergence fields separately. We argue that this approach has fundamental limitations which prevent its use for jointly modelling these two fields since the lognormal distribution's shape can prevent certain correlations to be attainable. Given the need of ongoing and future large-scale structure surveys for fast joint simulations of clustering and weak lensing, we propose two ways of overcoming these limitations. The first approach slightly distorts the power spectra of the fields using one of two algorithms that minimises either the absolute or the fractional distortions. The second one is by obtaining more accurate convergence marginal distributions, for which we provide a fitting function, by integrating the lognormal density along the line of sight. The latter approach also provides a way to determine directly from theory the skewness of the convergence distribution and, therefore, the parameters for a lognormal fit. We present the public code Full-sky Lognormal Astro-fields Simulation Kit (FLASK) which can make tomographic realisations on the sphere of an arbitrary number of correlated lognormal or Gaussian random fields by applying either of the two proposed solutions, and show that it can create joint simulations of clustering and lensing with sub-per-cent accuracy over relevant angular scales and redshift ranges.

We test extensions to the standard cosmological model with weak gravitational lensing tomography using 450 deg$^2$ of imaging data from the Kilo Degree Survey (KiDS). In these extended cosmologies, which include massive neutrinos, nonzero curvature, evolving dark energy, modified gravity, and running of the scalar spectral index, we also examine the discordance between KiDS and cosmic microwave background measurements from Planck. The discordance between the two datasets is largely unaffected by a more conservative treatment of the lensing systematics and the removal of angular scales most sensitive to nonlinear physics. The only extended cosmology that simultaneously alleviates the discordance with Planck and is at least moderately favored by the data includes evolving dark energy with a time-dependent equation of state (in the form of the $w_0-w_a$ parameterization). In this model, the respective $S_8 = \sigma_8 \sqrt{\Omega_{\rm m}/0.3}$ constraints agree at the $1\sigma$ level, and there is `substantial concordance' between the KiDS and Planck datasets when accounting for the full parameter space. Moreover, the Planck constraint on the Hubble constant is wider than in LCDM and in agreement with the Riess et al. (2016) direct measurement of $H_0$. The dark energy model is moderately favored as compared to LCDM when combining the KiDS and Planck measurements, and remains moderately favored after including an informative prior on the Hubble constant. In both of these scenarios, marginalized constraints in the $w_0-w_a$ plane are discrepant with a cosmological constant at the $3\sigma$ level. Moreover, KiDS constrains the sum of neutrino masses to 4.0 eV (95% CL), finds no preference for time or scale dependent modifications to the metric potentials, and is consistent with flatness and no running of the spectral index. The analysis code is public at https://github.com/sjoudaki/kids450

We study the stellar-to-halo mass relation of central galaxies in the range 9.7<log_10(M_*/h^-2 M_sun)<11.7 and z<0.4, obtained from a combined analysis of the Kilo Degree Survey (KiDS) and the Galaxy And Mass Assembly (GAMA) survey. We use ~100 deg^2 of KiDS data to study the lensing signal around galaxies for which spectroscopic redshifts and stellar masses were determined by GAMA. We show that lensing alone results in poor constraints on the stellar-to-halo mass relation due to a degeneracy between the satellite fraction and the halo mass, which is lifted when we simultaneously fit the stellar mass function. At M_sun>5x10^10 h^-2 M_sun, the stellar mass increases with halo mass as ~M_h^0.25. The ratio of dark matter to stellar mass has a minimum at a halo mass of 8x10^11 h^-1 M_sun with a value of M_h/M_*=56_-10^+16 [h]. We also use the GAMA group catalogue to select centrals and satellites in groups with five or more members, which trace regions in space where the local matter density is higher than average, and determine for the first time the stellar-to-halo mass relation in these denser environments. We find no significant differences compared to the relation from the full sample, which suggests that the stellar-to-halo mass relation does not vary strongly with local density. Furthermore, we find that the stellar-to-halo mass relation of central galaxies can also be obtained by modelling the lensing signal and stellar mass function of satellite galaxies only, which shows that the assumptions to model the satellite contribution in the halo model do not significantly bias the stellar-to-halo mass relation. Finally, we show that the combination of weak lensing with the stellar mass function can be used to test the purity of group catalogues.

We measure the alignment of the shapes of galaxy clusters, as traced by their satellite distributions, with the matter density field using the public redMaPPer catalogue based on SDSS-DR8, which contains 26 111 clusters up to z~0.6. The clusters are split into nine redshift and richness samples; in each of them we detect a positive alignment, showing that clusters point towards density peaks. We interpret the measurements within the tidal alignment paradigm, allowing for a richness and redshift dependence. The intrinsic alignment (IA) amplitude at the pivot redshift z=0.3 and pivot richness \lambda=30 is A_{IA}^{gen}=12.6_{-1.2}^{+1.5}. We obtain tentative evidence that the signal increases towards higher richness and lower redshift. Our measurements agree well with results of maxBCG clusters and with dark-matter-only simulations. Comparing our results to IA measurements of luminous red galaxies, we find that the IA amplitude of galaxy clusters forms a smooth extension towards higher mass. This suggests that these systems share a common alignment mechanism, which can be exploited to improve our physical understanding of IA.

We constrain the average halo ellipticity of ~2 600 galaxy groups from the Galaxy And Mass Assembly (GAMA) survey, using the weak gravitational lensing signal measured from the overlapping Kilo Degree Survey (KiDS). To do so, we quantify the azimuthal dependence of the stacked lensing signal around seven different proxies for the orientation of the dark matter distribution, as it is a priori unknown which one traces the orientation best. On small scales, the major axis of the brightest group/cluster member (BCG) provides the best proxy, leading to a clear detection of an anisotropic signal. In order to relate that to a halo ellipticity, we have to adopt a model density profile. We derive new expressions for the quadrupole moments of the shear field given an elliptical model surface mass density profile. Modeling the signal with an elliptical Navarro-Frenk-White (NFW) profile on scales < 250 kpc, which roughly corresponds to half the virial radius, and assuming that the BCG is perfectly aligned with the dark matter, we find an average halo ellipticity of e_h=0.38 +/- 0.12. This agrees well with results from cold-dark-matter-only simulations, which typically report values of e_h ~ 0.3. On larger scales, the lensing signal around the BCGs does not trace the dark matter distribution well, and the distribution of group satellites provides a better proxy for the halo's orientation instead, leading to a 3--4 sigma detection of a non-zero halo ellipticity at scales between 250 kpc and 750 kpc. Our results suggest that the distribution of stars enclosed within a certain radius forms a good proxy for the orientation of the dark matter within that radius, which has also been observed in hydrodynamical simulations.

In many astrophysical settings covariance matrices of large datasets have to be determined empirically from a finite number of mock realisations. The resulting noise degrades inference and precludes it completely if there are fewer realisations than data points. This work applies a recently proposed non-linear shrinkage estimator of covariance to a realistic example from large-scale structure cosmology. After optimising its performance for the usage in likelihood expressions, the shrinkage estimator yields subdominant bias and variance comparable to that of the standard estimator with a factor $\sim 50$ less realisations. This is achieved without any prior information on the properties of the data or the structure of the covariance matrix, at negligible computational cost.

We investigate the impact of astrophysical systematics on cosmic shear cosmological parameter constraints from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), and the concordance with cosmic microwave background measurements by Planck. We present updated CFHTLenS cosmic shear tomography measurements extended to degree scales using a covariance calibrated by a new suite of N-body simulations. We analyze these measurements with a new model fitting pipeline, accounting for key systematic uncertainties arising from intrinsic galaxy alignments, baryonic effects in the nonlinear matter power spectrum, and photometric redshift uncertainties. We examine the impact of the systematic degrees of freedom on the cosmological parameter constraints, both independently and jointly. When the systematic uncertainties are considered independently, the intrinsic alignment amplitude is the only degree of freedom that is substantially preferred by the data. When the systematic uncertainties are considered jointly, there is no consistently strong preference in favor of the more complex models. We quantify the level of concordance between the CFHTLenS and Planck datasets by employing two distinct data concordance tests, grounded in Bayesian evidence and information theory. We find that the two data concordance tests largely agree with one another, and that the level of concordance between the CFHTLenS and Planck datasets is sensitive to the exact details of the systematic uncertainties included in our analysis, ranging from decisive discordance to substantial concordance as the treatment of the systematic uncertainties becomes more conservative. The least conservative scenario is the one most favored by the cosmic shear data, but it is also the one that shows the greatest degree of discordance with Planck. The data and analysis code are public at https://github.com/sjoudaki/cfhtlens_revisited

We present cosmological parameter constraints from a tomographic weak gravitational lensing analysis of ~450deg$^2$ of imaging data from the Kilo Degree Survey (KiDS). For a flat $\Lambda$CDM cosmology with a prior on $H_0$ that encompasses the most recent direct measurements, we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.745\pm0.039$. This result is in good agreement with other low redshift probes of large scale structure, including recent cosmic shear results, along with pre-Planck cosmic microwave background constraints. A $2.3$-$\sigma$ tension in $S_8$ and `substantial discordance' in the full parameter space is found with respect to the Planck 2015 results. We use shear measurements for nearly 15 million galaxies, determined with a new improved `self-calibrating' version of $lens$fit validated using an extensive suite of image simulations. Four-band $ugri$ photometric redshifts are calibrated directly with deep spectroscopic surveys. The redshift calibration is confirmed using two independent techniques based on angular cross-correlations and the properties of the photometric redshift probability distributions. Our covariance matrix is determined using an analytical approach, verified numerically with large mock galaxy catalogues. We account for uncertainties in the modelling of intrinsic galaxy alignments and the impact of baryon feedback on the shape of the non-linear matter power spectrum, in addition to the small residual uncertainties in the shear and redshift calibration. The cosmology analysis was performed blind. Our high-level data products, including shear correlation functions, covariance matrices, redshift distributions, and Monte Carlo Markov Chains are available at http://kids.strw.leidenuniv.nl.

We present the Red-sequence Cluster Lensing Survey (RCSLenS), an application of the methods developed for the Canada France Hawaii Telescope Lensing Survey (CFHTLenS) to the ~785deg$^2$, multi-band imaging data of the Red-sequence Cluster Survey 2 (RCS2). This project represents the largest public, sub-arcsecond seeing, multi-band survey to date that is suited for weak gravitational lensing measurements. With a careful assessment of systematic errors in shape measurements and photometric redshifts we extend the use of this data set to allow cross-correlation analyses between weak lensing observables and other data sets. We describe the imaging data, the data reduction, masking, multi-colour photometry, photometric redshifts, shape measurements, tests for systematic errors, and a blinding scheme to allow for more objective measurements. In total we analyse 761 pointings with r-band coverage, which constitutes our lensing sample. Residual large-scale B-mode systematics prevent the use of this shear catalogue for cosmic shear science. The effective number density of lensing sources over an unmasked area of 571.7deg$^2$ and down to a magnitude limit of r~24.5 is 8.1 galaxies per arcmin$^2$ (weighted: 5.5 arcmin$^{-2}$) distributed over 14 patches on the sky. Photometric redshifts based on 4-band griz data are available for 513 pointings covering an unmasked area of 383.5 deg$^2$ We present weak lensing mass reconstructions of some example clusters as well as the full survey representing the largest areas that have been mapped in this way. All our data products are publicly available through CADC at http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/en/community/rcslens/query.html in a format very similar to the CFHTLenS data release.

We present a method to transform multivariate unimodal non-Gaussian posterior probability densities into approximately Gaussian ones via non-linear mappings, such as Box--Cox transformations and generalisations thereof. This permits an analytical reconstruction of the posterior from a point sample, like a Markov chain, and simplifies the subsequent joint analysis with other experiments. This way, a multivariate posterior density can be reported efficiently, by compressing the information contained in MCMC samples. Further, the model evidence integral (i.e. the marginal likelihood) can be computed analytically. This method is analogous to the search for normal parameters in the cosmic microwave background, but is more general. The search for the optimally Gaussianising transformation is performed computationally through a maximum-likelihood formalism; its quality can be judged by how well the credible regions of the posterior are reproduced. We demonstrate that our method outperforms kernel density estimates in this objective. Further, we select marginal posterior samples from Planck data with several distinct strongly non-Gaussian features, and verify the reproduction of the marginal contours. To demonstrate evidence computation, we Gaussianise the joint distribution of data from weak lensing and baryon acoustic oscillations (BAO), for different cosmological models, and find a preference for flat $\Lambda$CDM. Comparing to values computed with the Savage-Dickey density ratio, and Population Monte Carlo, we find good agreement of our method within the spread of the other two.

It is well known that the probability distribution function (PDF) of galaxy density contrast is approximately lognormal; whether the PDF of mass fluctuations derived from weak lensing convergence (kappa_WL) is lognormal is less well established. We derive PDFs of the galaxy and projected matter density distributions via the Counts in Cells (CiC) method. We use maps of galaxies and weak lensing convergence produced from the Dark Energy Survey (DES) Science Verification data over 139 deg^2. We test whether the underlying density contrast is well described by a lognormal distribution for the galaxies, the convergence and their joint PDF. We confirm that the galaxy density contrast distribution is well modeled by a lognormal PDF convolved with Poisson noise at angular scales from 10-40 arcmin (corresponding to physical scales of 3-10 Mpc). We note that as kappa_WL is a weighted sum of the mass fluctuations along the line of sight, its PDF is expected to be only approximately lognormal. We find that the kappa_WL distribution is well modeled by a lognormal PDF convolved with Gaussian shape noise at scales between 10 and 20 arcmin, with a best-fit chi^2/DOF of 1.11 compared to 1.84 for a Gaussian model, corresponding to p-values 0.35 and 0.07 respectively, at a scale of 10 arcmin. Above 20 arcmin a simple Gaussian model is sufficient. The joint PDF is also reasonably fitted by a bivariate lognormal. As a consistency check we compare the variances derived from the lognormal modelling with those directly measured via CiC. Our methods are validated against maps from the MICE Grand Challenge N-body simulation.

We present the first measurement of individual cluster mass estimates using weak lensing size and flux magnification. Using data from the HST-STAGES survey of the A901/902 supercluster we detect the four known groups in the supercluster at high significance using magnification alone. We discuss the application of a fully Bayesian inference analysis, and investigate a broad range of potential systematics in the application of the method. We compare our results to a previous weak lensing shear analysis of the same field finding the recovered signal-to-noise of our magnification-only analysis to range from 45% to 110% of the signal-to-noise in the shear-only analysis. On a case-by-case basis we find consistent magnification and shear constraints on cluster virial radius, and finding that for the full sample, magnification constraints to be a factor $0.77 \pm 0.18$ lower than the shear measurements.

Galaxy shapes are not randomly oriented, rather they are statistically aligned in a way that can depend on formation environment, history and galaxy type. Studying the alignment of galaxies can therefore deliver important information about the physics of galaxy formation and evolution as well as the growth of structure in the Universe. In this review paper we summarise key measurements of galaxy alignments, divided by galaxy type, scale and environment. We also cover the statistics and formalism necessary to understand the observations in the literature. With the emergence of weak gravitational lensing as a precision probe of cosmology, galaxy alignments have taken on an added importance because they can mimic cosmic shear, the effect of gravitational lensing by large-scale structure on observed galaxy shapes. This makes galaxy alignments, commonly referred to as intrinsic alignments, an important systematic effect in weak lensing studies. We quantify the impact of intrinsic alignments on cosmic shear surveys and finish by reviewing practical mitigation techniques which attempt to remove contamination by intrinsic alignments.

The shapes of galaxies are not randomly oriented on the sky. During the galaxy formation and evolution process, environment has a strong influence, as tidal gravitational fields in the large-scale structure tend to align nearby galaxies. Additionally, events such as galaxy mergers affect the relative alignments of both the shapes and angular momenta of galaxies throughout their history. These "intrinsic galaxy alignments" are known to exist, but are still poorly understood. This review will offer a pedagogical introduction to the current theories that describe intrinsic galaxy alignments, including the apparent difference in intrinsic alignment between early- and late-type galaxies and the latest efforts to model them analytically. It will then describe the ongoing efforts to simulate intrinsic alignments using both N-body and hydrodynamic simulations. Due to the relative youth of this field, there is still much to be done to understand intrinsic galaxy alignments and this review summarises the current state of the field, providing a solid basis for future work.

The alignments between galaxies, their underlying matter structures, and the cosmic web constitute vital ingredients for a comprehensive understanding of gravity, the nature of matter, and structure formation in the Universe. We provide an overview on the state of the art in the study of these alignment processes and their observational signatures, aimed at a non-specialist audience. The development of the field over the past one hundred years is briefly reviewed. We also discuss the impact of galaxy alignments on measurements of weak gravitational lensing, and discuss avenues for making theoretical and observational progress over the coming decade.

The unknown nature of dark energy motivates continued cosmological tests of large-scale gravitational physics. We present a new consistency check based on the relative amplitude of non-relativistic galaxy peculiar motions, measured via redshift-space distortion, and the relativistic deflection of light by those same galaxies traced by galaxy-galaxy lensing. We take advantage of the latest generation of deep, overlapping imaging and spectroscopic datasets, combining the Red Cluster Sequence Lensing Survey (RCSLenS), the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), the WiggleZ Dark Energy Survey and the Baryon Oscillation Spectroscopic Survey (BOSS). We quantify the results using the "gravitational slip" statistic E_G, which we estimate as 0.48 +/- 0.10 at z=0.32 and 0.30 +/- 0.07 at z=0.57, the latter constituting the highest redshift at which this quantity has been determined. These measurements are consistent with the predictions of General Relativity, for a perturbed Friedmann-Robertson-Walker metric in a Universe dominated by a cosmological constant, which are E_G = 0.41 and 0.36 at these respective redshifts. The combination of redshift-space distortion and gravitational lensing data from current and future galaxy surveys will offer increasingly stringent tests of fundamental cosmology.

The fundamental plane (FP) is a widely used tool to investigate the properties of early-type galaxies, and the tight relation between its parameters has spawned several cosmological applications, including its use as a distance indicator for peculiar velocity surveys and as a means to suppress intrinsic noise in cosmic size magnification measurements. Systematic trends with the large-scale structure across the FP could cause serious biases for these cosmological probes, but may also yield new insights into the early-type population. Here we report the first detection of spatial correlations among offsets in galaxy size from an FP that explicitly accounts for redshift trends, using a sample of about $95,000$ elliptical galaxies from the Sloan Digital Sky Survey. We show that these offsets correlate with the density field out to at least $10h^{-1}$Mpc at $4\sigma$ significance in a way that cannot be explained by systematic errors in galaxy size estimates. We propose a physical explanation for the correlations by dividing the sample into central, satellite, and field galaxies, identifying trends for each galaxy type separately. Central (satellite) galaxies lie on average above (below) the FP, which we argue could be due to a higher (lower) than average mass-to-light ratio. We fit a simple model to the correlations of FP residuals and use it to predict the impact on peculiar velocity power spectra, finding a contamination larger than $10\,\%$ for $k>0.04\,h/$Mpc. Moreover, cosmic magnification measurements based on an FP could be severely contaminated over a wide range of scales by the intrinsic FP correlations.

We measure the weak gravitational lensing shear power spectra and their cross-power in two photometric redshift bins from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). The measurements are performed directly in multipole space in terms of adjustable band powers. For the extraction of the band powers from the data we have implemented and extended a quadratic estimator, a maximum likelihood method that allows us to readily take into account irregular survey geometries, masks, and varying sampling densities. We find the 68 per cent credible intervals in the $\sigma_8$-$\Omega_{\rm m}$-plane to be marginally consistent with results from $Planck$ for a simple five parameter $\Lambda$CDM model. For the projected parameter $S_8 \equiv \sigma_8(\Omega_{\rm m}/0.3)^{0.5}$ we obtain a best-fitting value of $S_8 = 0.768_{-0.039}^{+0.045}$. This constraint is consistent with results from other CFHTLenS studies as well as the Dark Energy Survey. Our most conservative model, including modifications to the power spectrum due to baryon feedback and marginalization over photometric redshift errors, yields an upper limit on the total mass of three degenerate massive neutrinos of $\Sigma m_\nu < 4.53 \, {\rm eV}$ at 95 per cent credibility, while a Bayesian model comparison does not favour any model extension beyond a simple five parameter $\Lambda$CDM model. Combining the shear likelihood with $Planck$ breaks the $\sigma_8$-$\Omega_{\rm m}$-degeneracy and yields $\sigma_8=0.818 \pm 0.013$ and $\Omega_{\rm m} = 0.300 \pm 0.011$ which is fully consistent with results from $Planck$ alone.

Applying a transformation to a non-Gaussian field can enhance the information content of the resulting power spectrum, by reducing the correlations between Fourier modes. In the context of weak gravitational lensing, it has been shown that this gain in information content is significantly compromised by the presence of shape noise. We apply clipping to mock convergence fields, a technique which is known to be robust in the presence of noise and has been successfully applied to galaxy number density fields. When analysed in isolation the resulting convergence power spectrum returns degraded constraints on cosmological parameters. However substantial gains can be achieved by performing a combined analysis of the power spectra derived from both the original and transformed fields. Even in the presence of realistic levels of shape noise, we demonstrate that this approach is capable of reducing the area of likelihood contours within the $\Omega_m - \sigma_8$ plane by more than a factor of three.

We present weak lensing constraints on the ellipticity of galaxy-scale matter haloes and the galaxy-halo misalignment. Using data from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), we measure the weighted-average ratio of the aligned projected ellipticity components of galaxy matter haloes and their embedded galaxies, $f_\mathrm{h}$, split by galaxy type. We then compare our observations to measurements taken from the Millennium Simulation, assuming different models of galaxy-halo misalignment. Using the Millennium Simulation we verify that the statistical estimator used removes contamination from cosmic shear. We also detect an additional signal in the simulation, which we interpret as the impact of intrinsic shape-shear alignments between the lenses and their large-scale structure environment. These alignments are likely to have caused some of the previous observational constraints on $f_\mathrm{h}$ to be biased high. From CFHTLenS we find $f_\mathrm{h}=-0.04 \pm 0.25$ for early-type galaxies, which is consistent with current models for the galaxy-halo misalignment predicting $f_\mathrm{h}\simeq 0.20$. For late-type galaxies we measure $f_\mathrm{h}=0.69_{-0.36}^{+0.37}$ from CFHTLenS. This can be compared to the simulated results which yield $f_\mathrm{h}\simeq 0.02$ for misaligned late-type models.

We use the first 100 sq. deg. of overlap between the Kilo-Degree Survey (KiDS) and the Galaxy And Mass Assembly (GAMA) survey to determine the galaxy halo mass of ~10,000 spectroscopically-confirmed satellite galaxies in massive ($M > 10^{13}h^{-1}{\rm M}_\odot$) galaxy groups. Separating the sample as a function of projected distance to the group centre, we jointly model the satellites and their host groups with Navarro-Frenk-White (NFW) density profiles, fully accounting for the data covariance. The probed satellite galaxies in these groups have total masses $\log M_{\rm sub} /(h^{-1}{\rm M}_\odot) \approx 11.7 - 12.2$ consistent across group-centric distance within the errorbars. Given their typical stellar masses, $\log M_{\rm \star,sat}/(h^{-2}{\rm M}_\odot) \sim 10.5$, such total masses imply stellar mass fractions of $M_{\rm \star,sat} /M_{\rm sub} \approx 0.04 h^{-1}$ . The average subhalo hosting these satellite galaxies has a mass $M_{\rm sub} \sim 0.015M_{\rm host}$ independent of host halo mass, in broad agreement with the expectations of structure formation in a $\Lambda$CDM universe.

The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the survey is mapping the large-scale matter distribution in the Universe back to a redshift of ~0.5. Secondary science cases are manifold, covering topics such as galaxy evolution, Milky Way structure, and the detection of high-redshift clusters and quasars. KiDS is an ESO Public Survey and dedicated to serving the astronomical community with high-quality data products derived from the survey data, as well as with calibration data. Public data releases will be made on a yearly basis, the first two of which are presented here. For a total of 148 survey tiles (~160 sq.deg.) astrometrically and photometrically calibrated, coadded ugri images have been released, accompanied by weight maps, masks, source lists, and a multi-band source catalog. A dedicated pipeline and data management system based on the Astro-WISE software system, combined with newly developed masking and source classification software, is used for the data production of the data products described here. The achieved data quality and early science projects based on the data products in the first two data releases are reviewed in order to validate the survey data. Early scientific results include the detection of nine high-z QSOs, fifteen candidate strong gravitational lenses, high-quality photometric redshifts and galaxy structural parameters for hundreds of thousands of galaxies. (Abridged)

The Kilo-Degree Survey (KiDS) is an optical wide-field survey designed to map the matter distribution in the Universe using weak gravitational lensing. In this paper, we use these data to measure the density profiles and masses of a sample of $\sim \mathrm{1400}$ spectroscopically identified galaxy groups and clusters from the Galaxy And Mass Assembly (GAMA) survey. We detect a highly significant signal (signal-to-noise-ratio $\sim$ 120), allowing us to study the properties of dark matter haloes over one and a half order of magnitude in mass, from $M \sim 10^{13}-10^{14.5} h^{-1}\mathrm{M_{\odot}}$. We interpret the results for various subsamples of groups using a halo model framework which accounts for the mis-centring of the Brightest Cluster Galaxy (used as the tracer of the group centre) with respect to the centre of the group's dark matter halo. We find that the density profiles of the haloes are well described by an NFW profile with concentrations that agree with predictions from numerical simulations. In addition, we constrain scaling relations between the mass and a number of observable group properties. We find that the mass scales with the total r-band luminosity as a power-law with slope $1.16 \pm 0.13$ (1-sigma) and with the group velocity dispersion as a power-law with slope $1.89 \pm 0.27$ (1-sigma). Finally, we demonstrate the potential of weak lensing studies of groups to discriminate between models of baryonic feedback at group scales by comparing our results with the predictions from the Cosmo-OverWhelmingly Large Simulations (Cosmo-OWLS) project, ruling out models without AGN feedback.

The Square Kilometer Array (SKA) has the potential to produce galaxy redshift surveys which will be competitive with other state of the art cosmological experiments in the next decade. In this chapter we summarise what capabilities the first and the second phases of the SKA will be able to achieve in its current state of design. We summarise the different cosmological experiments which are outlined in further detail in other chapters of this Science Book. The SKA will be able to produce competitive Baryonic Oscillation (BAOs) measurements in both its phases. The first phase of the SKA will provide similar measurements as optical and IR experiments with completely different systematic effects whereas the second phase being transformational in terms of its statistical power. The SKA will produce very accurate Redshift Space Distortions (RSD) measurements, being superior to other experiments at lower redshifts, due to the large number of galaxies. Cross correlations of the galaxy redshift data from the SKA with radio continuum surveys and optical surveys will provide extremely good calibration of photometric redshifts as well as extremely good bounds on modifications of gravity. Basing on a Principle Component Analysis (PCA) approach, we find that the SKA will be able to provide competitive constraints on dark energy and modified gravity models. Due to the large area covered the SKA it will be a transformational experiment in measuring physics from the largest scales such as non-Gaussian signals from $\textrm{f}_{\textrm{nl}}$. Finally, the SKA might produce the first real time measurement of the redshift drift. The SKA will be a transformational machine for cosmology as it grows from an early Phase 1 to its full power.

By the time that the first phase of the Square Kilometre Array is deployed it will be able to perform state of the art Large Scale Structure (LSS) as well as Weak Gravitational Lensing (WGL) measurements of the distribution of matter in the Universe. In this chapter we concentrate on the synergies that result from cross-correlating these different SKA data products as well as external correlation with the weak lensing measurements available from CMB missions. We show that the Dark Energy figures of merit obtained individually from WGL/LSS measurements and their independent combination is significantly increased when their full cross-correlations are taken into account. This is due to the increased knowledge of galaxy bias as a function of redshift as well as the extra information from the different cosmological dependences of the cross-correlations. We show that the cross-correlation between a spectroscopic LSS sample and a weak lensing sample with photometric redshifts can calibrate these same photometric redshifts, and their scatter, to high accuracy by modelling them as nuisance parameters and fitting them simultaneously cosmology. Finally we show that Modified Gravity parameters are greatly constrained by this cross-correlations because weak lensing and redshift space distortions (from the LSS survey) break strong degeneracies in common parameterisations of modified gravity.

We investigate the capabilities of various stages of the SKA to perform world-leading weak gravitational lensing surveys. We outline a way forward to develop the tools needed for pursuing weak lensing in the radio band. We identify the key analysis challenges and the key pathfinder experiments that will allow us to address them in the run up to the SKA. We identify and summarize the unique and potentially very powerful aspects of radio weak lensing surveys, facilitated by the SKA, that can solve major challenges in the field of weak lensing. These include the use of polarization and rotational velocity information to control intrinsic alignments, and the new area of weak lensing using intensity mapping experiments. We show how the SKA lensing surveys will both complement and enhance corresponding efforts in the optical wavebands through cross-correlation techniques and by way of extending the reach of weak lensing to high redshift.

Current and forthcoming cosmological data analyses share the challenge of huge datasets alongside increasingly tight requirements on the precision and accuracy of extracted cosmological parameters. The community is becoming increasingly aware that these requirements not only apply to the central values of parameters but, equally important, also to the error bars. Due to non-linear effects in the astrophysics, the instrument, and the analysis pipeline, data covariance matrices are usually not well known a priori and need to be estimated from the data itself, or from suites of large simulations. In either case, the finite number of realisations available to determine data covariances introduces significant biases and additional variance in the errors on cosmological parameters in a standard likelihood analysis. Here, we review recent work on quantifying these biases and additional variances and discuss approaches to remedy these effects.

We investigate the bias and error in estimates of the cosmological parameter covariance matrix, due to sampling or modelling the data covariance matrix, for likelihood width and peak scatter estimators. We show that these estimators do not coincide unless the data covariance is exactly known. For sampled data covariances, with Gaussian distributed data and parameters, the parameter covariance matrix estimated from the width of the likelihood has a Wishart distribution, from which we derive the mean and covariance. This mean is biased and we propose an unbiased estimator of the parameter covariance matrix. Comparing our analytic results to a numerical Wishart sampler of the data covariance matrix we find excellent agreement. An accurate ansatz for the mean parameter covariance for the peak scatter estimator is found, and we fit its covariance to our numerical analysis. The mean is again biased and we propose an unbiased estimator for the peak parameter covariance. For sampled data covariances the width estimator is more accurate than the peak scatter estimator. We investigate modelling the data covariance, or equivalently data compression, and shown that the peak scatter estimator is less sensitive to biases in the model data covariance matrix than the width estimator, but requires independent realisations of the data to reduce the statistical error. If the model bias on the peak estimator is sufficiently low this is promising, otherwise the sampled width estimator is preferable.

Intrinsic galaxy alignments constitute the major astrophysical systematic of forthcoming weak gravitational lensing surveys but also yield unique insights into galaxy formation and evolution. We build analytic models for the distribution of galaxy shapes based on halo properties extracted from the Millennium Simulation, differentiating between early- and late-type galaxies as well as central galaxies and satellites. The resulting ellipticity correlations are investigated for their physical properties and compared to a suite of current observations. The best-faring model is then used to predict the intrinsic alignment contamination of planned weak lensing surveys. We find that late-type galaxy models generally have weak intrinsic ellipticity correlations, marginally increasing towards smaller galaxy separation and higher redshift. The signal for early-type models at fixed halo mass strongly increases by three orders of magnitude over two decades in galaxy separation, and by one order of magnitude from z=0 to z=2. The intrinsic alignment strength also depends strongly on halo mass, but not on galaxy luminosity at fixed mass, or galaxy number density in the environment. We identify models that are in good agreement with all observational data, except that all models over-predict alignments of faint early-type galaxies. The best model yields an intrinsic alignment contamination of a Euclid-like survey between 0.5-10% at z>0.6 and on angular scales larger than a few arcminutes. Cutting 20% of red foreground galaxies using observer-frame colours can suppress this contamination by up to a factor of two.

With 3rd-order statistics of gravitational shear it will be possible to extract valuable cosmological information from ongoing and future weak lensing surveys which is not contained in standard 2nd-order statistics, due to the non-Gaussianity of the shear field. Aperture mass statistics are an appropriate choice for 3rd-order statistics due to their simple form and their ability to separate E- and B-modes of the shear. However, it has been demonstrated that 2nd-order aperture mass statistics suffer from E-/B-mode mixing because it is impossible to reliably estimate the shapes of close pairs of galaxies. This finding has triggered developments of several new 2nd-order statistical measures for cosmic shear. Whether the same developments are needed for 3rd-order shear statistics is largely determined by how severe this E-/B-mixing is for 3rd-order statistics. We test 3rd-order aperture mass statistics against E-/B-mode mixing, and find that the level of contamination is well-described by a function of $\theta/\theta_{\rm min}$, with $\theta_{\rm min}$ being the cut-off scale. At angular scales of $\theta > 10 \;\theta_{\rm min}$, the decrease in the E-mode signal due to E-/B-mode mixing is smaller than 1 percent, and the leakage into B-modes is even less. For typical small-scale cut-offs this E-/B-mixing is negligible on scales larger than a few arcminutes. Therefore, 3rd-order aperture mass statistics can safely be used to separate E- and B-modes and infer cosmological information, for ground-based surveys as well as forthcoming space-based surveys such as Euclid.

In this paper we derive an exact full expression for the 2D probability distribution of the ellipticity of an object measured from data, only assuming Gaussian noise in pixel values. This is a generalisation of the probability distribution for the ratio of single random variables, that is well-known, to the multivariate case. This expression is derived within the context of the measurement of weak gravitational lensing from noisy galaxy images. We find that the third flattening, or epsilon-ellipticity, has a biased maximum likelihood but an unbiased mean; and that the third eccentricity, or normalised polarisation chi, has both a biased maximum likelihood and a biased mean. The very fact that the bias in the ellipticity is itself a function of the ellipticity requires an accurate knowledge of the intrinsic ellipticity distribution of the galaxies in order to properly calibrate shear measurements. We use this expression to explore strategies for calibration of biases caused by measurement processes in weak gravitational lensing. We find that upcoming weak lensing surveys like KiDS or DES require calibration fields of order of several square degrees and 1.2 magnitude deeper than the wide survey in order to correct for the noise bias. Future surveys like Euclid will require calibration fields of order 40 square degree and several magnitude deeper than the wide survey. We also investigate the use of the Stokes parameters to estimate the shear as an alternative to the ellipticity. We find that they can provide unbiased shear estimates at the cost of a very large variance in the measurement. The python code used to compute the distributions presented in the paper and to perform the numerical calculations are available on request.

We outline the prospects for performing pioneering radio weak gravitational lensing analyses using observations from a potential forthcoming JVLA Sky Survey program. A large-scale survey with the JVLA can offer interesting and unique opportunities for performing weak lensing studies in the radio band, a field which has until now been the preserve of optical telescopes. In particular, the JVLA has the capacity for large, deep radio surveys with relatively high angular resolution, which are the key characteristics required for a successful weak lensing study. We highlight the potential advantages and unique aspects of performing weak lensing in the radio band. In particular, the inclusion of continuum polarisation information can greatly reduce noise in weak lensing reconstructions and can also remove the effects of intrinsic galaxy alignments, the key astrophysical systematic effect that limits weak lensing at all wavelengths. We identify a VLASS "deep fields" program (total area ~10-20 square degs), to be conducted at L-band and with high-resolution (A-array configuration), as the optimal survey strategy from the point of view of weak lensing science. Such a survey will build on the unique strengths of the JVLA and will remain unsurpassed in terms of its combination of resolution and sensitivity until the advent of the Square Kilometre Array. We identify the best fields on the JVLA-accessible sky from the point of view of overlapping with existing deep optical and near infra-red data which will provide crucial redshift information and facilitate a host of additional compelling multi-wavelength science.

In this paper, we motivate the use of galaxy clustering measurements using photometric redshift information, including a contribution from flux magnification, as a probe of cosmology. We present cosmological forecasts when clustering data alone is used, and when clustering is combined with a cosmic shear analysis. We consider two types of clustering analysis: firstly, clustering with only redshift auto-correlations in tomographic redshift bins; secondly, using all available redshift bin correlations. Finally, we consider how inferred cosmological parameters may be biased using each analysis when flux magnification is neglected. Results are presented for a Stage III ground-based survey, and a Stage IV space-based survey modelled with photometric redshift errors, and values for the slope of the luminosity function inferred from CFHTLenS catalogues. We find that combining clustering information with shear can improve constraints on cosmological parameters, giving an improvement to a Dark Energy Task Force-like figure of merit by a factor of 1.33 when only auto-correlations in redshift are used for the clustering analysis, rising to 1.52 when cross-correlations in redshift are also included. The addition of galaxy-galaxy lensing gives further improvement, with increases in figure of merit by a factor of 2.82 and 3.7 for each type of clustering analysis respectively. The presence of flux magnification in a clustering analysis does not significantly affect the precision of cosmological constraints when combined with cosmic shear and galaxy-galaxy lensing. However if magnification is neglected, inferred cosmological parameter values are biased, with biases in some cosmological parameters larger than statistical errors. (Abridged)

Forecasts of statistical constraints on model parameters using the Fisher matrix abound in many fields of astrophysics. The Fisher matrix formalism involves the assumption of Gaussianity in parameter space and hence fails to predict complex features of posterior probability distributions. Combining the standard Fisher matrix with Box-Cox transformations, we propose a novel method that accurately predicts arbitrary posterior shapes. The Box-Cox transformations are applied to parameter space to render it approximately multivariate Gaussian, performing the Fisher matrix calculation on the transformed parameters. We demonstrate that, after the Box-Cox parameters have been determined from an initial likelihood evaluation, the method correctly predicts changes in the posterior when varying various parameters of the experimental setup and the data analysis, with marginally higher computational cost than a standard Fisher matrix calculation. We apply the Box-Cox-Fisher formalism to forecast cosmological parameter constraints by future weak gravitational lensing surveys. The characteristic non-linear degeneracy between matter density parameter and normalisation of matter density fluctuations is reproduced for several cases, and the capabilities of breaking this degeneracy by weak lensing three-point statistics is investigated. Possible applications of Box-Cox transformations of posterior distributions are discussed, including the prospects for performing statistical data analysis steps in the transformed Gaussianised parameter space.

The statistical properties of the ellipticities of galaxy images depend on how galaxies form and evolve, and therefore constrain models of galaxy morphology, which are key to the removal of the intrinsic alignment contamination of cosmological weak lensing surveys, as well as to the calibration of weak lensing shape measurements. We construct such models based on the halo properties of the Millennium Simulation and confront them with a sample of 90,000 galaxies from the COSMOS Survey, covering three decades in luminosity and redshifts out to z=2. The ellipticity measurements are corrected for effects of point spread function smearing, spurious image distortions, and measurement noise. Dividing galaxies into early, late, and irregular types, we find that early-type galaxies have up to a factor of two lower intrinsic ellipticity dispersion than late-type galaxies. None of the samples shows evidence for redshift evolution, while the ellipticity dispersion for late-type galaxies scales strongly with absolute magnitude at the bright end. The simulation-based models reproduce the main characteristics of the intrinsic ellipticity distributions although which model fares best depends on the selection criteria of the galaxy sample. We observe fewer close-to-circular late-type galaxy images in COSMOS than expected for a sample of randomly oriented circular thick disks and discuss possible explanations for this deficit.

Cosmological parameter estimation requires that the likelihood function of the data is accurately known. Assuming that cosmological large-scale structure power spectra data are multivariate Gaussian-distributed, we show the accuracy of parameter estimation is limited by the accuracy of the inverse data covariance matrix - the precision matrix. If the data covariance and precision matrices are estimated by sampling independent realisations of the data, their statistical properties are described by the Wishart and Inverse-Wishart distributions, respectively. Independent of any details of the survey, we show that the fractional error on a parameter variance, or a Figure-of-Merit, is equal to the fractional variance of the precision matrix. In addition, for the only unbiased estimator of the precision matrix, we find that the fractional accuracy of the parameter error depends only on the difference between the number of independent realisations and the number of data points, and so can easily diverge. For a 5% error on a parameter error and N_D << 100 data-points, a minimum of 200 realisations of the survey are needed, with 10% accuracy for the data covariance. If the number of data-points N_D >>100 we need N_S > N_D realisations and a fractional accuracy of <sqrt[2/N_D] in the data covariance. As the number of power spectra data points grows to N_D>10^4 -10^6 this approach will be problematic. We discuss possible ways to relax these conditions: improved theoretical modelling; shrinkage methods; data-compression; simulation and data resampling methods.

The spatial variation of the colour of a galaxy may introduce a bias in the measurement of its shape if the PSF profile depends on wavelength. We study how this bias depends on the properties of the PSF and the galaxies themselves. The bias depends on the scales used to estimate the shape, which may be used to optimise methods to reduce the bias. Here we develop a general approach to quantify the bias. Although applicable to any weak lensing survey, we focus on the implications for the ESA Euclid mission. Based on our study of synthetic galaxies we find that the bias is a few times 10^-3 for a typical galaxy observed by Euclid. Consequently, it cannot be neglected and needs to be accounted for. We demonstrate how one can do so using spatially resolved observations of galaxies in two filters. We show that HST observations in the F606W and F814W filters allow us to model and reduce the bias by an order of magnitude, sufficient to meet Euclid's scientific requirements. The precision of the correction is ultimately determined by the number of galaxies for which spatially-resolved observations in at least two filters are available. We use results from the Millennium Simulation to demonstrate that archival HST data will be sufficient for the tomographic cosmic shear analysis with the Euclid dataset.

Euclid is a space-based survey mission from the European Space Agency designed to understand the origin of the Universe's accelerating expansion. It will use cosmological probes to investigate the nature of dark energy, dark matter and gravity by tracking their observational signatures on the geometry of the universe and on the cosmic history of structure formation. The mission is optimised for two independent primary cosmological probes: Weak gravitational Lensing (WL) and Baryonic Acoustic Oscillations (BAO). The Euclid payload consists of a 1.2 m Korsch telescope designed to provide a large field of view. It carries two instruments with a common field-of-view of ~0.54 deg2: the visual imager (VIS) and the near infrared instrument (NISP) which contains a slitless spectrometer and a three bands photometer. The Euclid wide survey will cover 15,000 deg2 of the extragalactic sky and is complemented by two 20 deg2 deep fields. For WL, Euclid measures the shapes of 30-40 resolved galaxies per arcmin2 in one broad visible R+I+Z band (550-920 nm). The photometric redshifts for these galaxies reach a precision of dz/(1+z) < 0.05. They are derived from three additional Euclid NIR bands (Y, J, H in the range 0.92-2.0 micron), complemented by ground based photometry in visible bands derived from public data or through engaged collaborations. The BAO are determined from a spectroscopic survey with a redshift accuracy dz/(1+z) =0.001. The slitless spectrometer, with spectral resolution ~250, predominantly detects Ha emission line galaxies. Euclid is a Medium Class mission of the ESA Cosmic Vision 2015-2025 programme, with a foreseen launch date in 2019. This report (also known as the Euclid Red Book) describes the outcome of the Phase A study.

We investigate the information on cosmology contained in Gaussianised weak gravitational lensing convergence fields. Employing Box-Cox transformations to determine optimal transformations to Gaussianity, we develop analytical models for the transformed power spectrum, including effects of noise and smoothing. We find that optimised Box-Cox transformations perform substantially better than an offset logarithmic transformation in Gaussianising the convergence, but both yield very similar results for the signal-to-noise and parameter constraints. None of the transformations is capable of eliminating correlations of the power spectra between different angular frequencies, which we demonstrate to have a significant impact on the errors on cosmology. Analytic models of the Gaussianised power spectrum yield good fits to the simulations and produce unbiased parameter estimates in the majority of cases, where the exceptions can be traced back to the limitations in modelling the higher-order correlations of the original convergence. In the idealistic case, without galaxy shape noise, we find an increase in cumulative signal-to-noise by a factor of 2.6 for angular frequencies up to 1500, and a decrease in the area of the confidence region in the Omega_m-sigma_8 plane by a factor of 4.4 in terms of q-values for the best-performing transformation. When adding a realistic level of shape noise, all transformations perform poorly with little decorrelation of angular frequencies, a maximum increase in signal-to-noise of 34%, and even marginally degraded errors on cosmological parameters. We argue that, to find Gaussianising transformations of practical use, one will need to go beyond transformations of the one-point distribution of the convergence, extend the analysis deeper into the non-linear regime, and resort to an exploration of parameter space via simulations. (abridged)

Correlations between the intrinsic shapes of galaxies and the large-scale galaxy density field provide an important tool to investigate galaxy intrinsic alignments, which constitute a major astrophysical systematic in cosmological weak lensing (cosmic shear) surveys, but also yield insight into the formation and evolution of galaxies. We measure galaxy position-shape correlations in the MegaZ-LRG sample for more than 800,000 luminous red galaxies, making the first such measurement with a photometric redshift sample. In combination with a re-analysis of several spectroscopic SDSS samples, we constrain an intrinsic alignment model for early-type galaxies over long baselines in redshift (z ~ 0.7) and luminosity (4mag). We develop and test the formalism to incorporate photometric redshift scatter in the modelling. For r_p > 6 Mpc/h, the fits to galaxy position-shape correlation functions are consistent with the scaling with r_p and redshift of a revised, nonlinear version of the linear alignment model for all samples. An extra redshift dependence proportional to (1+z)^n is constrained to n=-0.3+/-0.8 (1sigma). To obtain consistent amplitudes for all data, an additional dependence on galaxy luminosity proportional to L^b with b=1.1+0.3-0.2 is required. The normalisation of the intrinsic alignment power spectrum is found to be (0.077 +/- 0.008)/rho_{cr} for galaxies at redshift 0.3 and r band magnitude of -22 (k- and evolution-corrected to z=0). Assuming zero intrinsic alignments for blue galaxies, we assess the bias on cosmological parameters for a tomographic CFHTLS-like lensing survey. Both the resulting mean bias and its uncertainty are smaller than the 1sigma statistical errors when using the constraints from all samples combined. The addition of MegaZ-LRG data reduces the uncertainty in intrinsic alignment bias on cosmological parameters by factors of three to seven. (abridged)

With the growing interest in and ability of using weak lensing studies to probe the non-Gaussian properties of the matter density field, there is an increasing need for the study of suitable statistical measures, e.g. shear three-point statistics. In this paper we establish the relations between the three-point configuration space shear and convergence statistics, which are an important missing link between different weak lensing three-point statistics and provide an alternative way of relating observation and theory. The method we use also allows us to derive the relations between other two- and three-point correlation functions. We show the consistency of the relations obtained with already established results and demonstrate how they can be evaluated numerically. As a direct application, we use these relations to formulate the condition for E/B-mode decomposition of lensing three-point statistics, which is the basis for constructing new three-point statistics which allow for exact E/B-mode separation. Our work applies also to other two-dimensional polarization fields such as that of the Cosmic Microwave Background.

We investigate the extent to which the pure magnification effect of gravitational lensing can be extracted from galaxy clustering statistics, by a nulling method which aims to eliminate terms arising from the intrinsic clustering of galaxies. The aim is to leave statistics which are free from the uncertainties of galaxy bias. We find that nulling can be done effectively, leaving data which are relatively insensitive to uncertainties in galaxy bias and its evolution, leading to cosmological parameter estimation which is effectively unbiased. This advantage comes at the expense of increased statistical errors, which are in some cases large, but it offers a robust alternative analysis method to cosmic shear for cosmological imaging surveys designed for weak lensing studies, or to full modelling of the clustering signal including magnification effects.

Three-point weak lensing statistics provide cosmic information complementary to that of two-point statistics. However, both statistics suffer from intrinsic-shear alignment, which is one of their limiting systematics. The nulling technique is a model-independent method developed to eliminate intrinsic-shear alignment at the two-point level. In this paper we demonstrate that the nulling technique can also be naturally generalized to the three-point level, controlling the corresponding GGI systematics. We show that under the assumption of exact redshift information the intrinsic-shear alignment contamination can be completely eliminated. To show how well the nulling technique performs on data with limited redshift information, we apply the nulling technique to three-point weak lensing statistics from a fictitious survey analogous to a typical future deep imaging survey, in which the three-point intrinsic-shear alignment systematics is generated from a power-law toy model. Using 10 redshift bins, the nulling technique leads to a factor of 10 suppression of the GGI/GGG ratio, and reduces the bias on cosmological parameters to less than the original statistical error. More detailed redshift information allowing for finer redshift bins leads to better reduction of bias. The information loss during the nulling procedure doubles the statistical error on cosmological parameters. A comparison of the nulling technique with an unconditioned compression of the data suggests that part of the information loss can be retained by considering higher-order nulling weights during the nulling procedure. A combined analysis of two- and three-point statistics confirms that the information contained in them is of comparable size and is complementary to each other, both before and after nulling.

Cosmic shear is a powerful method to constrain cosmology, provided that any systematic effects are under control. The intrinsic alignment of galaxies is expected to severely bias parameter estimates if not taken into account. We explore the potential of a joint analysis of tomographic galaxy ellipticity, galaxy number density, and ellipticity-number density cross-correlations to simultaneously constrain cosmology and self-calibrate unknown intrinsic alignment and galaxy bias contributions. We treat intrinsic alignments and galaxy biasing as free functions of scale and redshift and marginalise over the resulting parameter sets. Constraints on cosmology are calculated by combining the likelihoods from all two-point correlations between galaxy ellipticity and galaxy number density. The information required for these calculations is already available in a standard cosmic shear dataset. We include contributions to these functions from cosmic shear, intrinsic alignments, galaxy clustering and magnification effects. In a Fisher matrix analysis we compare our constraints with those from cosmic shear alone in the absence of intrinsic alignments. For a potential future large area survey, such as Euclid, the extra information from the additional correlation functions can make up for the additional free parameters in the intrinsic alignment and galaxy bias terms, depending on the flexibility in the models. For example, the Dark Energy Task Force figure of merit is recovered even when more than 100 free parameters are marginalised over. We find that the redshift quality requirements are similar to those calculated in the absence of intrinsic alignments.

The intrinsic alignment of galaxies constitutes the major astrophysical source of systematic errors in surveys of weak gravitational lensing by the large-scale structure. We discuss the principles, summarise the implementation, and highlight the performance of two model-independent methods that control intrinsic alignment signals in weak lensing data: the nulling technique which eliminates intrinsic alignments to ensure unbiased constraints on cosmology, and the boosting technique which extracts intrinsic alignments and hence allows one to further study this contribution. Making only use of the characteristic dependence on redshift of the signals, both approaches are robust, but reduce the statistical power due to the similar redshift scaling of intrinsic alignment and lensing signals.

Intrinsic alignments constitute the major astrophysical systematic for cosmological weak lensing surveys. We present a purely geometrical method with which one can study gravitational shear-intrinsic ellipticity correlations directly in weak lensing data. Linear combinations of second-order cosmic shear measures are constructed such that the intrinsic alignment signal is boosted while suppressing the contribution by gravitational lensing. We then assess the performance of a specific parametrisation of the weights entering these linear combinations for three representative survey models. Moreover a relation between this boosting technique and the intrinsic alignment removal via nulling is derived. For future all-sky weak lensing surveys with photometric redshift information the boosting technique yields statistical errors on model parameters of intrinsic alignments whose order of magnitude is compatible with current constraints determined from indirect measurements. Parameter biases due to a residual cosmic shear signal are negligible in case of quasi-spectroscopic redshifts and remain sub-dominant for typical values of the photometric redshift scatter. We find good agreement between the performance of the intrinsic alignment removal based on the boosting technique and standard nulling methods, possibly indicating a fundamental limit in the separation of lensing and intrinsic alignment signals.

We present a tomographic cosmological weak lensing analysis of the HST COSMOS Survey. Applying our lensing-optimized data reduction, principal component interpolation for the ACS PSF, and improved modelling of charge-transfer inefficiency, we measure a lensing signal which is consistent with pure gravitational modes and no significant shape systematics. We carefully estimate the statistical uncertainty from simulated COSMOS-like fields obtained from ray-tracing through the Millennium Simulation. We test our pipeline on simulated space-based data, recalibrate non-linear power spectrum corrections using the ray-tracing, employ photometric redshifts to reduce potential contamination by intrinsic galaxy alignments, and marginalize over systematic uncertainties. We find that the lensing signal scales with redshift as expected from General Relativity for a concordance LCDM cosmology, including the full cross-correlations between different redshift bins. For a flat LCDM cosmology, we measure sigma_8(Omega_m/0.3)^0.51=0.75+-0.08 from lensing, in perfect agreement with WMAP-5, yielding joint constraints Omega_m=0.266+0.025-0.023, sigma_8=0.802+0.028-0.029 (all 68% conf.). Dropping the assumption of flatness and using HST Key Project and BBN priors only, we find a negative deceleration parameter q_0 at 94.3% conf. from the tomographic lensing analysis, providing independent evidence for the accelerated expansion of the Universe. For a flat wCDM cosmology and prior w in [-2,0], we obtain w<-0.41 (90% conf.). Our dark energy constraints are still relatively weak solely due to the limited area of COSMOS. However, they provide an important demonstration for the usefulness of tomographic weak lensing measurements from space. (abridged)

To probe cosmological fields beyond the Gaussian level, three-point statistics can be used, all of which are related to the bispectrum. Hence, measurements of CMB anisotropies, galaxy clustering, and weak gravitational lensing alike have to rely upon an accurate theoretical background concerning the bispectrum and its noise properties. If only small portions of the sky are considered, it is often desirable to perform the analysis in the flat-sky limit. We aim at a formal, detailed derivation of the bispectrum covariance in the flat-sky approximation, focusing on a pure two-dimensional Fourier-plane approach. We define an unbiased estimator of the bispectrum, which takes the average over the overlap of annuli in Fourier space, and compute its full covariance. The outcome of our formalism is compared to the flat-sky spherical harmonic approximation in terms of the covariance, the behavior under parity transformations, and the information content. We introduce a geometrical interpretation of the averaging process in the estimator, thus providing an intuitive understanding. Contrary to foregoing work, we find a difference by a factor of two between the covariances of the Fourier-plane and the spherical harmonic approach. We argue that this discrepancy can be explained by the differing behavior with respect to parity. However, in an exemplary analysis it is demonstrated that the Fisher information of both formalisms agrees to high accuracy. Via the geometrical interpretation we are able to link the normalization in the bispectrum estimator to the area enclosed by the triangle configuration at consideration as well as to the Wigner symbol, which leads to convenient approximation formulae for the covariances of both approaches.

Cosmic shear is regarded one of the most powerful probes to reveal the properties of dark matter and dark energy. To fully utilize its potential, one has to be able to control systematic effects down to below the level of the statistical parameter errors. Particularly worrisome in this respect is intrinsic alignment, causing considerable parameter biases via correlations between the intrinsic ellipticities of galaxies and the gravitational shear, which mimic lensing. In an earlier work we have proposed a nulling technique that downweights this systematic, only making use of its well-known redshift dependence. We assess the practicability of nulling, given realistic conditions on photometric redshift information. For several simplified intrinsic alignment models and a wide range of photometric redshift characteristics we calculate an average bias before and after nulling. Modifications of the technique are introduced to optimize the bias removal and minimize the information loss by nulling. We demonstrate that one of the presented versions is close to optimal in terms of bias removal, given high quality of photometric redshifts. For excellent photometric redshift information, i.e. at least 10 bins with a small dispersion, a negligible fraction of catastrophic outliers, and precise knowledge about the redshift distributions, one version of nulling is capable of reducing the shear-intrinsic ellipticity contamination by at least a factor of 100. Alternatively, we describe a robust nulling variant which suppresses the systematic signal by about 10 for a very broad range of photometric redshift configurations. Irrespective of the photometric redshift quality, a loss of statistical power is inherent to nulling, which amounts to a decrease of the order 50% in terms of our figure of merit.

In the absence of any compelling physical model, cosmological systematics are often misrepresented as statistical effects and the approach of marginalising over extra nuisance systematic parameters is used to gauge the effect of the systematic. In this article we argue that such an approach is risky at best since the key choice of function can have a large effect on the resultant cosmological errors. As an alternative we present a functional form filling technique in which an unknown, residual, systematic is treated as such. Since the underlying function is unknown we evaluate the effect of every functional form allowed by the information available (either a hard boundary or some data). Using a simple toy model we introduce the formalism of functional form filling. We show that parameter errors can be dramatically affected by the choice of function in the case of marginalising over a systematic, but that in contrast the functional form filling approach is independent of the choice of basis set. We then apply the technique to cosmic shear shape measurement systematics and show that a shear calibration bias of |m(z)|< 0.001(1+z)^0.7 is required for a future all-sky photometric survey to yield unbiased cosmological parameter constraints to percent accuracy. A module associated with the work in this paper is available through the open source iCosmo code available at http://www.icosmo.org .

To render cosmic shear an astronomical tool of high precision, it is essential to eliminate systematic effects upon its signal, one of the most significant ones being correlations between the gravitational shear and the intrinsic ellipticity of source galaxies. Because of the crudeness of current models of intrinsic alignment, we have developed a model-independent, purely geometrical method to remove shear-ellipticity correlations. We eliminate the contributions to a tomographic cosmic shear signal that may be subject to contamination by shear-ellipticity correlations, making use of the characteristic dependence of these correlations on redshift. By introducing an appropriately chosen weight function to the lensing efficiency that nulls signals stemming from certain distances, new second-order measures of cosmic shear, free of intrinsic alignment, can be constructed. The loss of information induced by this nulling technique and the subsequent degradation of constraints on cosmological parameters is quantified in a likelihood analysis. Concerning the construction of optimal weight functions, good agreement is achieved between all approaches considered. For a survey divided into 20 redshift bins we find that the area of credible regions increases by 20% up to about 50% after the application of nulling, depending on the cosmological parameters considered. We demonstrate that due to an optimization of the weight functions nearly all information is contained in a small subset of the new second-order measures. The use of a significantly smaller number of redshift bins than 20 for the nulling considerably degrades parameter constraints under conservative assumptions, emphasizing the need for detailed redshift information.

In recent years cosmic shear, the weak gravitational lensing effect by the large-scale structure of the Universe, has proven to be one of the observational pillars on which the cosmological concordance model is founded. Several cosmic shear statistics have been developed in order to analyze data from surveys. For the covariances of the prevalent second-order measures we present simple and handy formulae, valid under the assumptions of Gaussian density fluctuations and a simple survey geometry. We also formulate these results in the context of shear tomography, i.e. the inclusion of redshift information, and generalize them to arbitrary data field geometries. We define estimators for the E- and B-mode projected power spectra and show them to be unbiased in the case of Gaussianity and a simple survey geometry. From the covariance of these estimators we demonstrate how to derive covariances of arbitrary combinations of second-order cosmic shear measures. We then recalculate the power spectrum covariance for general survey geometries and examine the bias thereby introduced on the estimators for exemplary configurations. Our results for the covariances are considerably simpler than and analytically shown to be equivalent to the real-space approach presented in the first paper of this series. We find good agreement with other numerical evaluations and confirm the general properties of the covariance matrices. The studies of the specific survey configurations suggest that our simplified covariances may be employed for realistic survey geometries to good approximation.