We present weak lensing shear catalogues from the fourth data release of the Kilo-Degree Survey, KiDS-1000, spanning 1006 square degrees of deep and high-resolution imaging. Our `gold-sample' of galaxies, with well calibrated photometric redshift distributions, consists of 21 million galaxies with an effective number density of $\sim 6.2$ galaxies per square arcminute. We quantify the accuracy of the spatial, temporal and flux-dependent point-spread function (PSF) model, verifying that the model meets our requirements to induce less than a $0.1\sigma$ change in the inferred cosmic shear constraints on the clustering cosmological parameter $S_8 = \sigma_8\sqrt{\Omega_{\rm m}/0.3}$. Through a series of two-point null-tests we validate the shear estimates, finding no evidence for significant non-lensing B-mode distortions in the data. PSF residuals are detected in the highest-redshift bins, originating from object selection and/or weight bias. The amplitude is however shown to be sufficiently low and within our stringent requirements. With a shear-ratio null-test we verify the expected redshift scaling of the galaxy-galaxy lensing signal around luminous red galaxies. We conclude that the joint KiDS-1000 shear and photometric redshift calibration is sufficiently robust for combined-probe gravitational lensing and spectroscopic clustering analyses.

On the scale of galactic haloes, the distribution of matter in the cosmos is affected by energetic, non-gravitational processes; so-called baryonic feedback. A lack of knowledge about the details of how feedback processes redistribute matter is a source of uncertainty for weak-lensing surveys, which accurately probe the clustering of matter in the Universe over a wide range of scales. We develop a cosmology-dependent model for the matter distribution that simultaneously accounts for the clustering of dark matter, gas and stars. We inform our model by comparing it to power spectra measured from the BAHAMAS suite of hydrodynamical simulations. As well as considering matter power spectra, we also consider spectra involving the electron-pressure field, which directly relates to the thermal Sunyaev-Zel'dovich (tSZ) effect. We fit parameters in our model so that it can simultaneously model both matter and pressure data and such that the distribution of gas as inferred from tSZ has influence on the matter spectrum predicted by our model. We present two variants; one that matches the feedback-induced suppression seen in the matter-matter power spectrum at the per-cent level and a second that matches the matter-matter data slightly less well (~2 per cent), but that is able to simultaneously model the matter-electron pressure spectrum at the ~15 per-cent level. We envisage our models being used to simultaneously learn about cosmological parameters and the strength of baryonic feedback using a combination of tSZ and lensing auto- and cross-correlation data.

We present the methodology for a joint cosmological analysis of weak gravitational lensing from the fourth data release of the ESO Kilo-Degree Survey (KiDS-1000) and galaxy clustering from the partially overlapping BOSS and 2dFLenS surveys. Cross-correlations between galaxy positions and ellipticities are incorporated into the analysis, developing a hybrid model of non-linear scales that blends perturbative and non-perturbative approaches, and assessing signal contributions by astrophysical effects. All weak lensing signals are measured consistently via Fourier-space statistics that are insensitive to the survey mask and display low levels of mode mixing. The calibration of photometric redshift distributions and multiplicative gravitational shear bias is updated, and a more complete tally of residual calibration uncertainties is propagated into the likelihood. A dedicated suite of more than 20000 mocks is used to assess the performance of covariance models and to quantify the impact of survey geometry and spatial variations of survey depth on signals and their errors. The sampling distributions for the likelihood and the $\chi^2$ goodness-of-fit statistic are validated, with proposed changes to calculating the effective number of degrees of freedom. Standard weak lensing point estimates on $S_8=\sigma_8\,(\Omega_{\rm m}/0.3)^{1/2}$ derived from its marginal posterior are easily misinterpreted to be biased low, and an alternative estimator and associated credible interval are proposed. Known systematic effects pertaining to weak lensing modelling and inference are shown to bias $S_8$ by no more than 0.1 standard deviations, with the caveat that no conclusive validation data exist for models of intrinsic galaxy alignments. Compared to the previous KiDS analyses, $S_8$ constraints are expected to improve by 20% for weak lensing alone and by 29% for the joint analysis. [abridged]

Measuring cosmic shear in wide-field imaging surveys requires accurate knowledge of the redshift distribution of all sources. The clustering-redshift technique exploits the angular cross-correlation of a target galaxy sample with unknown redshifts and a reference sample with known redshifts, and is an attractive alternative to colour-based methods of redshift calibration. We test the performance of such clustering redshift measurements using mock catalogues that resemble the Kilo-Degree Survey (KiDS). These mocks are created from the MICE simulation and closely mimic the properties of the KiDS source sample and the overlapping spectroscopic reference samples. We quantify the performance of the clustering redshifts by comparing the cross-correlation results with the true redshift distributions in each of the five KiDS photometric redshift bins. Such a comparison to an informative model is necessary due to the incompleteness of the reference samples at high redshifts. Clustering mean redshifts are unbiased at $|\Delta z|<0.006$ under these conditions. The redshift evolution of the galaxy bias can be reliably mitigated at this level of precision using auto-correlation measurements and self-consistency relations, and will not become a dominant source of systematic error until the arrival of Stage-IV cosmic shear surveys. Using redshift distributions from a direct colour-based estimate instead of the true redshift distributions as a model for comparison with the clustering redshifts increases the biases in the mean to up to $|\Delta z|\sim0.04$. This indicates that the interpretation of clustering redshifts in real-world applications will require more sophisticated (parameterised) models of the redshift distribution in the future. If such better models are available, the clustering-redshift technique promises to be a highly complementary alternative to other methods of redshift calibration.

Context. The density split statistics in weak gravitational lensing analyses probes the correlation between regions of different (foreground) galaxy number densities and their weak lensing signal, measured by the shape distortion of background galaxies. Aims. In this paper, we reconsider density split statistics, by constructing a new angular filter function that is adapted to the expected relation between galaxy number density and shear pattern, in a way that the filter weighting the galaxy number density is matched to the filter that is used to quantify the shear signal. Methods. We use the results of numerical ray-tracing simulations, specifically through Millennium Simulation supplemented by a galaxy distribution based on a semi-analytic model, to construct a matched pair of adapted filter functions for the galaxy density and the tangential shear signal. We compare the performance of our new filter to the previously used top-hat filter, applying both to a different and independent set of numerical simulations (SLICS, cosmo-SLICS). Results. We show that the adapted filter yields a better correlation between the total matter and the galaxy distribution. Furthermore, the adapted filter provides a larger signal-to-noise ratio to constrain the bias between the total matter and the galaxy distribution, and we show that it is, in general, a more sensitive discriminator between different cosmologies, with the exception of cosmologies with very large $\sigma_8$ values. All analyses lead to the conclusion that our adapted filter should be favored in future density split statistic works.

We simultaneously present constraints on the stellar-to-halo mass relation for central and satellite galaxies through a weak lensing analysis of spectroscopically classified galaxies. Using overlapping data from the 4th data release of the Kilo-Degree Survey (KiDS), and the Galaxy And Mass Assembly survey (GAMA), we find that satellite galaxies are hosted by halo masses that are $0.53 \pm 0.39$ dex (68% confidence, $3\sigma$ detection) smaller than those of central galaxies of the same stellar mass (for a stellar mass of $\log(M_{\star}/M_{\odot}) = 10.6$). This is consistent with galaxy formation models, whereby infalling satellite galaxies are preferentially stripped of their dark matter. We find consistent results with similar uncertainties when comparing constraints from a standard azimuthally averaged galaxy-galaxy lensing analysis and a two-dimensional likelihood analysis of the full shear field. As the latter approach is somewhat biased due to the lens incompleteness, and does not provide any improvement in precision when applied to actual data, we conclude that stacked tangential shear measurements are best-suited for studies of the galaxy-halo connection.

Several semi-analytic models (SAMs) try to explain how galaxies form, evolve and interact inside the dark matter large-scale structure. These SAMs can be tested by comparing their predictions for galaxy-galaxy-galaxy-lensing (G3L), which is weak gravitational lensing around galaxy pairs, with observations. We evaluate the SAMs by Henriques et al. (2015; H15) and by Lagos et al. (2012; L12), implemented in the Millennium Run, by comparing their predictions for G3L to observations at smaller scales than previous studies and also for pairs of lens galaxies from different populations. We compare the G3L signal predicted by the SAMs to measurements in the overlap of the Galaxy And Mass Assembly survey (GAMA), the Kilo-Degree Survey (KiDS), and the VISTA Kilo-degree Infrared Galaxy survey (VIKING), splitting lens galaxies into two colour and five stellar-mass samples. Using an improved G3L estimator, we measure the three-point correlation of the matter distribution for mixed lens pairs with galaxies from different samples, and unmixed lens pairs with galaxies from the same sample. Predictions by the H15 SAM agree with the observations for all colour-selected and all but one stellar-mass-selected sample with 95% confidence. Deviations occur for lenses with stellar masses below $9.5h^{-2}\mathrm{M}_\odot$ at scales below $0.2h^{-1}\mathrm{Mpc}$. Predictions by the L12 SAM for stellar-mass selected samples and red galaxies are significantly higher than observed, while the predicted signal for blue galaxy pairs is too low. The L12 SAM predicts more pairs of small stellar-mass and red galaxies than the H15 SAM and the observations, as well as fewer pairs of blue galaxies. Likely explanations are different treatments of environmental effects by the SAMs and different models of the initial mass function. We conclude that G3L provides a stringent test for models of galaxy formation and evolution.

The physics of gravity on cosmological scales affects both the rate of assembly of large-scale structure, and the gravitational lensing of background light through this cosmic web. By comparing the amplitude of these different observational signatures, we can construct tests that can distinguish General Relativity from its potential modifications. We use the latest weak gravitational lensing dataset from the Kilo-Degree Survey, KiDS-1000, in conjunction with overlapping galaxy spectroscopic redshift surveys BOSS and 2dFLenS, to perform the most precise existing amplitude-ratio test. We measure the associated E_G statistic with 15-20% errors, in five dz = 0.1 tomographic redshift bins in the range 0.2 < z < 0.7, on projected scales up to 100 Mpc/h. The scale-independence and redshift-dependence of these measurements are consistent with the theoretical expectation of General Relativity in a Universe with matter density Omega_m = 0.27 +/- 0.04. We demonstrate that our results are robust against different analysis choices, including schemes for correcting the effects of source photometric redshift errors, and compare the performance of angular and projected galaxy-galaxy lensing statistics.

To effectively exploit large-scale structure surveys, we depend on accurate and reliable predictions of non-linear cosmological structure formation. Tools for efficient and comprehensive computational modelling are therefore essential to perform cosmological parameter inference analyses. We present the public software package ReACT, demonstrating its capability for the fast and accurate calculation of non-linear power spectra from non-standard physics. We showcase ReACT through a series of forecast constraints on the DGP and $f(R)$ gravity models, adopting LSST-like cosmic shear power spectra. Accurate non-linear modelling with ReACT more than doubles LSST's constraining power on the $f(R)$ parameter, in contrast to an analysis that is limited to the quasi-linear regime. We find that ReACT is sufficiently robust for the inference of consistent constraints on theories beyond $\Lambda$CDM for current and ongoing surveys. With further improvement, particularly in terms of the accuracy of the non-linear $\Lambda$CDM power spectrum, ReACT can, in principle, meet the accuracy requirements for future surveys such as Euclid and LSST.

We present updated cosmological constraints for the KiDS+VIKING-450 cosmic shear dataset (KV450), estimated using redshift distributions and photometric samples defined using self organising maps (SOMs). Our fiducial analysis finds marginal posterior constraints of $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.716^{+0.043}_{-0.038}$; smaller than, but fully consistent with, previous work using this dataset ($|\Delta S_8| = 0.023$). We analyse additional samples and redshift distributions constructed in three ways: excluding certain spectroscopic surveys during redshift calibration, excluding lower-confidence spectroscopic redshifts in redshift calibration, and considering only photometric sources which are jointly calibrated by at least three spectroscopic surveys. In all cases, the method utilised here proves robust: we find a maximal deviation from our fiducial analysis of $|\Delta S_8| \leq 0.009$ for all samples defined and analysed using our SOM. Our largest shift in $S_8$ is found when calibrating redshift distributions without the DEEP2 spectroscopic subset, where we find $S_8=0.707_{-0.042}^{+0.046}$. This difference with respect to the fiducial is both significantly smaller than, and in the opposite direction to, the equivalent shift from previous work. No sample analysed in this work results in a meaningful positive shift in $S_8$ with respect to our fiducial constraints. These results suggest that our improved cosmological parameter estimates are insensitive to pathological misrepresentation of photometric sources by the spectroscopy used for direct redshift calibration, and therefore that this systematic effect cannot be responsible for the observed difference between $S_8$ estimates made with KV450 and Planck CMB probes.

We present a combined tomographic weak gravitational lensing analysis of the Kilo Degree Survey (KV450) and the Dark Energy Survey (DES-Y1). We homogenize the analysis of these two public cosmic shear datasets by adopting consistent priors and modeling of nonlinear scales, and determine new redshift distributions for DES-Y1 based on deep public spectroscopic surveys. Adopting these revised redshifts results in a $0.8\sigma$ reduction in the DES-inferred value for $S_8$, which decreases to a $0.5\sigma$ reduction when including a systematic redshift calibration error model from mock DES data based on the MICE2 simulation. The combined KV450 + DES-Y1 constraint on $S_8 = 0.762^{+0.025}_{-0.024}$ is in tension with the Planck 2018 constraint from the cosmic microwave background at the level of $2.5\sigma$. This result highlights the importance of developing methods to provide accurate redshift calibration for current and future weak lensing surveys.

We present a mitigation strategy to reduce the impact of non-linear galaxy bias on the joint `$3 \times 2 $pt' cosmological analysis of weak lensing and galaxy surveys. The $\Psi$-statistics that we adopt are based on Complete Orthogonal Sets of E/B Integrals (COSEBIs). As such they are designed to minimise the contributions to the observable from the smallest physical scales where models are highly uncertain. We demonstrate that $\Psi$-statistics carry the same constraining power as the standard two-point galaxy clustering and galaxy-galaxy lensing statistics, but are significantly less sensitive to scale-dependent galaxy bias. Using two galaxy bias models, motivated by halo-model fits to data and simulations, we quantify the error in a standard $3 \times 2$pt analysis where constant galaxy bias is assumed. Even when adopting conservative angular scale cuts, that degrade the overall cosmological parameter constraints, we find of order $1 \sigma$ biases for Stage III surveys on the cosmological parameter $S_8 = \sigma_8(\Omega_{\rm m}/0.3)^{\alpha}$. This arises from a leakage of the smallest physical scales to all angular scales in the standard two-point correlation functions. In contrast, when analysing $\Psi$-statistics under the same approximation of constant galaxy bias, we show that the bias on the recovered value for $S_8$ can be decreased by a factor of $\sim 2$, with less conservative scale cuts. Given the challenges in determining accurate galaxy bias models in the highly non-linear regime, we argue that $3 \times 2$pt analyses should move towards new statistics that are less sensitive to the smallest physical scales.

Accurate photometric redshift calibration is central to the robustness of all cosmology constraints from cosmic shear surveys. Analyses of the KiDS re-weighted training samples from all overlapping spectroscopic surveys to provide a direct redshift calibration. Using self-organising maps (SOMs) we demonstrate that this spectroscopic compilation is sufficiently complete for KiDS, representing $99\%$ of the effective 2D cosmic shear sample. We use the SOM to define a $100\%$ represented `gold' cosmic shear sample, per tomographic bin. Using mock simulations of KiDS and the spectroscopic training set, we estimate the uncertainty on the SOM redshift calibration, and find that photometric noise, sample variance, and spectroscopic selection effects (including redshift and magnitude incompleteness) induce a combined maximal scatter on the bias of the redshift distribution reconstruction ($\Delta \langle z \rangle=\langle z \rangle_{\rm est}-\langle z \rangle_{\rm true}$) of $\sigma_{\Delta \langle z \rangle} \leq 0.006$ in all tomographic bins. We show that the SOM calibration is unbiased in the cases of noiseless photometry and perfectly representative spectroscopic datasets, as expected from theory. The inclusion of both photometric noise and spectroscopic selection effects in our mock data introduces a maximal bias of $\Delta \langle z \rangle =0.013\pm0.006$, or $\Delta \langle z \rangle \leq 0.025$ at $97.5\%$ confidence, once quality flags have been applied to the SOM. The method presented here represents a significant improvement over the previously adopted direct redshift calibration implementation for KiDS, owing to its diagnostic and quality assurance capabilities. The implementation of this method in future cosmic shear studies will allow better diagnosis, examination, and mitigation of systematic biases in photometric redshift calibration.

We present cosmological constraints from a joint cosmic shear analysis of the Kilo-Degree Survey (KV450) and the Dark Energy Survey (DES-Y1), conducted using Complete Orthogonal Sets of E/B-Integrals (COSEBIs). With COSEBIs we isolate any B-modes which have a non-cosmic shear origin and demonstrate the robustness of our cosmological E-mode analysis as no significant B-modes are detected. We highlight how COSEBIs are fairly insensitive to the amplitude of the non-linear matter power spectrum at high $k$-scales, mitigating the uncertain impact of baryon feedback in our analysis. COSEBIs, therefore, allow us to utilise additional small-scale information, improving the DES-Y1 joint constraints on $S_8=\sigma_8(\Omega_{\rm m}/0.3)^{0.5}$ and $\Omega_{\rm m}$ by $20\%$. Adopting a flat $\Lambda$CDM model we find $S_8=0.755^{+0.019}_{-0.021}$, which is in $3.2\sigma$ tension with the Planck Legacy analysis of the cosmic microwave background.

We reanalyse the anisotropic galaxy clustering measurement from the Baryon Oscillation Spectroscopic Survey (BOSS), demonstrating that using the full shape information provides cosmological constraints that are comparable to other low-redshift probes. We find $\Omega_\mathrm{m} = 0.317^{+0.015}_{-0.019}$, $\sigma_8 = 0.710\pm 0.049$, and $h = 0.704\pm 0.024$ for flat $\Lambda$CDM cosmologies using uninformative priors on $\Omega_\mathrm{c}h^2$, $100\theta_\mathrm{MC}$, $\ln 10^{10} A_{s}$, and $n_{s}$, and a prior on $\Omega_\mathrm{b}h^2$ that is much wider than current constraints. We quantify the agreement between the Planck 2018 constraints from the cosmic microwave background and BOSS, finding the two data sets to be consistent within a flat $\Lambda$CDM cosmology using the Bayes factor as well as the prior-insensitive suspiciousness statistic. Combining two low-redshift probes, we jointly analyse the clustering of BOSS galaxies with weak lensing measurements from the Kilo-Degree Survey (KV450). The combination of BOSS and KV450 improves the measurement by up to 45%, constraining $\sigma_8 = 0.702\pm 0.029$ and $S_8 = \sigma_8\sqrt{\Omega_\mathrm{m}/0.3} = 0.728\pm 0.026$. Over the full 5D parameter space, the odds in favour of single cosmology describing galaxy clustering, lensing, and the cosmic microwave background are $7\pm2$. The suspiciousness statistic signals a $2.1\pm0.3\sigma$ tension between the combined low-redshift probes and measurements from the cosmic microwave background.

We present a semi-analytic model for the shear two-point correlation function of a cosmic shear survey with non-uniform depth. Ground-based surveys are subject to depth variations that primarily arise through varying atmospheric conditions. For a survey like the Kilo-Degree Survey (KiDS), we find that the measured depth variation increases the amplitude of the observed shear correlation function at the level of a few percent out to degree-scales, relative to the assumed uniform-depth case. The impact on the inferred cosmological parameters is shown to be insignificant for a KiDS-like survey. For next-generation cosmic shear experiments, however, we conclude that variable depth should be accounted for.

We describe the observations and resultant galaxy cluster catalog from the 2770 deg$^2$ SPTpol Extended Cluster Survey (SPT-ECS). Clusters are identified via the Sunyaev-Zel'dovich (SZ) effect, and confirmed with a combination of archival and targeted follow-up data, making particular use of data from the Dark Energy Survey (DES). With incomplete followup we have confirmed as clusters 244 of 266 candidates at a detection significance $\xi \ge 5$ and an additional 204 systems at $4<\xi<5$. The confirmed sample has a median mass of $M_{500c} \sim {4.4 \times 10^{14} M_\odot h_{70}^{-1}}$, a median redshift of $z=0.49$, and we have identified 44 strong gravitational lenses in the sample thus far. Radio data are used to characterize contamination to the SZ signal; the median contamination for confirmed clusters is predicted to be $\sim$1% of the SZ signal at the $\xi>4$ threshold, and $<4\%$ of clusters have a predicted contamination $>10\% $ of their measured SZ flux. We associate SZ-selected clusters, from both SPT-ECS and the SPT-SZ survey, with clusters from the DES redMaPPer sample, and find an offset distribution between the SZ center and central galaxy in general agreement with previous work, though with a larger fraction of clusters with significant offsets. Adopting a fixed Planck-like cosmology, we measure the optical richness-to-SZ-mass ($\lambda-M$) relation and find it to be 28% shallower than that from a weak-lensing analysis of the DES data---a difference significant at the 4 $\sigma$ level---with the relations intersecting at $\lambda=60$ . The SPT-ECS cluster sample will be particularly useful for studying the evolution of massive clusters and, in combination with DES lensing observations and the SPT-SZ cluster sample, will be an important component of future cosmological analyses.

We present a weak lensing detection of filamentary structures in the cosmic web, combining data from the Kilo-Degree Survey, the Red Cluster Sequence Lensing Survey and the Canada-France-Hawaii Telescope Lensing Survey. The line connecting luminous red galaxies with a separation of $3 - 5\, h^{-1}\text{Mpc}$ is chosen as a proxy for the location of filaments. We measure the average weak lensing shear around $\sim$11,000 candidate filaments selected in this way from the Sloan Digital Sky Survey. After nulling the shear induced by the dark matter haloes around each galaxy, we report a $3.4\,\sigma$ detection of an anisotropic shear signal from the matter that connects them. Adopting a filament density profile, motivated from $N$-body simulations, the average density at the centre of these filamentary structures is found to be $15 \pm 4$ times the critical density.

We analytically model the non-linear effects induced by massive neutrinos on the total matter power spectrum using the halo model reaction framework of Cataneo et al. 2019. In this approach the halo model is used to determine the relative change to the matter power spectrum caused by new physics beyond the concordance cosmology. Using standard fitting functions for the halo abundance and the halo mass-concentration relation, the total matter power spectrum in the presence of massive neutrinos is predicted to percent-level accuracy, out to $k=10 \, h \, {\rm Mpc}^{-1}$. We find that refining the prescriptions for the halo properties using $N$-body simulations improves the recovered accuracy to better than 1%. This paper serves as another demonstration for how the halo model reaction framework, in combination with a single suite of standard $\Lambda$CDM simulations, can recover percent-level accurate predictions for beyond-$\Lambda$CDM matter power spectra, well into the non-linear regime.

We present a tomographic cosmic shear analysis of the Kilo-Degree Survey (KiDS) combined with the VISTA Kilo-Degree Infrared Galaxy Survey (VIKING). This is the first time that a full optical to near-infrared data set has been used for a wide-field cosmological weak lensing experiment. This unprecedented data, spanning $450~$deg$^2$, allows us to improve significantly the estimation of photometric redshifts, such that we are able to include robustly higher-redshift sources for the lensing measurement, and - most importantly - solidify our knowledge of the redshift distributions of the sources. Based on a flat $\Lambda$CDM model we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.737_{-0.036}^{+0.040}$ in a blind analysis from cosmic shear alone. The tension between KiDS cosmic shear and the Planck-Legacy CMB measurements remains in this systematically more robust analysis, with $S_8$ differing by $2.3\sigma$. This result is insensitive to changes in the priors on nuisance parameters for intrinsic alignment, baryon feedback, and neutrino mass. KiDS shear measurements are calibrated with a new, more realistic set of image simulations and no significant B-modes are detected in the survey, indicating that systematic errors are under control. When calibrating our redshift distributions by assuming the 30-band COSMOS-2015 photometric redshifts are correct (following the Dark Energy Survey and the Hyper Suprime-Cam Survey), we find the tension with Planck is alleviated. The robust determination of source redshift distributions remains one of the most challenging aspects for future cosmic shear surveys.

We introduce an emulator approach to predict the non-linear matter power spectrum for broad classes of beyond-$\Lambda$CDM cosmologies, using only a suite of $\Lambda$CDM $N$-body simulations. By including a range of suitably modified initial conditions in the simulations, and rescaling the resulting emulator predictions with analytical `halo model reactions', accurate non-linear matter power spectra for general extensions to the standard $\Lambda$CDM model can be calculated. We optimise the emulator design by substituting the simulation suite with non-linear predictions from the standard {\sc halofit} tool. We review the performance of the emulator for artificially generated departures from the standard cosmology as well as for theoretically motivated models, such as $f (R)$ gravity and massive neutrinos. For the majority of cosmologies we have tested, the emulator can reproduce the matter power spectrum with errors $\lesssim 1\%$ deep into the highly non-linear regime. This work demonstrates that with a well-designed suite of $\Lambda$CDM simulations, extensions to the standard cosmological model can be tested in the non-linear regime without any reliance on expensive beyond-$\Lambda$CDM simulations.

We present the curation and verification of a new combined optical and near infrared dataset for cosmology and astrophysics, derived from the combination of $ugri$-band imaging from the Kilo Degree Survey (KiDS) and $ZY\!J\!H\!K_{\rm s}$-band imaging from the VISTA Kilo degree Infrared Galaxy (VIKING) survey. This dataset is unrivaled in cosmological imaging surveys due to its combination of area ($458$ deg$^2$ before masking), depth ($r\le25$), and wavelength coverage ($ugriZY\!J\!H\!K_{\rm s}$). The combination of survey depth, area, and (most importantly) wavelength coverage allows significant reductions in systematic uncertainties (i.e. reductions of between 10 and 60\% in bias, outlier rate, and scatter) in photometric-to-spectroscopic redshift comparisons, compared to the optical-only case at photo-$z$ above $0.7$. The complementarity between our optical and NIR surveys means that over $80\%$ of our sources, across all photo-$z$, have significant detections (i.e. not upper limits) in our $8$ reddest bands. We derive photometry, photo-$z$, and stellar masses for all sources in the survey, and verify these data products against existing spectroscopic galaxy samples. We demonstrate the fidelity of our higher-level data products by constructing the survey stellar mass functions in 8 volume-complete redshift bins. We find that these photometrically derived mass functions provide excellent agreement with previous mass evolution studies derived using spectroscopic surveys. The primary data products presented in this paper are publicly available at http://kids.strw.leidenuniv.nl/.

We present a general method to compute the nonlinear matter power spectrum for dark energy and modified gravity scenarios with percent-level accuracy. By adopting the halo model and nonlinear perturbation theory, we predict the reaction of a $\Lambda$CDM matter power spectrum to the physics of an extended cosmological parameter space. By comparing our predictions to $N$-body simulations we demonstrate that with no-free parameters we can recover the nonlinear matter power spectrum for a wide range of different $w_0$-$w_a$ dark energy models to better than 1% accuracy out to $k \approx 1 \, h \, {\rm Mpc}^{-1}$. We obtain a similar performance for both DGP and $f(R)$ gravity, with the nonlinear matter power spectrum predicted to better than 3% accuracy over the same range of scales. When including direct measurements of the halo mass function from the simulations, this accuracy improves to 1%. With a single suite of standard $\Lambda$CDM $N$-body simulations, our methodology provides a direct route to constrain a wide range of non-standard extensions to the concordance cosmology in the high signal-to-noise nonlinear regime.

Intrinsic galaxy alignments are a source of bias for weak lensing measurements as well as a tool for understanding galaxy formation and evolution. In this work, we measure the alignment of shapes of satellite galaxies, in galaxy groups, with respect to the brightest group galaxy (BGG), as well as alignments of the BGG shape with the satellite positions, using the highly complete Galaxy And Mass Assembly (GAMA) spectroscopic survey and deep imaging from the Kilo Degree Survey. We control systematic errors with dedicated image simulations and measure accurate shapes using the DEIMOS shape measurement method. We find a significant satellite radial alignment signal, which vanishes at large separations from the BGG. We do not identify any strong trends of the signal with galaxy absolute magnitude or group mass. The alignment signal is dominated by red satellites. We also find that the outer regions of galaxies are aligned more strongly than their inner regions, by varying the radial weight employed during the shape measurement process. This behaviour is evident for both red and blue satellites. BGGs are also found to be aligned with satellite positions, with this alignment being stronger when considering the innermost satellites, using red BGGs and the shape of the outer region of the BGG. Lastly, we measure the global intrinsic alignment signal in the GAMA sample for two different radial weight functions and find no significant difference.

We analyse three public cosmic shear surveys; the Kilo-Degree Survey (KiDS-450), the Dark Energy Survey (DES-SV) and the Canada France Hawaii Telescope Lensing Survey (CFHTLenS). Adopting the COSEBIs statistic to cleanly and completely separate the lensing E-modes from the non-lensing B-modes, we detect B-modes in KiDS-450 and CFHTLenS at the level of about 2.7 $\sigma$. For DES- SV we detect B-modes at the level of 2.8 $\sigma$ in a non-tomographic analysis, increasing to a 5.5 $\sigma$ B-mode detection in a tomographic analysis. In order to understand the origin of these detected B-modes we measure the B-mode signature of a range of different simulated systematics including PSF leakage, random but correlated PSF modelling errors, camera-based additive shear bias and photometric redshift selection bias. We show that any correlation between photometric-noise and the relative orientation of the galaxy to the point-spread-function leads to an ellipticity selection bias in tomographic analyses. This work therefore introduces a new systematic for future lensing surveys to consider. We find that the B-modes in DES-SV appear similar to a superposition of the B-mode signatures from all of the systematics simulated. The KiDS-450 and CFHTLenS B-mode measurements show features that are consistent with a repeating additive shear bias.

The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope, specifically designed for measuring weak gravitational lensing by galaxies and large-scale structure. When completed it will consist of 1350 square degrees imaged in four filters (ugri). Here we present the fourth public data release which more than doubles the area of sky covered by data release 3. We also include aperture-matched ZYJHKs photometry from our partner VIKING survey on the VISTA telescope in the photometry catalogue. We illustrate the data quality and describe the catalogue content. Two dedicated pipelines are used for the production of the optical data. The Astro-WISE information system is used for the production of co-added images in the four survey bands, while a separate reduction of the r-band images using the theli pipeline is used to provide a source catalogue suitable for the core weak lensing science case. All data have been re-reduced for this data release using the latest versions of the pipelines. The VIKING photometry is obtained as forced photometry on the theli sources, using a re-reduction of the VIKING data that starts from the VISTA pawprints. Modifications to the pipelines with respect to earlier releases are described in detail. The photometry is calibrated to the Gaia DR2 G band using stellar locus regression. In this data release a total of 1006 square-degree survey tiles with stacked ugri images are made available, accompanied by weight maps, masks, and single-band source lists. We also provide a multi-band catalogue based on r-band detections, including homogenized photometry and photometric redshifts, for the whole dataset. Mean limiting magnitudes (5 sigma in a 2" aperture) are 24.23, 25.12, 25.02, 23.68 in ugri, respectively, and the mean r-band seeing is 0.70".

Observations of galaxies and galaxy clusters in the local universe can account for only $\sim\,10\%$ of the total baryon content. Cosmological simulations predict that the `missing baryons' are spread throughout filamentary structures in the cosmic web, forming a low-density gas with temperatures of $10^5-10^7\,\!$K. We search for this warm-hot intergalactic medium (WHIM) by stacking the Planck Compton $y$-parameter map of the thermal Sunyaev-Zel'dovich (tSZ) effect for 1,002,334 pairs of CMASS galaxies from the Sloan Digital Sky Survey. We model the contribution from the galaxy halo pairs assuming spherical symmetry, finding a residual tSZ signal at the $2.9\mbox{$\sigma$}$ level from a stacked filament of length $10.5\,h^{-1}\,\rm Mpc$ with a Compton parameter magnitude $y=(0.6\pm0.2)\times10^{-8}$. We consider possible sources of contamination and conclude that bound gas in haloes may contribute only up to $20\%$ of the measured filamentary signal. To estimate the filament gas properties we measure the gravitational lensing signal for the same sample of galaxy pairs; in combination with the tSZ signal, this yields an inferred gas density of $\rho_{\rm b}=(5.5\pm 2.9)\times\bar{\rho_{\rm b}}$ with a temperature $T=(2.7\pm 1.7) \times 10^6\,$K. This result is consistent with the predicted WHIM properties, and overall the filamentary gas can account for $ 11\pm 7\%$ of the total baryon content of the Universe. We also see evidence that the gas filament extends beyond the galaxy pair. Averaging over this longer baseline boosts the significance of the tSZ signal and increases the associated baryon content to $28\pm 12\%$ of the global value.

Exploiting the full statistical power of future cosmic shear surveys will necessitate improvements to the accuracy with which the gravitational lensing signal is measured. We present a framework for calibrating shear with image simulations that demonstrates the importance of including realistic correlations between galaxy morphology, size and more importantly, photometric redshifts. This realism is essential so that selection and shape measurement biases can be calibrated accurately for a tomographic cosmic shear analysis. We emulate Kilo-Degree Survey (KiDS) observations of the COSMOS field using morphological information from {\it Hubble} Space Telescope imaging, faithfully reproducing the measured galaxy properties from KiDS observations of the same field. We calibrate our shear measurements from lensfit, and find through a range of sensitivity tests that lensfit is robust and unbiased within the allowed 2 per cent tolerance of our study. Our results show that the calibration has to be performed by selecting the tomographic samples in the simulations, consistent with the actual cosmic shear analysis, because the joint distributions of galaxy properties are found to vary with redshift. Ignoring this redshift variation could result in misestimating the shear bias by an amount that exceeds the allowed tolerance. To improve the calibration for future cosmic shear analyses, it will be essential to also correctly account for the measurement of photometric redshifts, which requires simulating multi-band observations.

We directly constrain the non-linear alignment (NLA) model of intrinsic galaxy alignments, analysing the most representative and complete flux-limited sample of spectroscopic galaxies available for cosmic shear surveys. We measure the projected galaxy position-intrinsic shear correlations and the projected galaxy clustering signal using high-resolution imaging from the Kilo Degree Survey (KiDS) overlapping with the GAMA spectroscopic survey, and data from the Sloan Digital Sky Survey. Separating samples by colour, we make no significant detection of blue galaxy alignments, constraining the blue galaxy NLA amplitude $A_{\textrm{IA}}^{\textrm{B}}=0.21^{+0.37}_{-0.36}$ to be consistent with zero. We make robust detections ($\sim9\sigma$) for red galaxies, with $A_{\textrm{IA}}^{\textrm{R}}=3.18^{+0.47}_{-0.46}$, corresponding to a net radial alignment with the galaxy density field, and we find no evidence for any scaling of alignments with galaxy luminosity. We provide informative priors for current and future weak lensing surveys, an improvement over de facto wide priors that allow for unrealistic levels of intrinsic alignment contamination. For a colour-split cosmic shear analysis of the final KiDS survey area, we forecast that our priors will improve the constraining power on $S_{8}$ and the dark energy equation of state $w_{0}$, by up to $62\%$ and $51\%$, respectively. Our results indicate, however, that the modelling of red/blue-split galaxy alignments may be insufficient to describe samples with variable central/satellite galaxy fractions.

We study projected underdensities in the cosmic galaxy density field known as 'troughs', and their overdense counterparts, which we call 'ridges'. We identify these regions using a bright sample of foreground galaxies from the photometric Kilo-Degree Survey (KiDS), specifically selected to mimic the spectroscopic Galaxy And Mass Assembly survey (GAMA). Using background galaxies from KiDS, we measure the weak gravitational lensing profiles of the troughs/ridges. We quantify the amplitude of their lensing strength $A$ as a function of galaxy density percentile rank $P$ and galaxy overdensity $\delta$, and find that the skewness in the galaxy density distribution is reflected in the total mass distribution measured by weak lensing. We interpret our results using the mock galaxy catalogue from the Marenostrum Institut de Ci\`encies de l'Espai (MICE) simulation, and find a good agreement with our observations. Using signal-to-noise weights derived from the Scinet LIghtCone Simulations (SLICS) mock catalogue we optimally stack the lensing signal of KiDS troughs with an angular radius $\theta_A$ = {5,10,15,20} arcmin, resulting in {16.8,14.9,10.13,7.55} $\sigma$ detections. Finally, we select troughs using a volume-limited sample of galaxies, split into two redshift bins between 0.1 < z < 0.3. For troughs/ridges with transverse comoving radius $R_A$ = 1.9 Mpc/h, we find no significant difference in the comoving Excess Surface Density as a function of $P$ and $\delta$ between the low- and high-redshift sample. Using the MICE and SLICS mocks we predict that trough and ridge evolution could be detected with gravitational lensing using deeper and wider lensing surveys, such as those from the Large Synoptic Survey Telescope and Euclid.

We demonstrate that a joint analysis of LSST-like ground-based imaging with Euclid-like space-based imaging leads to increased precision and accuracy in galaxy shape measurements. At galaxy magnitudes of $i \sim 24.5$, a combined survey analysis increases the effective galaxy number density for cosmic shear studies by $\sim 50$ percent in comparison to an analysis of each survey alone. Using a realistic distribution of galaxy sizes, ellipticities and magnitudes down to $i = 25.2$, we simulate LSST-like and Euclid-like images of over one million isolated galaxies. We compare the precision and accuracy of the recovered galaxy ellipticities for four different analyses: LSST-only, Euclid-only, a simultaneous joint-pixel analysis of the two surveys, and a simple catalogue-level survey combination. In the faint and small-galaxy regime, where neither survey excels alone, we find a $\sim 20$ percent increase in the precision of galaxy shape measurement when we adopt a joint-pixel analysis, compared to a catalogue-level combination. As the statistical power of cosmic shear is dominated by intrinsic ellipticity noise, however, this improvement in shape measurement noise only leads to a $\sim 5$ percent improvement in the effective number density of galaxies for lensing studies. We view this as the minimum improvement that should be expected from a joint-pixel analysis over a less accurate catalogue-level combination, as the former will also improve the capability of LSST to de-blend close objects.

We present an independent cosmic shear analysis of the non-cosmological B-mode distortions within the public first year data from the Dark Energy Survey (DES). We find no significant detection of B-modes in a full tomographic analysis of the primary METACALIBRATION shear catalogue. This is in contrast to the secondary IM3SHAPE shear catalogue, where we detect B- modes at a significance of $\sim 3\sigma$ with a pattern that is consistent with the B-mode signature of a repeating additive shear bias across the survey. We use the COSEBIs statistic to cleanly separate the B-modes from the gravitational lensing signal (E-modes). We find good agreement between the measured E-modes and their theoretical expectation given the DES cosmological parameter constraints.

We present a new sample of galaxy-scale strong gravitational-lens candidates, selected from 904 square degrees of Data Release 4 of the Kilo-Degree Survey (KiDS), i.e., the "Lenses in the Kilo-Degree Survey" (LinKS) sample. We apply two Convolutional Neural Networks (ConvNets) to $\sim88\,000$ colour-magnitude selected luminous red galaxies yielding a list of 3500 strong-lens candidates. This list is further down-selected via human inspection. The resulting LinKS sample is composed of 1983 rank-ordered targets classified as "potential lens candidates" by at least one inspector. Of these, a high-grade subsample of 89 targets is identified with potential strong lenses by all inspectors. Additionally, we present a collection of another 200 strong lens candidates discovered serendipitously from various previous ConvNet runs. A straightforward application of our procedure to future Euclid or LSST data can select a sample of $\sim3000$ lens candidates with less than 10 per cent expected false positives and requiring minimal human intervention.

We present a new measurement of $E_{\rm G}$, which combines measurements of weak gravitational lensing, real-space galaxy clustering and redshift space distortions. This statistic was proposed as a consistency test of General Relativity (GR) that is insensitive to linear, deterministic galaxy bias and the matter clustering amplitude. We combine deep imaging data from KiDS with overlapping spectroscopy from 2dFLenS, BOSS DR12 and GAMA and find $E_{\rm G}(\overline{z}=0.267)=0.43 \pm 0.13$ (GAMA), $E_{\rm G}(\overline{z}=0.305)=0.27 \pm 0.08$ (LOWZ+2dFLOZ) and $E_{\rm G}(\overline{z}=0.554)=0.26 \pm 0.07$ (CMASS+2dFHIZ). We demonstrate that the existing tension in the value of the matter density parameter hinders the robustness of this statistic as solely a test of GR. We find that our $E_{\rm G}$ measurements, as well as existing ones in the literature, favour a lower matter density cosmology than the Cosmic Microwave Background. For a flat $\Lambda$CDM Universe and assuming GR, we find $\Omega_{\rm m}(z=0)=0.25\pm0.03$. With this paper we publicly release the 2dFLenS dataset at: \url{http://2dflens.swin.edu.au}.

We present a weak gravitational lensing analysis of 815 square degree of $i$-band imaging from the Kilo-Degree Survey (KiDS-$i$-800). In contrast to the deep $r$-band observations, which take priority during excellent seeing conditions and form the primary KiDS dataset (KiDS-$r$-450), the complementary yet shallower KiDS-$i$-800 spans a wide range of observing conditions. The overlapping KiDS-$i$-800 and KiDS-$r$-450 imaging therefore provides a unique opportunity to assess the robustness of weak lensing measurements. In our analysis, we introduce two new `null' tests. The `nulled' two-point shear correlation function uses a matched catalogue to show that the calibrated KiDS-$i$-800 and KiDS-$r$-450 shear measurements agree at the level of $1 \pm 4$\%. We use five galaxy lens samples to determine a `nulled' galaxy-galaxy lensing signal from the full KiDS-$i$-800 and KiDS-$r$-450 surveys and find that the measurements agree to $7 \pm 5$\% when the KiDS-$i$-800 source redshift distribution is calibrated using either spectroscopic redshifts, or the 30-band photometric redshifts from the COSMOS survey.

We present a public suite of weak lensing mock data, extending the Scinet Light Cone Simulations (SLICS) to simulate cross-correlation analyses with different cosmological probes. These mocks include KiDS-450- and LSST-like lensing data, cosmic microwave background lensing maps and simulated spectroscopic surveys that emulate the GAMA, BOSS and 2dFLenS galaxy surveys. With 844 independent realisations, our mocks are optimised for combined-probe covariance estimation, which we illustrate for the case of a joint measurement involving cosmic shear, galaxy-galaxy lensing and galaxy clustering from KiDS-450 and BOSS data. With their high spatial resolution, the SLICS are also optimal for predicting the signal for novel lensing estimators, for the validation of analysis pipelines, and for testing a range of systematic effects such as the impact of neighbour-exclusion bias on the measured tomographic cosmic shear signal. For surveys like KiDS and DES, where the rejection of neighbouring galaxies occurs within ~2 arcseconds, we show that the measured cosmic shear signal will be biased low, but by less than a percent on the angular scales that are typically used in cosmic shear analyses. The amplitude of the neighbour-exclusion bias doubles in deeper, LSST-like data. The simulation products described in this paper are made available at http://slics.roe.ac.uk/.

We present the first "clipped" cosmic shear measurement using data from the Kilo-Degree Survey (KiDS-450). "Clipping" transformations suppress the signal from the highest density, non-linear regions of cosmological fields. We demonstrate that these transformations improve constraints on $S_8=\sigma_8(\Omega_{\rm{m}}/0.3)^{0.5}$ when used in combination with conventional two-point statistics. For the KiDS-450 data, we find that the combined measurements improve the constraints on $S_8$ by 17%, compared to shear correlation functions alone. We determine the expectation value of the clipped shear correlation function using a suite of numerical simulations, and develop methodology to mitigate the impact of masking and shot noise. Future improvements in numerical simulations and mass reconstruction methodology will permit the precise calibration of clipped cosmic shear statistics such that clipping can become a standard tool in weak lensing analyses.

In the past few years, several independent collaborations have presented cosmological constraints from tomographic cosmic shear analyses. These analyses differ in many aspects: the datasets, the shear and photometric redshift estimation algorithms, the theory model assumptions, and the inference pipelines. To assess the robustness of the existing cosmic shear results, we present in this paper a unified analysis of four of the recent cosmic shear surveys: the Deep Lens Survey (DLS), the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), the Science Verification data from the Dark Energy Survey (DES-SV), and the 450 deg$^{2}$ release of the Kilo-Degree Survey (KiDS-450). By using a unified pipeline, we show how the cosmological constraints are sensitive to the various details of the pipeline. We identify several analysis choices that can shift the cosmological constraints by a significant fraction of the uncertainties. For our fiducial analysis choice, considering a Gaussian covariance, conservative scale cuts, assuming no baryonic feedback contamination, identical cosmological parameter priors and intrinsic alignment treatments, we find the constraints (mean, 16% and 84% confidence intervals) on the parameter $S_{8}\equiv \sigma_{8}(\Omega_{\rm m}/0.3)^{0.5}$ to be $S_{8}=0.94_{-0.045}^{+0.046}$ (DLS), $0.66_{-0.071}^{+0.070}$ (CFHTLenS), $0.84_{-0.061}^{+0.062}$ (DES-SV) and $0.76_{-0.049}^{+0.048}$ (KiDS-450). From the goodness-of-fit and the Bayesian evidence ratio, we determine that amongst the four surveys, the two more recent surveys, DES-SV and KiDS-450, have acceptable goodness-of-fit and are consistent with each other. The combined constraints are $S_{8}=0.79^{+0.042}_{-0.041}$, which is in good agreement with the first year of DES cosmic shear results and recent CMB constraints from the Planck satellite.

We measure the projected galaxy clustering and galaxy-galaxy lensing signals using the Galaxy And Mass Assembly (GAMA) survey and Kilo-Degree Survey (KiDS) to study galaxy bias. We use the concept of non-linear and stochastic galaxy biasing in the framework of halo occupation statistics to constrain the parameters of the halo occupation statistics and to unveil the origin of galaxy biasing. The bias function $\Gamma_{\text{gm}}(r_{\text{p}})$, where $r_{\text{p}}$ is the projected comoving separation, is evaluated using the analytical halo model from which the scale dependence of $\Gamma_{\text{gm}}(r_{\text{p}})$, and the origin of the non-linearity and stochasticity in halo occupation models can be inferred. Our observations unveil the physical reason for the non-linearity and stochasticity, further explored using hydrodynamical simulations, with the stochasticity mostly originating from the non-Poissonian behaviour of satellite galaxies in the dark matter haloes and their spatial distribution, which does not follow the spatial distribution of dark matter in the halo. The observed non-linearity is mostly due to the presence of the central galaxies, as was noted from previous theoretical work on the same topic. We also see that overall, more massive galaxies reveal a stronger scale dependence, and out to a larger radius. Our results show that a wealth of information about galaxy bias is hidden in halo occupation models. These models should therefore be used to determine the influence of galaxy bias in cosmological studies.

We present a machine-learning photometric redshift analysis of the Kilo-Degree Survey Data Release 3, using two neural-network based techniques: ANNz2 and MLPQNA. Despite limited coverage of spectroscopic training sets, these ML codes provide photo-zs of quality comparable to, if not better than, those from the BPZ code, at least up to zphot<0.9 and r<23.5. At the bright end of r<20, where very complete spectroscopic data overlapping with KiDS are available, the performance of the ML photo-zs clearly surpasses that of BPZ, currently the primary photo-z method for KiDS. Using the Galaxy And Mass Assembly (GAMA) spectroscopic survey as calibration, we furthermore study how photo-zs improve for bright sources when photometric parameters additional to magnitudes are included in the photo-z derivation, as well as when VIKING and WISE infrared bands are added. While the fiducial four-band ugri setup gives a photo-z bias $\delta z=-2e-4$ and scatter $\sigma_z<0.022$ at mean z = 0.23, combining magnitudes, colours, and galaxy sizes reduces the scatter by ~7% and the bias by an order of magnitude. Once the ugri and IR magnitudes are joined into 12-band photometry spanning up to 12 $\mu$, the scatter decreases by more than 10% over the fiducial case. Finally, using the 12 bands together with optical colours and linear sizes gives $\delta z<4e-5$ and $\sigma_z<0.019$. This paper also serves as a reference for two public photo-z catalogues accompanying KiDS DR3, both obtained using the ANNz2 code. The first one, of general purpose, includes all the 39 million KiDS sources with four-band ugri measurements in DR3. The second dataset, optimized for low-redshift studies such as galaxy-galaxy lensing, is limited to r<20, and provides photo-zs of much better quality than in the full-depth case thanks to incorporating optical magnitudes, colours, and sizes in the GAMA-calibrated photo-z derivation.

We present cosmological parameter constraints from a joint analysis of three cosmological probes: the tomographic cosmic shear signal in $\sim$450 deg$^2$ of data from the Kilo Degree Survey (KiDS), the galaxy-matter cross-correlation signal of galaxies from the Galaxies And Mass Assembly (GAMA) survey determined with KiDS weak lensing, and the angular correlation function of the same GAMA galaxies. We use fast power spectrum estimators that are based on simple integrals over the real-space correlation functions, and show that they are practically unbiased over relevant angular frequency ranges. We test our full pipeline on numerical simulations that are tailored to KiDS and retrieve the input cosmology. By fitting different combinations of power spectra, we demonstrate that the three probes are internally consistent. For all probes combined, we obtain $S_8\equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3}=0.800_{-0.027}^{+0.029}$, consistent with Planck and the fiducial KiDS-450 cosmic shear correlation function results. Marginalising over wide priors on the mean of the tomographic redshift distributions yields consistent results for $S_8$ with an increase of $28\%$ in the error. The combination of probes results in a $26\%$ reduction in uncertainties of $S_8$ over using the cosmic shear power spectra alone. The main gain from these additional probes comes through their constraining power on nuisance parameters, such as the galaxy intrinsic alignment amplitude or potential shifts in the redshift distributions, which are up to a factor of two better constrained compared to using cosmic shear alone, demonstrating the value of large-scale structure probe combination.

We compute the spherical-sky weak-lensing power spectrum of the shear and convergence. We discuss various approximations, such as flat-sky, and first- and second- order Limber equations for the projection. We find that the impact of adopting these approximations is negligible when constraining cosmological parameters from current weak lensing surveys. This is demonstrated using data from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). We find that the reported tension with Planck Cosmic Microwave Background (CMB) temperature anisotropy results cannot be alleviated. For future large-scale surveys with unprecedented precision, we show that the spherical second-order Limber approximation will provide sufficient accuracy. In this case, the cosmic-shear power spectrum is shown to be in agreement with the full projection at the sub-percent level for l > 3, with the corresponding errors an order of magnitude below cosmic variance for all l. When computing the two-point shear correlation function, we show that the flat-sky fast Hankel transformation results in errors below two percent compared to the full spherical transformation. In the spirit of reproducible research, our numerical implementation of all approximations and the full projection are publicly available within the package nicaea at http://www.cosmostat.org/software/nicaea.

We study the scaling relations between the baryonic content and total mass of groups of galaxies, as these systems provide a unique way to examine the role of non-gravitational processes in structure formation. Using Planck and ROSAT data, we conduct detailed comparisons of the stacked thermal Sunyaev-Zel'dovich (tSZ) effect and X-ray scaling relations of galaxy groups found in the the Galaxy And Mass Assembly (GAMA) survey and the BAHAMAS hydrodynamical simulation. We use weak gravitational lensing data from the Kilo Degree Survey (KiDS) to determine the average halo mass of the studied systems. We analyse the simulation in the same way, using realistic weak lensing, X-ray, and tSZ synthetic observations. Furthermore, to keep selection biases under control, we employ exactly the same galaxy selection and group identification procedures to the observations and simulation. Applying this careful comparison, we find that the simulations are in agreement with the observations, particularly with regards to the scaling relations of the lensing and tSZ results. This finding demonstrates that hydrodynamical simulation have reached the level of realism that is required to interpret observational survey data and study the baryon physics within dark matter haloes, where analytical modelling is challenging. Finally, using simulated data, we demonstrate that our observational processing of the X-ray and tSZ signals is free of significant biases. We find that our optical group selection procedure has, however, some room for improvement.

We derive the essentials of the skewed weak lensing likelihood via a simple Hierarchical Model. Our likelihood passes four objective and cosmology-independent tests which a standard Gaussian likelihood fails. We demonstrate that sound weak lensing analyses are naturally biased low, and this does not indicate any new physics such as deviations from $\Lambda$CDM. Mathematically, the biases arise because noisy two-point functions follow skewed distributions. This form of bias is already known from CMB analyses, where the low multipoles have asymmetric error bars. Weak lensing is more strongly affected by this asymmetry as galaxies form a discrete set of shear tracer particles, in contrast to a smooth shear field. We demonstrate that the biases can be up to 30 percent of the standard deviation per data point, dependent on the properties of the weak lensing survey. Our likelihood provides a versatile framework with which to address this bias in future weak lensing analyses.

We describe data release 3 (DR3) of the Galaxy And Mass Assembly (GAMA) survey. The GAMA survey is a spectroscopic redshift and multi-wavelength photometric survey in three equatorial regions each of 60.0 deg^2 (G09, G12, G15), and two southern regions of 55.7 deg^2 (G02) and 50.6 deg^2 (G23). DR3 consists of: the first release of data covering the G02 region and of data on H-ATLAS sources in the equatorial regions; and updates to data on sources released in DR2. DR3 includes 154809 sources with secure redshifts across four regions. A subset of the G02 region is 95.5% redshift complete to r<19.8 over an area of 19.5 deg^2, with 20086 galaxy redshifts, that overlaps substantially with the XXL survey (X-ray) and VIPERS (redshift survey). In the equatorial regions, the main survey has even higher completeness (98.5%), and spectra for about 75% of H-ATLAS filler targets were also obtained. This filler sample extends spectroscopic redshifts, for probable optical counterparts to H-ATLAS sub-mm sources, to 0.8 mag deeper (r<20.6) than the GAMA main survey. There are 25814 galaxy redshifts for H-ATLAS sources from the GAMA main or filler surveys. GAMA DR3 is available at the survey website (www.gama-survey.org/dr3/).

We perform a combined analysis of cosmic shear tomography, galaxy-galaxy lensing tomography, and redshift-space multipole power spectra (monopole and quadrupole) using 450 deg$^2$ of imaging data by the Kilo Degree Survey (KiDS) overlapping with two spectroscopic surveys: the 2-degree Field Lensing Survey (2dFLenS) and the Baryon Oscillation Spectroscopic Survey (BOSS). We restrict the galaxy-galaxy lensing and multipole power spectrum measurements to the overlapping regions with KiDS, and self-consistently compute the full covariance between the different observables using a large suite of $N$-body simulations. We methodically analyze different combinations of the observables, finding that galaxy-galaxy lensing measurements are particularly useful in improving the constraint on the intrinsic alignment amplitude (by 30%, positive at $3.5\sigma$ in the fiducial data analysis), while the multipole power spectra are useful in tightening the constraints along the lensing degeneracy direction (e.g. factor of two stronger matter density constraint in the fiducial analysis). The fully combined constraint on $S_8 \equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3} = 0.742 \pm 0.035$, which is an improvement by 20% compared to KiDS alone, corresponds to a $2.6\sigma$ discordance with Planck, and is not significantly affected by fitting to a more conservative set of scales. Given the tightening of the parameter space, we are unable to resolve the discordance with an extended cosmology that is simultaneously favored in a model selection sense, including the sum of neutrino masses, curvature, evolving dark energy, and modified gravity. The complementarity of our observables allows for constraints on modified gravity degrees of freedom that are not simultaneously bounded with either probe alone, and up to a factor of three improvement in the $S_8$ constraint in the extended cosmology compared to KiDS alone.

This paper is the first of a series of papers constraining cosmological parameters with weak lensing peak statistics using $\sim 450~\rm deg^2$ of imaging data from the Kilo Degree Survey (KiDS-450). We measure high signal-to-noise ratio (SNR: $\nu$) weak lensing convergence peaks in the range of $3<\nu<5$, and employ theoretical models to derive expected values. These models are validated using a suite of simulations. We take into account two major systematic effects, the boost factor and the effect of baryons on the mass-concentration relation of dark matter haloes. In addition, we investigate the impacts of other potential astrophysical systematics including the projection effects of large scale structures, intrinsic galaxy alignments, as well as residual measurement uncertainties in the shear and redshift calibration. Assuming a flat $\Lambda$CDM model, we find constraints for $S_{\rm 8}=\sigma_{\rm 8}(\Omega_{\rm m}/0.3)^{0.5}=0.746^{+0.046}_{-0.107}$ according to the degeneracy direction of the cosmic shear analysis and $\Sigma_{\rm 8}=\sigma_{\rm 8}(\Omega_{\rm m}/0.3)^{0.38}=0.696^{+0.048}_{-0.050}$ based on the derived degeneracy direction of our high-SNR peak statistics. The difference between the power index of $S_{\rm 8}$ and in $\Sigma_{\rm 8}$ indicates that combining the two probes has the potential to break the degeneracy in $\sigma_{\rm 8}$ and $\Omega_{\rm m}$. Our results are consistent with the cosmic shear tomographic correlation analysis of the same dataset and $\sim 2\sigma$ lower than the Planck 2016 results.

We study the statistics of peaks in a weak lensing reconstructed mass map of the first 450 square degrees of the Kilo Degree Survey. The map is computed with aperture masses directly applied to the shear field with an NFW-like compensated filter. We compare the peak statistics in the observations with that of simulations for various cosmologies to constrain the cosmological parameter $S_8 = \sigma_8 \sqrt{\Omega_{\rm m}/0.3}$, which probes the ($\Omega_{\rm m}, \sigma_8$) plane perpendicularly to its main degeneracy. We estimate $S_8=0.750\pm0.059$, using peaks in the signal-to-noise range $0 \leq {\rm S/N} \leq 4$, and accounting for various systematics, such as multiplicative shear bias, mean redshift bias, baryon feedback, intrinsic alignment, and shear-position coupling. These constraints are $\sim25\%$ tighter than the constraints from the high significance peaks alone ($3 \leq {\rm S/N} \leq 4$) which typically trace single-massive halos. This demonstrates the gain of information from low-S/N peaks. However we find that including ${\rm S/N} < 0$ peaks does not add further information. Our results are in good agreement with the tomographic shear two-point correlation function measurement in KiDS-450. Combining shear peaks with non-tomographic measurements of the shear two-point correlation functions yields a $\sim20\%$ improvement in the uncertainty on $S_8$ compared to the shear two-point correlation functions alone, highlighting the great potential of peaks as a cosmological probe.

In recent years, multiple studies have reported substantial populations of large, low-surface-brightness galaxies in local galaxy clusters. Various theories that aim to explain the presence of such ultra-diffuse galaxies (UDGs) have since been proposed. A key question that will help to differentiate between models is whether UDGs have counterparts in lower-mass host haloes, and what their abundance as a function of halo mass is. In this study we extend our previous study of UDGs in galaxy clusters to galaxy groups. We measure the abundance of UDGs in 325 spectroscopically-selected groups from the Galaxy And Mass Assembly (GAMA) survey. We make use of the overlapping imaging from the ESO Kilo-Degree Survey (KiDS), from which we can identify galaxies with mean surface brightnesses within their effective radii down to ~25.5 mag arcsec$^{-2}$ in the r-band. We are able to measure a significant overdensity of UDGs (with sizes r_eff > 1.5 kpc) in galaxy groups down to M200=10^12 Msun, a regime where approximately only 1 in 10 groups contains a UDG that we can detect. We combine measurements of the abundance of UDGs in haloes that cover three orders of magnitude in halo mass, finding that their numbers scale quite steeply with halo mass; N_UDG (R<R200) $\propto$ M200^(1.11+/-0.07). To better interpret this, we also measure the mass-richness relation for brighter galaxies down to $M^*_r$+2.5 in the same GAMA groups, and find a much shallower relation of N_Bright (R<R200) $\propto$ M200^(0.78+/-0.05). This shows that UDGs are relatively more abundant, compared to bright galaxies, in massive clusters than in groups. We discuss implications, but whether this difference is related to a higher destruction rate of UDGs in groups, or whether massive haloes have a positive effect on their formation, is not yet clear.

We present measurements of the weak gravitational lensing shear power spectrum based on $450$ sq. deg. of imaging data from the Kilo Degree Survey. We employ a quadratic estimator in two and three redshift bins and extract band powers of redshift auto-correlation and cross-correlation spectra in the multipole range $76 \leq \ell \leq 1310$. The cosmological interpretation of the measured shear power spectra is performed in a Bayesian framework assuming a $\Lambda$CDM model with spatially flat geometry, while accounting for small residual uncertainties in the shear calibration and redshift distributions as well as marginalising over intrinsic alignments, baryon feedback and an excess-noise power model. Moreover, massive neutrinos are included in the modelling. The cosmological main result is expressed in terms of the parameter combination $S_8 \equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3}$ yielding $S_8 = \ 0.651 \pm 0.058$ (3 z-bins), confirming the recently reported tension in this parameter with constraints from Planck at $3.2\sigma$ (3 z-bins). We cross-check the results of the 3 z-bin analysis with the weaker constraints from the 2 z-bin analysis and find them to be consistent. The high-level data products of this analysis, such as the band power measurements, covariance matrices, redshift distributions, and likelihood evaluation chains are available at http://kids.strw.leidenuniv.nl/

The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products, adding further area, homogenized photometric calibration, photometric redshifts and weak lensing shear measurements to the first two releases. A dedicated pipeline embedded in the Astro-WISE information system is used for the production of the main release. Modifications with respect to earlier releases are described in detail. Photometric redshifts have been derived using both Bayesian template fitting, and machine-learning techniques. For the weak lensing measurements, optimized procedures based on the THELI data reduction and lensfit shear measurement packages are used. In DR3 stacked ugri images, weight maps, masks, and source lists for 292 new survey tiles (~300 sq.deg) are made available. The multi-band catalogue, including homogenized photometry and photometric redshifts, covers the combined DR1, DR2 and DR3 footprint of 440 survey tiles (447 sq.deg). Limiting magnitudes are typically 24.3, 25.1, 24.9, 23.8 (5 sigma in a 2 arcsec aperture) in ugri, respectively, and the typical r-band PSF size is less than 0.7 arcsec. The photometric homogenization scheme ensures accurate colors and an absolute calibration stable to ~2% for gri and ~3% in u. Separately released are a weak lensing shear catalogue and photometric redshifts based on two different machine-learning techniques.

We test extensions to the standard cosmological model with weak gravitational lensing tomography using 450 deg$^2$ of imaging data from the Kilo Degree Survey (KiDS). In these extended cosmologies, which include massive neutrinos, nonzero curvature, evolving dark energy, modified gravity, and running of the scalar spectral index, we also examine the discordance between KiDS and cosmic microwave background measurements from Planck. The discordance between the two datasets is largely unaffected by a more conservative treatment of the lensing systematics and the removal of angular scales most sensitive to nonlinear physics. The only extended cosmology that simultaneously alleviates the discordance with Planck and is at least moderately favored by the data includes evolving dark energy with a time-dependent equation of state (in the form of the $w_0-w_a$ parameterization). In this model, the respective $S_8 = \sigma_8 \sqrt{\Omega_{\rm m}/0.3}$ constraints agree at the $1\sigma$ level, and there is `substantial concordance' between the KiDS and Planck datasets when accounting for the full parameter space. Moreover, the Planck constraint on the Hubble constant is wider than in LCDM and in agreement with the Riess et al. (2016) direct measurement of $H_0$. The dark energy model is moderately favored as compared to LCDM when combining the KiDS and Planck measurements, and remains moderately favored after including an informative prior on the Hubble constant. In both of these scenarios, marginalized constraints in the $w_0-w_a$ plane are discrepant with a cosmological constant at the $3\sigma$ level. Moreover, KiDS constrains the sum of neutrino masses to 4.0 eV (95% CL), finds no preference for time or scale dependent modifications to the metric potentials, and is consistent with flatness and no running of the spectral index. The analysis code is public at https://github.com/sjoudaki/kids450

We study the stellar-to-halo mass relation of central galaxies in the range 9.7<log_10(M_*/h^-2 M_sun)<11.7 and z<0.4, obtained from a combined analysis of the Kilo Degree Survey (KiDS) and the Galaxy And Mass Assembly (GAMA) survey. We use ~100 deg^2 of KiDS data to study the lensing signal around galaxies for which spectroscopic redshifts and stellar masses were determined by GAMA. We show that lensing alone results in poor constraints on the stellar-to-halo mass relation due to a degeneracy between the satellite fraction and the halo mass, which is lifted when we simultaneously fit the stellar mass function. At M_sun>5x10^10 h^-2 M_sun, the stellar mass increases with halo mass as ~M_h^0.25. The ratio of dark matter to stellar mass has a minimum at a halo mass of 8x10^11 h^-1 M_sun with a value of M_h/M_*=56_-10^+16 [h]. We also use the GAMA group catalogue to select centrals and satellites in groups with five or more members, which trace regions in space where the local matter density is higher than average, and determine for the first time the stellar-to-halo mass relation in these denser environments. We find no significant differences compared to the relation from the full sample, which suggests that the stellar-to-halo mass relation does not vary strongly with local density. Furthermore, we find that the stellar-to-halo mass relation of central galaxies can also be obtained by modelling the lensing signal and stellar mass function of satellite galaxies only, which shows that the assumptions to model the satellite contribution in the halo model do not significantly bias the stellar-to-halo mass relation. Finally, we show that the combination of weak lensing with the stellar mass function can be used to test the purity of group catalogues.

We constrain the average halo ellipticity of ~2 600 galaxy groups from the Galaxy And Mass Assembly (GAMA) survey, using the weak gravitational lensing signal measured from the overlapping Kilo Degree Survey (KiDS). To do so, we quantify the azimuthal dependence of the stacked lensing signal around seven different proxies for the orientation of the dark matter distribution, as it is a priori unknown which one traces the orientation best. On small scales, the major axis of the brightest group/cluster member (BCG) provides the best proxy, leading to a clear detection of an anisotropic signal. In order to relate that to a halo ellipticity, we have to adopt a model density profile. We derive new expressions for the quadrupole moments of the shear field given an elliptical model surface mass density profile. Modeling the signal with an elliptical Navarro-Frenk-White (NFW) profile on scales < 250 kpc, which roughly corresponds to half the virial radius, and assuming that the BCG is perfectly aligned with the dark matter, we find an average halo ellipticity of e_h=0.38 +/- 0.12. This agrees well with results from cold-dark-matter-only simulations, which typically report values of e_h ~ 0.3. On larger scales, the lensing signal around the BCGs does not trace the dark matter distribution well, and the distribution of group satellites provides a better proxy for the halo's orientation instead, leading to a 3--4 sigma detection of a non-zero halo ellipticity at scales between 250 kpc and 750 kpc. Our results suggest that the distribution of stars enclosed within a certain radius forms a good proxy for the orientation of the dark matter within that radius, which has also been observed in hydrodynamical simulations.

We investigate possible signatures of halo assembly bias for spectroscopically selected galaxy groups from the GAMA survey using weak lensing measurements from the spatially overlapping regions of the deeper, high-imaging-quality photometric KiDS survey. We use GAMA groups with an apparent richness larger than 4 to identify samples with comparable mean host halo masses but with a different radial distribution of satellite galaxies, which is a proxy for the formation time of the haloes. We measure the weak lensing signal for groups with a steeper than average and with a shallower than average satellite distribution and find no sign of halo assembly bias, with the bias ratio of $0.85^{+0.37}_{-0.25}$, which is consistent with the $\Lambda$CDM prediction. Our galaxy groups have typical masses of $10^{13} M_{\odot}/h$, naturally complementing previous studies of halo assembly bias on galaxy cluster scales.

We present the tomographic cross-correlation between galaxy lensing measured in the Kilo Degree Survey (KiDS-450) with overlapping lensing measurements of the cosmic microwave background (CMB), as detected by Planck 2015. We compare our joint probe measurement to the theoretical expectation for a flat $\Lambda$CDM cosmology, assuming the best-fitting cosmological parameters from the KiDS-450 cosmic shear and Planck CMB analyses. We find that our results are consistent within $1\sigma$ with the KiDS-450 cosmology, with an amplitude re-scaling parameter $A_{\rm KiDS} = 0.86 \pm 0.19$. Adopting a Planck cosmology, we find our results are consistent within $2\sigma$, with $A_{\it Planck} = 0.68 \pm 0.15$. We show that the agreement is improved in both cases when the contamination to the signal by intrinsic galaxy alignments is accounted for, increasing $A$ by $\sim 0.1$. This is the first tomographic analysis of the galaxy lensing -- CMB lensing cross-correlation signal, and is based on five photometric redshift bins. We use this measurement as an independent validation of the multiplicative shear calibration and of the calibrated source redshift distribution at high redshifts. We find that constraints on these two quantities are strongly correlated when obtained from this technique, which should therefore not be considered as a stand-alone competitive calibration tool.

We measure the cross-correlation between Fermi-LAT gamma-ray photons and over 1000 deg$^2$ of weak lensing data from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), the Red Cluster Sequence Lensing Survey (RCSLenS), and the Kilo Degree Survey (KiDS). We present the first measurement of tomographic weak lensing cross-correlations and the first application of spectral binning to cross-correlations between gamma rays and weak lensing. The measurements are performed using an angular power spectrum estimator while the covariance is estimated using an analytical prescription. We verify the accuracy of our covariance estimate by comparing it to two internal covariance estimators. Based on the non-detection of a cross-correlation signal, we derive constraints on weakly interacting massive particle (WIMP) dark matter. We compute exclusion limits on the dark matter annihilation cross-section $\langle\sigma_\rm{ann} v \rangle$, decay rate $\Gamma_\rm{dec}$, and particle mass $m_\rm{DM}$. We find that in the absence of a cross-correlation signal, tomography does not significantly improve the constraining power of the analysis. Assuming a strong contribution to the gamma-ray flux due to small-scale clustering of dark matter and accounting for known astrophysical sources of gamma rays, we exclude the thermal relic cross-section for particle masses of $m_\rm{DM}\lesssim 20$ GeV.

We determine the accuracy of galaxy redshift distributions as estimated from photometric redshift probability distributions $p(z)$. Our method utilises measurements of the angular cross-correlation between photometric galaxies and an overlapping sample of galaxies with spectroscopic redshifts. We describe the redshift leakage from a galaxy photometric redshift bin $j$ into a spectroscopic redshift bin $i$ using the sum of the $p(z)$ for the galaxies residing in bin $j$. We can then predict the angular cross-correlation between photometric and spectroscopic galaxies due to intrinsic galaxy clustering when $i \neq j$ as a function of the measured angular cross-correlation when $i=j$. We also account for enhanced clustering arising from lensing magnification using a halo model. The comparison of this prediction with the measured signal provides a consistency check on the validity of using the summed $p(z)$ to determine galaxy redshift distributions in cosmological analyses, as advocated by the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). We present an analysis of the photometric redshifts measured by CFHTLenS, which overlaps the Baryon Oscillation Spectroscopic Survey (BOSS). We also analyse the Red-sequence Cluster Lensing Survey (RCSLenS), which overlaps both BOSS and the WiggleZ Dark Energy Survey. We find that the summed $p(z)$ from both surveys are generally biased with respect to the true underlying distributions. If unaccounted for, this bias would lead to errors in cosmological parameter estimation from CFHTLenS by less than $\sim 4\%$. For photometric redshift bins which spatially overlap in 3-D with our spectroscopic sample, we determine redshift bias corrections which can be used in future cosmological analyses that rely on accurate galaxy redshift distributions.

We present a re-analysis of the CFHTLenS weak gravitational lensing survey using Complete Orthogonal Sets of E/B-mode Integrals, known as COSEBIs. COSEBIs provide a complete set of functions to efficiently separate E-modes from B-modes and hence allow for robust and stringent tests for systematic errors in the data. This analysis reveals significant B-modes on large angular scales that were not previously seen using the standard E/B decomposition analyses. We find that the significance of the B-modes is enhanced when the data is split by galaxy type and analysed in tomographic redshift bins. Adding tomographic bins to the analysis increases the number of COSEBIs modes, which results in a less accurate estimation of the covariance matrix from a set of simulations. We therefore also present the first compressed COSEBIs analysis of survey data, where the COSEBIs modes are optimally combined based on their sensitivity to cosmological parameters. In this tomographic CCOSEBIs analysis we find the B-modes to be consistent with zero when the full range of angular scales are considered.

Verlinde (2016) proposed that the observed excess gravity in galaxies and clusters is the consequence of Emergent Gravity (EG). In this theory the standard gravitational laws are modified on galactic and larger scales due to the displacement of dark energy by baryonic matter. EG gives an estimate of the excess gravity (described as an apparent dark matter density) in terms of the baryonic mass distribution and the Hubble parameter. In this work we present the first test of EG using weak gravitational lensing, within the regime of validity of the current model. Although there is no direct description of lensing and cosmology in EG yet, we can make a reasonable estimate of the expected lensing signal of low redshift galaxies by assuming a background LambdaCDM cosmology. We measure the (apparent) average surface mass density profiles of 33,613 isolated central galaxies, and compare them to those predicted by EG based on the galaxies' baryonic masses. To this end we employ the ~180 square degrees overlap of the Kilo-Degree Survey (KiDS) with the spectroscopic Galaxy And Mass Assembly (GAMA) survey. We find that the prediction from EG, despite requiring no free parameters, is in good agreement with the observed galaxy-galaxy lensing profiles in four different stellar mass bins. Although this performance is remarkable, this study is only a first step. Further advancements on both the theoretical framework and observational tests of EG are needed before it can be considered a fully developed and solidly tested theory.

We present predictions for the galaxy-galaxy lensing profile from the EAGLE hydrodynamical cosmological simulation at redshift z=0.18, in the spatial range 0.02 < R/(Mpc/h) < 2, and for five logarithmically equi-spaced stellar mass bins in the range 10.3 < $\log_{10}$(Mstar/ $M_{\odot}$) < 11.8. We compare these excess surface density profiles to the observed signal from background galaxies imaged by the Kilo Degree Survey around spectroscopically confirmed foreground galaxies from the GAMA survey. Exploiting the GAMA galaxy group catalogue, the profiles of central and satellite galaxies are computed separately for groups with at least five members to minimise contamination. EAGLE predictions are in broad agreement with the observed profiles for both central and satellite galaxies, although the signal is underestimated at R$\approx$0.5-2 Mpc/h for the highest stellar mass bins. When central and satellite galaxies are considered simultaneously, agreement is found only when the selection function of lens galaxies is taken into account in detail. Specifically, in the case of GAMA galaxies, it is crucial to account for the variation of the fraction of satellite galaxies in bins of stellar mass induced by the flux-limited nature of the survey. We report the inferred stellar-to-halo mass relation and we find good agreement with recent published results. We note how the precision of the galaxy-galaxy lensing profiles in the simulation holds the potential to constrain fine-grained aspects of the galaxy-dark matter connection.

We present a new training set for estimating empirical photometric redshifts of galaxies, which was created as part of the 2dFLenS project. This training set is located in a 700 sq deg area of the KiDS South field and is randomly selected and nearly complete at r<19.5. We investigate the photometric redshift performance obtained with ugriz photometry from VST-ATLAS and W1/W2 from WISE, based on several empirical and template methods. The best redshift errors are obtained with kernel-density estimation, as are the lowest biases, which are consistent with zero within statistical noise. The 68th percentiles of the redshift scatter for magnitude-limited samples at r<(15.5, 17.5, 19.5) are (0.014, 0.017, 0.028). In this magnitude range, there are no known ambiguities in the colour-redshift map, consistent with a small rate of redshift outliers. In the fainter regime, the KDE method produces p(z) estimates per galaxy that represent unbiased and accurate redshift frequency expectations. The p(z) sum over any subsample is consistent with the true redshift frequency plus Poisson noise. Further improvements in redshift precision at r<20 would mostly be expected from filter sets with narrower passbands to increase the sensitivity of colours to small changes in redshift.

We present high signal-to-noise galaxy-galaxy lensing measurements of the BOSS CMASS sample using 250 square degrees of weak lensing data from CFHTLenS and CS82. We compare this signal with predictions from mock catalogs trained to match observables including the stellar mass function and the projected and two dimensional clustering of CMASS. We show that the clustering of CMASS, together with standard models of the galaxy-halo connection, robustly predicts a lensing signal that is 20-40% larger than observed. Detailed tests show that our results are robust to a variety of systematic effects. Lowering the value of $S_{\rm 8}=\sigma_{\rm 8} \sqrt{\Omega_{\rm m}/0.3}$ compared to Planck2015 reconciles the lensing with clustering. However, given the scale of our measurement ($r<10$ $h^{-1}$ Mpc), other effects may also be at play and need to be taken into consideration. We explore the impact of baryon physics, assembly bias, massive neutrinos, and modifications to general relativity on $\Delta\Sigma$ and show that several of these effects may be non-negligible given the precision of our measurement. Disentangling cosmological effects from the details of the galaxy-halo connection, the effects of baryons, and massive neutrinos, is the next challenge facing joint lensing and clustering analyses. This is especially true in the context of large galaxy samples from Baryon Acoustic Oscillation surveys with precise measurements but complex selection functions.

We develop a statistical estimator to infer the redshift probability distribution of a photometric sample of galaxies from its angular cross-correlation in redshift bins with an overlapping spectroscopic sample. This estimator is a minimum variance weighted quadratic function of the data: a quadratic estimator. This extends and modifies the methodology presented by McQuinn & White (2013). The derived source redshift distribution is degenerate with the source galaxy bias, which must be constrained via additional assumptions. We apply this estimator to constrain source galaxy redshift distributions in the Kilo-Degree imaging survey through cross-correlation with the spectroscopic 2-degree Field Lensing Survey, presenting results first as a binned step-wise distribution in the range z < 0.8, and then building a continuous distribution using a Gaussian process model. We demonstrate the robustness of our methodology using mock catalogues constructed from N-body simulations, and comparisons with other techniques for inferring the redshift distribution.

We investigate the impact of astrophysical systematics on cosmic shear cosmological parameter constraints from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), and the concordance with cosmic microwave background measurements by Planck. We present updated CFHTLenS cosmic shear tomography measurements extended to degree scales using a covariance calibrated by a new suite of N-body simulations. We analyze these measurements with a new model fitting pipeline, accounting for key systematic uncertainties arising from intrinsic galaxy alignments, baryonic effects in the nonlinear matter power spectrum, and photometric redshift uncertainties. We examine the impact of the systematic degrees of freedom on the cosmological parameter constraints, both independently and jointly. When the systematic uncertainties are considered independently, the intrinsic alignment amplitude is the only degree of freedom that is substantially preferred by the data. When the systematic uncertainties are considered jointly, there is no consistently strong preference in favor of the more complex models. We quantify the level of concordance between the CFHTLenS and Planck datasets by employing two distinct data concordance tests, grounded in Bayesian evidence and information theory. We find that the two data concordance tests largely agree with one another, and that the level of concordance between the CFHTLenS and Planck datasets is sensitive to the exact details of the systematic uncertainties included in our analysis, ranging from decisive discordance to substantial concordance as the treatment of the systematic uncertainties becomes more conservative. The least conservative scenario is the one most favored by the cosmic shear data, but it is also the one that shows the greatest degree of discordance with Planck. The data and analysis code are public at https://github.com/sjoudaki/cfhtlens_revisited

We present cosmological parameter constraints from a tomographic weak gravitational lensing analysis of ~450deg$^2$ of imaging data from the Kilo Degree Survey (KiDS). For a flat $\Lambda$CDM cosmology with a prior on $H_0$ that encompasses the most recent direct measurements, we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.745\pm0.039$. This result is in good agreement with other low redshift probes of large scale structure, including recent cosmic shear results, along with pre-Planck cosmic microwave background constraints. A $2.3$-$\sigma$ tension in $S_8$ and `substantial discordance' in the full parameter space is found with respect to the Planck 2015 results. We use shear measurements for nearly 15 million galaxies, determined with a new improved `self-calibrating' version of $lens$fit validated using an extensive suite of image simulations. Four-band $ugri$ photometric redshifts are calibrated directly with deep spectroscopic surveys. The redshift calibration is confirmed using two independent techniques based on angular cross-correlations and the properties of the photometric redshift probability distributions. Our covariance matrix is determined using an analytical approach, verified numerically with large mock galaxy catalogues. We account for uncertainties in the modelling of intrinsic galaxy alignments and the impact of baryon feedback on the shape of the non-linear matter power spectrum, in addition to the small residual uncertainties in the shear and redshift calibration. The cosmology analysis was performed blind. Our high-level data products, including shear correlation functions, covariance matrices, redshift distributions, and Monte Carlo Markov Chains are available at http://kids.strw.leidenuniv.nl.

We present the Red-sequence Cluster Lensing Survey (RCSLenS), an application of the methods developed for the Canada France Hawaii Telescope Lensing Survey (CFHTLenS) to the ~785deg$^2$, multi-band imaging data of the Red-sequence Cluster Survey 2 (RCS2). This project represents the largest public, sub-arcsecond seeing, multi-band survey to date that is suited for weak gravitational lensing measurements. With a careful assessment of systematic errors in shape measurements and photometric redshifts we extend the use of this data set to allow cross-correlation analyses between weak lensing observables and other data sets. We describe the imaging data, the data reduction, masking, multi-colour photometry, photometric redshifts, shape measurements, tests for systematic errors, and a blinding scheme to allow for more objective measurements. In total we analyse 761 pointings with r-band coverage, which constitutes our lensing sample. Residual large-scale B-mode systematics prevent the use of this shear catalogue for cosmic shear science. The effective number density of lensing sources over an unmasked area of 571.7deg$^2$ and down to a magnitude limit of r~24.5 is 8.1 galaxies per arcmin$^2$ (weighted: 5.5 arcmin$^{-2}$) distributed over 14 patches on the sky. Photometric redshifts based on 4-band griz data are available for 513 pointings covering an unmasked area of 383.5 deg$^2$ We present weak lensing mass reconstructions of some example clusters as well as the full survey representing the largest areas that have been mapped in this way. All our data products are publicly available through CADC at http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/en/community/rcslens/query.html in a format very similar to the CFHTLenS data release.

We present measurements of the spatial mapping between (hot) baryons and the total matter in the Universe, via the cross-correlation between the thermal Sunyaev-Zeldovich (tSZ) map from Planck and the weak gravitational lensing maps from the Red Sequence Cluster Survey (RCSLenS). The cross-correlations are performed on the map level where all the sources (including diffuse intergalactic gas) contribute to the signal. We consider two configuration-space correlation function estimators, $\xi^{ y-\kappa}$ and $\xi^ {y-\gamma_{t}}$, and a Fourier space estimator, $C_{\ell}^{y-\kappa}$, in our analysis. We detect a significant correlation out to three degrees of angular separation on the sky. Based on statistical noise only, we can report 13$\sigma$ and 17$\sigma$ detections of the cross-correlation using the configuration-space $y-\kappa$ and $y-\gamma_{t}$ estimators, respectively. Including a heuristic estimate of the sampling variance yields a detection significance of 6$\sigma$ and 8$\sigma$, respectively. A similar level of detection is obtained from the Fourier-space estimator, $C_{\ell}^{y-\kappa}$. As each estimator probes different dynamical ranges, their combination improves the significance of the detection. We compare our measurements with predictions from the cosmo-OWLS suite of cosmological hydrodynamical simulations, where different galactic feedback models are implemented. We find that a model with considerable AGN feedback that removes large quantities of hot gas from galaxy groups and WMAP-7yr best-fit cosmological parameters provides the best match to the measurements. All baryonic models in the context of a Planck cosmology over-predict the observed signal. Similar cosmological conclusions are drawn when we employ a halo model with the observed `universal' pressure profile.

We study the spherical evolution model for voids in $\Lambda$CDM, where the evolution of voids is governed by dark energy at an earlier time than that for the whole universe or in overdensities. We show that the presence of dark energy suppresses the growth of peculiar velocities, causing void shell-crossing to occur at progressively later epochs as $\Omega_{\Lambda}$ increases. We apply the spherical model to evolve the initial conditions of N-body simulated voids and compare the resulting final void profiles. We find that the model is successful in tracking the evolution of voids with radii greater than $30 h^{-1} \rm Mpc$, implying that void profiles could be used to constrain dark energy. We find that the initial peculiar velocities of voids play a significant role in shaping their evolution. Excluding the peculiar velocity in the evolution model delays the time of shell crossing.

We present the 2-degree Field Lensing Survey (2dFLenS), a new galaxy redshift survey performed at the Anglo-Australian Telescope. 2dFLenS is the first wide-area spectroscopic survey specifically targeting the area mapped by deep-imaging gravitational lensing fields, in this case the Kilo-Degree Survey. 2dFLenS obtained 70,079 redshifts in the range z < 0.9 over an area of 731 sq deg, and is designed to extend the datasets available for testing gravitational physics and promote the development of relevant algorithms for joint imaging and spectroscopic analysis. The redshift sample consists first of 40,531 Luminous Red Galaxies (LRGs), which enable analyses of galaxy-galaxy lensing, redshift-space distortion, and the overlapping source redshift distribution by cross-correlation. An additional 28,269 redshifts form a magnitude-limited (r < 19.5) nearly-complete sub-sample, allowing direct source classification and photometric-redshift calibration. In this paper, we describe the motivation, target selection, spectroscopic observations, and clustering analysis of 2dFLenS. We use power spectrum multipole measurements to fit the redshift-space distortion parameter of the LRG sample in two redshift ranges 0.15 < z < 0.43 and 0.43 < z < 0.7 as beta = 0.49 +/- 0.15 and beta = 0.26 +/- 0.09, respectively. These values are consistent with those obtained from LRGs in the Baryon Oscillation Spectroscopic Survey. 2dFLenS data products will be released via our website http://2dflens.swin.edu.au.

Euclid is a European Space Agency medium class mission selected for launch in 2020 within the Cosmic Vision 2015 2025 program. The main goal of Euclid is to understand the origin of the accelerated expansion of the universe. Euclid will explore the expansion history of the universe and the evolution of cosmic structures by measuring shapes and redshifts of galaxies as well as the distribution of clusters of galaxies over a large fraction of the sky. Although the main driver for Euclid is the nature of dark energy, Euclid science covers a vast range of topics, from cosmology to galaxy evolution to planetary research. In this review we focus on cosmology and fundamental physics, with a strong emphasis on science beyond the current standard models. We discuss five broad topics: dark energy and modified gravity, dark matter, initial conditions, basic assumptions and questions of methodology in the data analysis. This review has been planned and carried out within Euclid's Theory Working Group and is meant to provide a guide to the scientific themes that will underlie the activity of the group during the preparation of the Euclid mission.

We present an optimized variant of the halo model, designed to produce accurate matter power spectra well into the non-linear regime for a wide range of cosmological models. To do this, we introduce physically motivated free parameters into the halo-model formalism and fit these to data from high-resolution $N$-body simulations. For a variety of $\Lambda$CDM and $w$CDM models the halo-model power is accurate to $\simeq 5$ per cent for $k\leq 10h\,\mathrm{Mpc}$ and $z\leq 2$. An advantage of our new halo model is that it can be adapted to account for the effects of baryonic feedback on the power spectrum. We demonstrate this by fitting the halo model to power spectra from the OWLS hydrodynamical simulation suite via parameters that govern halo internal structure. We are able to fit all feedback models investigated at the 5 per cent level using only two free parameters, and we place limits on the range of these halo parameters for feedback models investigated by the OWLS simulations. Accurate predictions to high-$k$ are vital for weak lensing surveys, and these halo parameters could be considered nuisance parameters to marginalize over in future analyses to mitigate uncertainty regarding the details of feedback. Finally, we investigate how lensing observables predicted by our model compare to those from simulations and from HALOFIT for a range of $k$-cuts and feedback models and quantify the angular scales at which these effects become important. Code to calculate power spectra from the model presented in this paper can be found at https://github.com/alexander-mead/hmcode

We measure the cross-correlation signature between the Planck CMB lensing map and the weak lensing observations from both the Red-sequence Cluster Lensing Survey (RCSLenS) and the Canada-France-Hawai Telescope Lensing Survey (CFHTLenS). In addition to a Fourier analysis, we include the first configuration-space detection, based on the estimators $\langle \kappa_{\rm CMB} \kappa_{\rm gal} \rangle$ and $\langle \kappa_{\rm CMB} \gamma_{t} \rangle$. Combining 747.2 deg$^2$ from both surveys, we find a detection significance that exceeds $4.2\sigma$ in both Fourier- and configuration-space analyses. Scaling the predictions by a free parameter $A$, we obtain $A^{\rm Planck}_{\rm CFHT}= 0.68\pm 0.31 $ and $A^{\rm Planck}_{\rm RCS}= 1.31\pm 0.33$. In preparation for the next generation of measurements similar to these, we quantify the impact of different analysis choices on these results. First, since none of these estimators probes the exact same dynamical range, we improve our detection by combining them. Second, we carry out a detailed investigation on the effect of apodization, zero-padding and mask multiplication, validated on a suite of high-resolution simulations, and find that the latter produces the largest systematic bias in the cosmological interpretation. Finally, we show that residual contamination from intrinsic alignment and the effect of photometric redshift error are both largely degenerate with the characteristic signal from massive neutrinos, however the signature of baryon feedback might be easier to distinguish. The three lensing datasets are now publicly available.

Galaxies and their dark matter haloes are part of a complex network of mass structures, collectively called the cosmic web. Using the tidal tensor prescription these structures can be classified into four cosmic environments: voids, sheets, filaments and knots. As the cosmic web may influence the formation and evolution of dark matter haloes and the galaxies they host, we aim to study the effect of these cosmic environments on the average mass of galactic haloes. To this end we measure the galaxy-galaxy lensing profile of 91,195 galaxies, within 0.039 < z < 0.263, from the spectroscopic Galaxy And Mass Assembly (GAMA) survey, using ~100 square degrees of overlapping data from the Kilo-Degree Survey (KiDS). In each of the four cosmic environments we model the contributions from group centrals, satellites and neighbouring groups to the stacked galaxy-galaxy lensing profiles. After correcting the lens samples for differences in the stellar mass distribution, we find no dependence of the average halo mass of central galaxies on their cosmic environment. We do find a significant increase in the average contribution of neighbouring groups to the lensing profile in increasingly dense cosmic environments. We show, however, that the observed effect can be entirely attributed to the galaxy density at much smaller scales (within 4 Mpc/h), which is correlated with the density of the cosmic environments. Within our current uncertainties we find no direct dependence of galaxy halo mass on their cosmic environment.

We present an accurate non-linear matter power spectrum prediction scheme for a variety of extensions to the standard cosmological paradigm, which uses the tuned halo model previously developed in Mead (2015b). We consider dark energy models that are both minimally and non-minimally coupled, massive neutrinos and modified gravitational forces with chameleon and Vainshtein screening mechanisms. In all cases we compare halo-model power spectra to measurements from high-resolution simulations. We show that the tuned halo model method can predict the non-linear matter power spectrum measured from simulations of parameterised $w(a)$ dark energy models at the few per cent level for $k<10\,h\mathrm{Mpc}^{-1}$, and we present theoretically motivated extensions to cover non-minimally coupled scalar fields, massive neutrinos and Vainshtein screened modified gravity models that result in few per cent accurate power spectra for $k<10\,h\mathrm{Mpc}^{-1}$. For chameleon screened models we achieve only 10 per cent accuracy for the same range of scales. Finally, we use our halo model to investigate degeneracies between different extensions to the standard cosmological model, finding that the impact of baryonic feedback on the non-linear matter power spectrum can be considered independently of modified gravity or massive neutrino extensions. In contrast, considering the impact of modified gravity and massive neutrinos independently results in biased estimates of power at the level of 5 per cent at scales $k>0.5\,h\mathrm{Mpc}^{-1}$. An updated version of our publicly available HMcode can be found at https://github.com/alexander-mead/HMcode

We present a complete derivation of the observationally motivated definition of the modified gravity statistic $E_G$. Using this expression, we investigate how variations to theory and survey parameters may introduce uncertainty in the general relativistic prediction of $E_G$. We forecast errors on $E_G$ for measurements using two combinations of upcoming surveys, and find that theoretical uncertainties may dominate for a futuristic measurement. Finally, we compute predictions of $E_G$ under modifications to general relativity in the quasistatic regime, and comment on the pros and cons of using $E_G$ to test gravity with future surveys.

We present the first measurement of individual cluster mass estimates using weak lensing size and flux magnification. Using data from the HST-STAGES survey of the A901/902 supercluster we detect the four known groups in the supercluster at high significance using magnification alone. We discuss the application of a fully Bayesian inference analysis, and investigate a broad range of potential systematics in the application of the method. We compare our results to a previous weak lensing shear analysis of the same field finding the recovered signal-to-noise of our magnification-only analysis to range from 45% to 110% of the signal-to-noise in the shear-only analysis. On a case-by-case basis we find consistent magnification and shear constraints on cluster virial radius, and finding that for the full sample, magnification constraints to be a factor $0.77 \pm 0.18$ lower than the shear measurements.

We present the first cosmological measurement derived from a galaxy density field subject to a `clipping' transformation. By enforcing an upper bound on the galaxy number density field in the Galaxy and Mass Assembly survey (GAMA), contributions from the nonlinear processes of virialisation and galaxy bias are greatly reduced. This leads to a galaxy power spectrum which is easier to model, without calibration from numerical simulations. We develop a theoretical model for the power spectrum of a clipped field in redshift space, which is exact for the case of anisotropic Gaussian fields. Clipping is found to extend the applicability of the conventional Kaiser prescription by more than a factor of three in wavenumber, or a factor of thirty in terms of the number of Fourier modes. By modelling the galaxy power spectrum on scales k < 0.3 h/Mpc and density fluctuations $\delta_g < 4$ we measure the normalised growth rate $f\sigma_8(z = 0.18) = 0.29 \pm 0.10$.

In this paper we present results of applying the shear-ratio method to the RCSLenS data. The method takes the ratio of the mean of the weak lensing tangential shear signal about galaxy clusters, averaged over all clusters of the same redshift, in multiple background redshift bins. In taking a ratio the mass-dependency of the shear signal is cancelled-out leaving a statistic that is dependent on the geometric part of the lensing kernel only. We apply this method to 535 clusters and measure a cosmology-independent distance-redshift relation to redshifts z~1. In combination with Planck data the method lifts the degeneracies in the CMB measurements, resulting in cosmological parameter constraints of OmegaM=0.31 +/- 0.10 and w0 = -1.02 +/- 0.37, for a flat wCDM cosmology.

We present measurements of the galaxy bias $b$ and the galaxy-matter cross-correlation coefficient $r$ for the BOSS LOWZ luminous red galaxy sample. Using a new statistical weak lensing analysis of the Red Sequence Cluster Lensing Survey (RCSLenS) we find the bias properties of this sample to be higher than previously reported with $b=2.45^{+0.05}_{-0.05}$ and $r=1.64^{+0.17}_{-0.16}$ on scales between $3'$ and $20'$. We repeat the measurement for angular scales of $20'\leq \vartheta \leq70'$, which yields $b=2.39^{+0.07}_{-0.07}$ and $r=1.24^{+0.26}_{-0.25}$. This is the first application of a data compression analysis using a complete set of discrete estimators for galaxy-galaxy lensing and galaxy clustering. As cosmological data sets grow, our new method of data compression will become increasingly important in order to interpret joint weak lensing and galaxy clustering measurements and to estimate the data covariance. In future studies this formalism can be used as a tool to study the large-scale structure of the Universe to yield a precise determination of cosmological parameters.

The unknown nature of dark energy motivates continued cosmological tests of large-scale gravitational physics. We present a new consistency check based on the relative amplitude of non-relativistic galaxy peculiar motions, measured via redshift-space distortion, and the relativistic deflection of light by those same galaxies traced by galaxy-galaxy lensing. We take advantage of the latest generation of deep, overlapping imaging and spectroscopic datasets, combining the Red Cluster Sequence Lensing Survey (RCSLenS), the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), the WiggleZ Dark Energy Survey and the Baryon Oscillation Spectroscopic Survey (BOSS). We quantify the results using the "gravitational slip" statistic E_G, which we estimate as 0.48 +/- 0.10 at z=0.32 and 0.30 +/- 0.07 at z=0.57, the latter constituting the highest redshift at which this quantity has been determined. These measurements are consistent with the predictions of General Relativity, for a perturbed Friedmann-Robertson-Walker metric in a Universe dominated by a cosmological constant, which are E_G = 0.41 and 0.36 at these respective redshifts. The combination of redshift-space distortion and gravitational lensing data from current and future galaxy surveys will offer increasingly stringent tests of fundamental cosmology.

Applying a transformation to a non-Gaussian field can enhance the information content of the resulting power spectrum, by reducing the correlations between Fourier modes. In the context of weak gravitational lensing, it has been shown that this gain in information content is significantly compromised by the presence of shape noise. We apply clipping to mock convergence fields, a technique which is known to be robust in the presence of noise and has been successfully applied to galaxy number density fields. When analysed in isolation the resulting convergence power spectrum returns degraded constraints on cosmological parameters. However substantial gains can be achieved by performing a combined analysis of the power spectra derived from both the original and transformed fields. Even in the presence of realistic levels of shape noise, we demonstrate that this approach is capable of reducing the area of likelihood contours within the $\Omega_m - \sigma_8$ plane by more than a factor of three.

Euclid is a European Space Agency medium class mission selected for launch in 2019 within the Cosmic Vision 2015-2025 programme. The main goal of Euclid is to understand the origin of the accelerated expansion of the Universe. Euclid will explore the expansion history of the Universe and the evolution of cosmic structures by measuring shapes and redshifts of galaxies as well as the distribution of clusters of galaxies over a large fraction of the sky. Although the main driver for Euclid is the nature of dark energy, Euclid science covers a vast range of topics, from cosmology to galaxy evolution to planetary research. In this review we focus on cosmology and fundamental physics, with a strong emphasis on science beyond the current standard models. We discuss five broad topics: dark energy and modified gravity, dark matter, initial conditions, basic assumptions and questions of methodology in the data analysis. This review has been planned and carried out within Euclid's Theory Working Group and is meant to provide a guide to the scientific themes that will underlie the activity of the group during the preparation of the Euclid mission.

We present weak lensing constraints on the ellipticity of galaxy-scale matter haloes and the galaxy-halo misalignment. Using data from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), we measure the weighted-average ratio of the aligned projected ellipticity components of galaxy matter haloes and their embedded galaxies, $f_\mathrm{h}$, split by galaxy type. We then compare our observations to measurements taken from the Millennium Simulation, assuming different models of galaxy-halo misalignment. Using the Millennium Simulation we verify that the statistical estimator used removes contamination from cosmic shear. We also detect an additional signal in the simulation, which we interpret as the impact of intrinsic shape-shear alignments between the lenses and their large-scale structure environment. These alignments are likely to have caused some of the previous observational constraints on $f_\mathrm{h}$ to be biased high. From CFHTLenS we find $f_\mathrm{h}=-0.04 \pm 0.25$ for early-type galaxies, which is consistent with current models for the galaxy-halo misalignment predicting $f_\mathrm{h}\simeq 0.20$. For late-type galaxies we measure $f_\mathrm{h}=0.69_{-0.36}^{+0.37}$ from CFHTLenS. This can be compared to the simulated results which yield $f_\mathrm{h}\simeq 0.02$ for misaligned late-type models.

Dark energy may be the first sign of new fundamental physics in the Universe, taking either a physical form or revealing a correction to Einsteinian gravity. Weak gravitational lensing and galaxy peculiar velocities provide complementary probes of General Relativity, and in combination allow us to test modified theories of gravity in a unique way. We perform such an analysis by combining measurements of cosmic shear tomography from the Canada-France Hawaii Telescope Lensing Survey (CFHTLenS) with the growth of structure from the WiggleZ Dark Energy Survey and the Six-degree-Field Galaxy Survey (6dFGS), producing the strongest existing joint constraints on the metric potentials that describe general theories of gravity. For scale-independent modifications to the metric potentials which evolve linearly with the effective dark energy density, we find present-day cosmological deviations in the Newtonian potential and curvature potential from the prediction of General Relativity to be (Delta Psi)/Psi = 0.05 \pm 0.25 and (Delta Phi)/Phi = -0.05 \pm 0.3 respectively (68 per cent CL).

The Kilo-Degree Survey (KiDS) is a multi-band imaging survey designed for cosmological studies from weak lensing and photometric redshifts. It uses the ESO VLT Survey Telescope with its wide-field camera OmegaCAM. KiDS images are taken in four filters similar to the SDSS ugri bands. The best-seeing time is reserved for deep r-band observations that reach a median 5-sigma limiting AB magnitude of 24.9 with a median seeing that is better than 0.7arcsec. Initial KiDS observations have concentrated on the GAMA regions near the celestial equator, where extensive, highly complete redshift catalogues are available. A total of 109 survey tiles, one square degree each, form the basis of the first set of lensing analyses, which focus on measurements of halo properties of GAMA galaxies. 9 galaxies per square arcminute enter the lensing analysis, for an effective inverse shear variance of 69 per square arcminute. Accounting for the shape measurement weight, the median redshift of the sources is 0.53. KiDS data processing follows two parallel tracks, one optimized for galaxy shape measurement (for weak lensing), and one for accurate matched-aperture photometry in four bands (for photometric redshifts). This technical paper describes how the lensing and photometric redshift catalogues have been produced (including an extensive description of the Gaussian Aperture and Photometry pipeline), summarizes the data quality, and presents extensive tests for systematic errors that might affect the lensing analyses. We also provide first demonstrations of the suitability of the data for cosmological measurements, and explain how the shear catalogues were blinded to prevent confirmation bias in the scientific analyses. The KiDS shear and photometric redshift catalogues, presented in this paper, are released to the community through http://kids.strw.leidenuniv.nl .

We use the first 100 sq. deg. of overlap between the Kilo-Degree Survey (KiDS) and the Galaxy And Mass Assembly (GAMA) survey to determine the galaxy halo mass of ~10,000 spectroscopically-confirmed satellite galaxies in massive ($M > 10^{13}h^{-1}{\rm M}_\odot$) galaxy groups. Separating the sample as a function of projected distance to the group centre, we jointly model the satellites and their host groups with Navarro-Frenk-White (NFW) density profiles, fully accounting for the data covariance. The probed satellite galaxies in these groups have total masses $\log M_{\rm sub} /(h^{-1}{\rm M}_\odot) \approx 11.7 - 12.2$ consistent across group-centric distance within the errorbars. Given their typical stellar masses, $\log M_{\rm \star,sat}/(h^{-2}{\rm M}_\odot) \sim 10.5$, such total masses imply stellar mass fractions of $M_{\rm \star,sat} /M_{\rm sub} \approx 0.04 h^{-1}$ . The average subhalo hosting these satellite galaxies has a mass $M_{\rm sub} \sim 0.015M_{\rm host}$ independent of host halo mass, in broad agreement with the expectations of structure formation in a $\Lambda$CDM universe.

The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the survey is mapping the large-scale matter distribution in the Universe back to a redshift of ~0.5. Secondary science cases are manifold, covering topics such as galaxy evolution, Milky Way structure, and the detection of high-redshift clusters and quasars. KiDS is an ESO Public Survey and dedicated to serving the astronomical community with high-quality data products derived from the survey data, as well as with calibration data. Public data releases will be made on a yearly basis, the first two of which are presented here. For a total of 148 survey tiles (~160 sq.deg.) astrometrically and photometrically calibrated, coadded ugri images have been released, accompanied by weight maps, masks, source lists, and a multi-band source catalog. A dedicated pipeline and data management system based on the Astro-WISE software system, combined with newly developed masking and source classification software, is used for the data production of the data products described here. The achieved data quality and early science projects based on the data products in the first two data releases are reviewed in order to validate the survey data. Early scientific results include the detection of nine high-z QSOs, fifteen candidate strong gravitational lenses, high-quality photometric redshifts and galaxy structural parameters for hundreds of thousands of galaxies. (Abridged)

The Kilo-Degree Survey (KiDS) is an optical wide-field survey designed to map the matter distribution in the Universe using weak gravitational lensing. In this paper, we use these data to measure the density profiles and masses of a sample of $\sim \mathrm{1400}$ spectroscopically identified galaxy groups and clusters from the Galaxy And Mass Assembly (GAMA) survey. We detect a highly significant signal (signal-to-noise-ratio $\sim$ 120), allowing us to study the properties of dark matter haloes over one and a half order of magnitude in mass, from $M \sim 10^{13}-10^{14.5} h^{-1}\mathrm{M_{\odot}}$. We interpret the results for various subsamples of groups using a halo model framework which accounts for the mis-centring of the Brightest Cluster Galaxy (used as the tracer of the group centre) with respect to the centre of the group's dark matter halo. We find that the density profiles of the haloes are well described by an NFW profile with concentrations that agree with predictions from numerical simulations. In addition, we constrain scaling relations between the mass and a number of observable group properties. We find that the mass scales with the total r-band luminosity as a power-law with slope $1.16 \pm 0.13$ (1-sigma) and with the group velocity dispersion as a power-law with slope $1.89 \pm 0.27$ (1-sigma). Finally, we demonstrate the potential of weak lensing studies of groups to discriminate between models of baryonic feedback at group scales by comparing our results with the predictions from the Cosmo-OverWhelmingly Large Simulations (Cosmo-OWLS) project, ruling out models without AGN feedback.

Ultra-deep observations of ECDF-S with Chandra and XMM-Newton enable a search for extended X-ray emission down to an unprecedented flux of $2\times10^{-16}$ ergs s$^{-1}$ cm$^{-2}$. We present the search for the extended emission on spatial scales of 32$^{\prime\prime}$ in both Chandra and XMM data, covering 0.3 square degrees and model the extended emission on scales of arcminutes. We present a catalog of 46 spectroscopically identified groups, reaching a redshift of 1.6. We show that the statistical properties of ECDF-S, such as logN-logS and X-ray luminosity function are broadly consistent with LCDM, with the exception that dn/dz/d$\Omega$ test reveals that a redshift range of $0.2<z<0.5$ in ECDF-S is sparsely populated. The lack of nearby structure, however, makes studies of high-redshift groups particularly easier both in X-rays and lensing, due to a lower level of clustered foreground. We present one and two point statistics of the galaxy groups as well as weak-lensing analysis to show that the detected low-luminosity systems are indeed low-mass systems. We verify the applicability of the scaling relations between the X-ray luminosity and the total mass of the group, derived for the COSMOS survey to lower masses and higher redshifts probed by ECDF-S by means of stacked weak lensing and clustering analysis, constraining any possible departures to be within 30% in mass. Abridged.

We present weak lensing and X-ray analysis of 12 low mass clusters from the CFHTLenS and XMM-CFHTLS surveys. We combine these systems with high-mass systems from CCCP and low-mass systems from COSMOS to obtain a sample of 70 systems, spanning over two orders of magnitude in mass. We measure core-excised Lx-Tx, M-Lx and M-Tx scaling relations and include corrections for observational biases. By providing fully bias corrected relations, we give the current limitations for Lx and Tx as cluster mass proxies. We demonstrate that Tx benefits from a significantly lower intrinsic scatter at fixed mass than Lx. By studying the residuals of the bias corrected relations, we show for the first time using weak lensing masses that galaxy groups seem more luminous and warmer for their mass than clusters. This implies a steepening of the M-Lx and M-Tx relations at low masses. We verify the inferred steepening using a different high mass sample from the literature and show that variance between samples is the dominant effect leading to discrepant scaling relations. We divide our sample into subsamples of merging and relaxed systems, and find that mergers may have enhanced scatter in lensing measurements, most likely due to stronger triaxiality and more substructure. For the Lx-Tx relation, which is unaffected by lensing measurements, we find the opposite trend in scatter. We also explore the effects of X-ray cross-calibration and find that Chandra calibration leads to flatter Lx-Tx and M-Tx relations than XMM-Newton.

We study the correlations of the shear signal between triplets of sources in the Canada-France-Hawaii Lensing Survey (CFHTLenS) to probe cosmological parameters via the matter bispectrum. In contrast to previous studies, we adopted a non-Gaussian model of the data likelihood which is supported by our simulations of the survey. We find that for state-of-the-art surveys, similar to CFHTLenS, a Gaussian likelihood analysis is a reasonable approximation, albeit small differences in the parameter constraints are already visible. For future surveys we expect that a Gaussian model becomes inaccurate. Our algorithm for a refined non-Gaussian analysis and data compression is then of great utility especially because it is not much more elaborate if simulated data are available. Applying this algorithm to the third-order correlations of shear alone in a blind analysis, we find a good agreement with the standard cosmological model: $\Sigma_8$=$\sigma_8$ $(\Omega_{\rm m}/0.27)^{0.64}$=$0.79^{+0.08}_{-0.11}$ for a flat $\Lambda\rm CDM$ cosmology with $h=0.7\pm0.04$ ($68\%$ credible interval). Nevertheless our models provide only moderately good fits as indicated by $\chi^2/{\rm dof}=2.9$, including a $20\%$ r.m.s. uncertainty in the predicted signal amplitude. The models cannot explain a signal drop on scales around 15 arcmin, which may be caused by systematics. It is unclear whether the discrepancy can be fully explained by residual PSF systematics of which we find evidence at least on scales of a few arcmin. Therefore we need a better understanding of higher-order correlations of cosmic shear and their systematics to confidently apply them as cosmological probes.

We present new constraints on the relationship between galaxies and their host dark matter halos, measured from the location of the peak of the stellar-to-halo mass ratio (SHMR), up to the most massive galaxy clusters at redshift $z\sim0.8$ and over a volume of nearly 0.1~Gpc$^3$. We use a unique combination of deep observations in the CFHTLenS/VIPERS field from the near-UV to the near-IR, supplemented by $\sim60\,000$ secure spectroscopic redshifts, analysing galaxy clustering, galaxy-galaxy lensing and the stellar mass function. We interpret our measurements within the halo occupation distribution (HOD) framework, separating the contributions from central and satellite galaxies. We find that the SHMR for the central galaxies peaks at $M_{\rm h, peak} = 1.9^{+0.2}_{-0.1}\times10^{12} M_{\odot}$ with an amplitude of $0.025$, which decreases to $\sim0.001$ for massive halos ($M_{\rm h} > 10^{14} M_{\odot}$). Compared to central galaxies only, the total SHMR (including satellites) is boosted by a factor 10 in the high-mass regime (cluster-size halos), a result consistent with cluster analyses from the literature based on fully independent methods. After properly accounting for differences in modelling, we have compared our results with a large number of results from the literature up to $z=1$: we find good general agreement, independently of the method used, within the typical stellar-mass systematic errors at low to intermediate mass (${M}_{\star} < 10^{11} M_{\odot}$) and the statistical errors above. We have also compared our SHMR results to semi-analytic simulations and found that the SHMR is tilted compared to our measurements in such a way that they over- (under-) predict star formation efficiency in central (satellite) galaxies.

This paper presents the first application of 3D cosmic shear to a wide-field weak lensing survey. 3D cosmic shear is a technique that analyses weak lensing in three dimensions using a spherical harmonic approach, and does not bin data in the redshift direction. This is applied to CFHTLenS, a 154 square degree imaging survey with a median redshift of 0.7 and an effective number density of 11 galaxies per square arcminute usable for weak lensing. To account for survey masks we apply a 3D pseudo-Cl approach on weak lensing data, and to avoid uncertainties in the highly non-linear regime, we separately analyse radial wave numbers k<=1.5h/Mpc and k<=5.0h/Mpc, and angular wavenumbers l~400-5000. We show how one can recover 2D and tomographic power spectra from the full 3D cosmic shear power spectra and present a measurement of the 2D cosmic shear power spectrum, and measurements of a set of 2-bin and 6-bin cosmic shear tomographic power spectra; in doing so we find that using the 3D power in the calculation of such 2D and tomographic power spectra from data naturally accounts for a minimum scale in the matter power spectrum. We use 3D cosmic shear to constrain cosmologies with parameters OmegaM, OmegaB, sigma8, h, ns, w0, wa. For a non-evolving dark energy equation of state, and assuming a flat cosmology, lensing combined with WMAP7 results in h=0.78+/-0.12, OmegaM=0.252+/-0.079, sigma8=0.88+/-0.23 and w=-1.16+/-0.38 using only scales k<=1.5h/Mpc. We also present results of lensing combined with first year Planck results, where we find no tension with the results from this analysis, but we also find no significant improvement over the Planck results alone. We find evidence of a suppression of power compared to LCDM on small scales 1.5 < k < 5.0 h/Mpc in the lensing data, which is consistent with predictions of the effect of baryonic feedback on the matter power spectrum.

We measure the cross-correlation of cosmic microwave background lensing convergence maps derived from Atacama Cosmology Telescope data with galaxy lensing convergence maps as measured by the Canada-France-Hawaii Telescope Stripe 82 Survey. The CMB-galaxy lensing cross power spectrum is measured for the first time with a significance of 4.2{\sigma}, which corresponds to a 12% constraint on the amplitude of density fluctuations at redshifts ~ 0.9. With upcoming improved lensing data, this novel type of measurement will become a powerful cosmological probe, providing a precise measurement of the mass distribution at intermediate redshifts and serving as a calibrator for systematic biases in weak lensing measurements.

We present the cluster mass-richness scaling relation calibrated by a weak lensing analysis of >18000 galaxy cluster candidates in the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). Detected using the 3D-Matched-Filter cluster-finder of Milkeraitis et al., these cluster candidates span a wide range of masses, from the small group scale up to $\sim10^{15} M_{\odot}$, and redshifts 0.2 $\lesssim z\lesssim$ 0.9. The total significance of the stacked shear measurement amounts to 54$\sigma$. We compare cluster masses determined using weak lensing shear and magnification, finding the measurements in individual richness bins to yield 1$\sigma$ compatibility, but with magnification estimates biased low. This first direct mass comparison yields important insights for improving the systematics handling of future lensing magnification work. In addition, we confirm analyses that suggest cluster miscentring has an important effect on the observed 3D-MF halo profiles, and we quantify this by fitting for projected cluster centroid offsets, which are typically $\sim$ 0.4 arcmin. We bin the cluster candidates as a function of redshift, finding similar cluster masses and richness across the full range up to $z \sim$ 0.9. We measure the 3D-MF mass-richness scaling relation $M_{200} = M_0 (N_{200} / 20)^\beta$. We find a normalization $M_0 \sim (2.7^{+0.5}_{-0.4}) \times 10^{13} M_{\odot}$, and a logarithmic slope of $\beta \sim 1.4 \pm 0.1$, both of which are in 1$\sigma$ agreement with results from the magnification analysis. We find no evidence for a redshift-dependence of the normalization. The CFHTLenS 3D-MF cluster catalogue is now available at cfhtlens.org.

We investigate the dependence of the galaxy luminosity function on geometric environment within the Galaxy And Mass Assembly (GAMA) survey. The tidal tensor prescription, based on the Hessian of the pseudo-gravitational potential, is used to classify the cosmic web and define the geometric environments: for a given smoothing scale, we classify every position of the surveyed region, $0.04<{z}<0.26$, as either a void, a sheet, a filament or a knot. We consider how to choose appropriate thresholds in the eigenvalues of the Hessian in order to partition the galaxies approximately evenly between environments. We find a significant variation in the luminosity function of galaxies between different geometric environments; the normalisation, characterised by $\phi^{*}$ in a Schechter function fit, increases by an order of magnitude from voids to knots. The turnover magnitude, characterised by $M^*$, brightens by approximately $0.5$ mag from voids to knots. However, we show that the observed modulation can be entirely attributed to the indirect local-density dependence. We therefore find no evidence of a direct influence of the cosmic web on the galaxy luminosity function.

Galaxy-galaxy weak lensing is a direct probe of the mean matter distribution around galaxies. The depth and sky coverage of the CFHT Legacy Survey yield statistically significant galaxy halo mass measurements over a much wider range of stellar masses ($10^{8.75}$ to $10^{11.3} M_{\odot}$) and redshifts ($0.2 < z < 0.8$) than previous weak lensing studies. At redshift $z \sim 0.5$, the stellar-to-halo mass ratio (SHMR) reaches a maximum of $4.0\pm0.2$ percent as a function of halo mass at $\sim 10^{12.25} M_{\odot}$. We find, for the first time from weak lensing alone, evidence for significant evolution in the SHMR: the peak ratio falls as a function of cosmic time from $4.5 \pm 0.3$ percent at $z \sim 0.7$ to $3.4 \pm 0.2$ percent at $z \sim 0.3$, and shifts to lower stellar mass haloes. These evolutionary trends are dominated by red galaxies, and are consistent with a model in which the stellar mass above which star formation is quenched "downsizes" with cosmic time. In contrast, the SHMR of blue, star-forming galaxies is well-fit by a power law that does not evolve with time. This suggests that blue galaxies form stars at a rate that is balanced with their dark matter accretion in such a way that they evolve along the SHMR locus. The redshift dependence of the SHMR can be used to constrain the evolution of the galaxy population over cosmic time.

(Abridged) The effect of baryonic feedback on the dark matter mass distribution is generally considered to be a nuisance to weak gravitational lensing. Measurements of cosmological parameters are affected as feedback alters the cosmic shear signal on angular scales smaller than a few arcminutes. Recent progress on the numerical modelling of baryon physics has shown that this effect could be so large that, rather than being a nuisance, the effect can be constrained with current weak lensing surveys, hence providing an alternative astrophysical insight on one of the most challenging questions of galaxy formation. In order to perform our analysis, we construct an analytic fitting formula that describes the effect of the baryons on the mass power spectrum. This fitting formula is based on three scenarios of the OWL hydrodynamical simulations. It is specifically calibrated for $z<1.5$, where it models the simulations to an accuracy that is better than $2\%$ for scales $k<10 h\mbox{Mpc}^{-1}$ and better than $5\%$ for $10 < k < 100 h\mbox{Mpc}^{-1}$. Equipped with this precise tool, this paper presents the first constraint on baryonic feedback models using gravitational lensing data, from the Canada France Hawaii Telescope Lensing Survey (CFHTLenS). In this analysis, we show that the effect of neutrino mass on the mass power spectrum is degenerate with the baryonic feedback at small angular scales and cannot be ignored. Assuming a cosmology precision fixed by WMAP9, we find that a universe with no baryon feedback and massless neutrinos is rejected by the CFHTLenS lensing data with 96\% confidence. Our study shows that ongoing weak gravitational lensing surveys (KiDS, HSC and DES) will offer a unique opportunity to probe the physics of baryons at galactic scales, in addition to the expected constraints on the total neutrino mass.

Higher-order, non-Gaussian aspects of the large-scale structure carry valuable information on structure formation and cosmology, which is complementary to second-order statistics. In this work we measure second- and third-order weak-lensing aperture-mass moments from CFHTLenS and combine those with CMB anisotropy probes. The third moment is measured with a significance of $2\sigma$. The combined constraint on $\Sigma_8 = \sigma_8 (\Omega_{\rm m}/0.27)^\alpha$ is improved by 10%, in comparison to the second-order only, and the allowed ranges for $\Omega_{\rm m}$ and $\sigma_8$ are substantially reduced. Including general triangles of the lensing bispectrum yields tighter constraints compared to probing mainly equilateral triangles. Second- and third-order CFHTLenS lensing measurements improve Planck CMB constraints on $\Omega_{\rm m}$ and $\sigma_8$ by 26% for flat $\Lambda$CDM. For a model with free curvature, the joint CFHTLenS-Planck result is $\Omega_{\rm m} = 0.28 \pm 0.02$ (68% confidence), which is an improvement of 43% compared to Planck alone. We test how our results are potentially subject to three astrophysical sources of contamination: source-lens clustering, the intrinsic alignment of galaxy shapes, and baryonic effects. We explore future limitations of the cosmological use of third-order weak lensing, such as the nonlinear model and the Gaussianity of the likelihood function.

The GRavitational lEnsing Accuracy Testing 3 (GREAT3) challenge is the third in a series of image analysis challenges, with a goal of testing and facilitating the development of methods for analyzing astronomical images that will be used to measure weak gravitational lensing. This measurement requires extremely precise estimation of very small galaxy shape distortions, in the presence of far larger intrinsic galaxy shapes and distortions due to the blurring kernel caused by the atmosphere, telescope optics, and instrumental effects. The GREAT3 challenge is posed to the astronomy, machine learning, and statistics communities, and includes tests of three specific effects that are of immediate relevance to upcoming weak lensing surveys, two of which have never been tested in a community challenge before. These effects include realistically complex galaxy models based on high-resolution imaging from space; spatially varying, physically-motivated blurring kernel; and combination of multiple different exposures. To facilitate entry by people new to the field, and for use as a diagnostic tool, the simulation software for the challenge is publicly available, though the exact parameters used for the challenge are blinded. Sample scripts to analyze the challenge data using existing methods will also be provided. See http://great3challenge.info and http://great3.projects.phys.ucl.ac.uk/leaderboard/ for more information.

We outline the prospects for performing pioneering radio weak gravitational lensing analyses using observations from a potential forthcoming JVLA Sky Survey program. A large-scale survey with the JVLA can offer interesting and unique opportunities for performing weak lensing studies in the radio band, a field which has until now been the preserve of optical telescopes. In particular, the JVLA has the capacity for large, deep radio surveys with relatively high angular resolution, which are the key characteristics required for a successful weak lensing study. We highlight the potential advantages and unique aspects of performing weak lensing in the radio band. In particular, the inclusion of continuum polarisation information can greatly reduce noise in weak lensing reconstructions and can also remove the effects of intrinsic galaxy alignments, the key astrophysical systematic effect that limits weak lensing at all wavelengths. We identify a VLASS "deep fields" program (total area ~10-20 square degs), to be conducted at L-band and with high-resolution (A-array configuration), as the optimal survey strategy from the point of view of weak lensing science. Such a survey will build on the unique strengths of the JVLA and will remain unsurpassed in terms of its combination of resolution and sensitivity until the advent of the Square Kilometre Array. We identify the best fields on the JVLA-accessible sky from the point of view of overlapping with existing deep optical and near infra-red data which will provide crucial redshift information and facilitate a host of additional compelling multi-wavelength science.

In this paper, we motivate the use of galaxy clustering measurements using photometric redshift information, including a contribution from flux magnification, as a probe of cosmology. We present cosmological forecasts when clustering data alone is used, and when clustering is combined with a cosmic shear analysis. We consider two types of clustering analysis: firstly, clustering with only redshift auto-correlations in tomographic redshift bins; secondly, using all available redshift bin correlations. Finally, we consider how inferred cosmological parameters may be biased using each analysis when flux magnification is neglected. Results are presented for a Stage III ground-based survey, and a Stage IV space-based survey modelled with photometric redshift errors, and values for the slope of the luminosity function inferred from CFHTLenS catalogues. We find that combining clustering information with shear can improve constraints on cosmological parameters, giving an improvement to a Dark Energy Task Force-like figure of merit by a factor of 1.33 when only auto-correlations in redshift are used for the clustering analysis, rising to 1.52 when cross-correlations in redshift are also included. The addition of galaxy-galaxy lensing gives further improvement, with increases in figure of merit by a factor of 2.82 and 3.7 for each type of clustering analysis respectively. The presence of flux magnification in a clustering analysis does not significantly affect the precision of cosmological constraints when combined with cosmic shear and galaxy-galaxy lensing. However if magnification is neglected, inferred cosmological parameter values are biased, with biases in some cosmological parameters larger than statistical errors. (Abridged)

We present a study of the relation between dark matter halo mass and the baryonic content of host galaxies, quantified via luminosity and stellar mass. Our investigation uses 154 deg2 of Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS) lensing and photometric data, obtained from the CFHT Legacy Survey. We employ a galaxy-galaxy lensing halo model which allows us to constrain the halo mass and the satellite fraction. Our analysis is limited to lenses at redshifts between 0.2 and 0.4. We express the relationship between halo mass and baryonic observable as a power law. For the luminosity-halo mass relation we find a slope of 1.32+/-0.06 and a normalisation of 1.19+0.06-0.07x10^13 h70^-1 Msun for red galaxies, while for blue galaxies the best-fit slope is 1.09+0.20-0.13 and the normalisation is 0.18+0.04-0.05x10^13 h70^-1 Msun. Similarly, we find a best-fit slope of 1.36+0.06-0.07 and a normalisation of 1.43+0.11-0.08x10^13 h70^-1 Msun for the stellar mass-halo mass relation of red galaxies, while for blue galaxies the corresponding values are 0.98+0.08-0.07 and 0.84+0.20-0.16x10^13 h70^-1 Msun. For red lenses, the fraction which are satellites tends to decrease with luminosity and stellar mass, with the sample being nearly all satellites for a stellar mass of 2x10^9 h70^-2 Msun. The satellite fractions are generally close to zero for blue lenses, irrespective of luminosity or stellar mass. This, together with the shallower relation between halo mass and baryonic tracer, is a direct confirmation from galaxy-galaxy lensing that blue galaxies reside in less clustered environments than red galaxies. We also find that the halo model, while matching the lensing signal around red lenses well, is prone to over-predicting the large-scale signal for faint and less massive blue lenses. This could be a further indication that these galaxies tend to be more isolated than assumed. [abridged]

We present a method for suppressing contributions from higher-order terms in perturbation theory, greatly increasing the amount of information which may be extracted from the matter power spectrum. In an evolved cosmological density field the highest density regions are responsible for the bulk of the nonlinear power. By suitably down-weighting these problematic regions we find that the one- and two-loop terms are typically reduced in amplitude by ~70 per cent and ~95 per cent respectively, relative to the linear power spectrum. This greatly facilitates modelling the shape of the galaxy power spectrum, potentially increasing the number of useful Fourier modes by more than two orders of magnitude. We provide a demonstration of how this technique allows the galaxy bias and the amplitude of linear matter perturbations sigma_8 to be determined from the power spectrum on conventionally nonlinear scales, 0.1<k<0.7 h/Mpc.

We present a simulation analysis of weak gravitational lensing flexion and shear measurement using shapelet decomposition, and identify differences between flexion and shear measurement noise in deep survey data. Taking models of galaxies from the Hubble Space Telescope Ultra Deep Field (HUDF) and applying a correction for the HUDF point spread function we generate lensed simulations of deep, optical imaging data from Hubble's Advanced Camera for Surveys (ACS), with realistic galaxy morphologies. We find that flexion and shear estimates differ in our measurement pipeline: whereas intrinsic galaxy shape is typically the dominant contribution to noise in shear estimates, pixel noise due to finite photon counts and detector read noise is a major contributor to uncertainty in flexion estimates, across a broad range of galaxy signal-to-noise. This pixel noise also increases more rapidly as galaxy signal-to-noise decreases than is found for shear estimates. We provide simple power law fitting functions for this behaviour, for both flexion and shear, allowing the effect to be properly accounted for in future forecasts for flexion measurement. Using the simulations we also quantify the systematic biases of our shapelet flexion and shear measurement pipeline for deep Hubble data sets such as Galaxy Evolution from Morphology and SEDs, Space Telescope A901/902 Galaxy Evolution Survey or the Cosmic Evolution Survey. Flexion measurement biases are found to be significant but consistent with previous studies.

We present a quantitative analysis of the largest contiguous maps of projected mass density obtained from gravitational lensing shear. We use data from the 154 deg2 covered by the Canada-France-Hawaii Telescope Lensing Survey. Our study is the first attempt to quantitatively characterize the scientific value of lensing maps, which could serve in the future as a complementary approach to the study of the dark universe with gravitational lensing. We show that mass maps contain unique cosmological information beyond that of traditional two-points statistical analysis techniques. Using a series of numerical simulations, we first show how, reproducing the CFHTLenS observing conditions, gravitational lensing inversion provides a reliable estimate of the projected matter distribution of large scale structure. We validate our analysis by quantifying the robustness of the maps with various statistical estimators. We then apply the same process to the CFHTLenS data. We find that the 2-points correlation function of the projected mass is consistent with the cosmological analysis performed on the shear correlation function discussed in the CFHTLenS companion papers. The maps also lead to a significant measurement of the third order moment of the projected mass, which is in agreement with analytic predictions, and to a marginal detection of the fourth order moment. Tests for residual systematics are found to be consistent with zero for the statistical estimators we used. A new approach for the comparison of the reconstructed mass map to that predicted from the galaxy distribution reveals the existence of giant voids in the dark matter maps as large as 3 degrees on the sky. Our analysis shows that lensing mass maps can be used for new techniques such as peak statistics and the morphological analysis of the projected dark matter distribution.

We present a finely-binned tomographic weak lensing analysis of the Canada-France-Hawaii Telescope Lensing Survey, CFHTLenS, mitigating contamination to the signal from the presence of intrinsic galaxy alignments via the simultaneous fit of a cosmological model and an intrinsic alignment model. CFHTLenS spans 154 square degrees in five optical bands, with accurate shear and photometric redshifts for a galaxy sample with a median redshift of zm =0.70. We estimate the 21 sets of cosmic shear correlation functions associated with six redshift bins, each spanning the angular range of 1.5<theta<35 arcmin. We combine this CFHTLenS data with auxiliary cosmological probes: the cosmic microwave background with data from WMAP7, baryon acoustic oscillations with data from BOSS, and a prior on the Hubble constant from the HST distance ladder. This leads to constraints on the normalisation of the matter power spectrum sigma_8 = 0.799 +/- 0.015 and the matter density parameter Omega_m = 0.271 +/- 0.010 for a flat Lambda CDM cosmology. For a flat wCDM cosmology we constrain the dark energy equation of state parameter w = -1.02 +/- 0.09. We also provide constraints for curved Lambda CDM and wCDM cosmologies. We find the intrinsic alignment contamination to be galaxy-type dependent with a significant intrinsic alignment signal found for early-type galaxies, in contrast to the late-type galaxy sample for which the intrinsic alignment signal is found to be consistent with zero.

In this paper we present results from the weak lensing shape measurement GRavitational lEnsing Accuracy Testing 2010 (GREAT10) Galaxy Challenge. This marks an order of magnitude step change in the level of scrutiny employed in weak lensing shape measurement analysis. We provide descriptions of each method tested and include 10 evaluation metrics over 24 simulation branches. GREAT10 was the first shape measurement challenge to include variable fields; both the shear field and the Point Spread Function (PSF) vary across the images in a realistic manner. The variable fields enable a variety of metrics that are inaccessible to constant shear simulations including a direct measure of the impact of shape measurement inaccuracies, and the impact of PSF size and ellipticity, on the shear power spectrum. To assess the impact of shape measurement bias for cosmic shear we present a general pseudo-Cl formalism, that propagates spatially varying systematics in cosmic shear through to power spectrum estimates. We also show how one-point estimators of bias can be extracted from variable shear simulations. The GREAT10 Galaxy Challenge received 95 submissions and saw a factor of 3 improvement in the accuracy achieved by shape measurement methods. The best methods achieve sub-percent average biases. We find a strong dependence in accuracy as a function of signal-to-noise, and indications of a weak dependence on galaxy type and size. Some requirements for the most ambitious cosmic shear experiments are met above a signal-to-noise ratio of 20. These results have the caveat that the simulated PSF was a ground-based PSF. Our results are a snapshot of the accuracy of current shape measurement methods and are a benchmark upon which improvement can continue. This provides a foundation for a better understanding of the strengths and limitations of shape measurement methods.

We use weak gravitational lensing to analyse the dark matter halos around satellite galaxies in galaxy groups in the CFHTLenS dataset. This dataset is derived from the CFHTLS-Wide survey, and encompasses 154 sq. deg of high-quality shape data. Using the photometric redshifts, we divide the sample of lens galaxies with stellar masses in the range 10^9 Msun to 10^10.5 Msun into those likely to lie in high-density environments (HDE) and those likely to lie in low-density environments (LDE). Through comparison with galaxy catalogues extracted from the Millennium Simulation, we show that the sample of HDE galaxies should primarily (~61%) consist of satellite galaxies in groups, while the sample of LDE galaxies should consist of mostly (~87%) non-satellite (field and central) galaxies. Comparing the lensing signals around samples of HDE and LDE galaxies matched in stellar mass, the lensing signal around HDE galaxies clearly shows a positive contribution from their host groups on their lensing signals at radii of ~500--1000 kpc, the typical separation between satellites and group centres. More importantly, the subhalos of HDE galaxies are less massive than those around LDE galaxies by a factor 0.65 +/- 0.12, significant at the 2.9 sigma level. A natural explanation is that the halos of satellite galaxies are stripped through tidal effects in the group environment. Our results are consistent with a typical tidal truncation radius of ~40 kpc.

The Galaxy And Mass Assembly (GAMA) survey is a multiwavelength photometric and spectroscopic survey, using the AAOmega spectrograph on the Anglo-Australian Telescope to obtain spectra for up to ~300000 galaxies over 280 square degrees, to a limiting magnitude of r_pet < 19.8 mag. The target galaxies are distributed over 0<z<0.5 with a median redshift of z~0.2, although the redshift distribution includes a small number of systems, primarily quasars, at higher redshifts, up to and beyond z=1. The redshift accuracy ranges from sigma_v~50km/s to sigma_v~100km/s depending on the signal-to-noise of the spectrum. Here we describe the GAMA spectroscopic reduction and analysis pipeline. We present the steps involved in taking the raw two-dimensional spectroscopic images through to flux-calibrated one-dimensional spectra. The resulting GAMA spectra cover an observed wavelength range of 3750<lambda<8850 A at a resolution of R~1300. The final flux calibration is typically accurate to 10-20%, although the reliability is worse at the extreme wavelength ends, and poorer in the blue than the red. We present details of the measurement of emission and absorption features in the GAMA spectra. These measurements are characterised through a variety of quality control analyses detailing the robustness and reliability of the measurements. We illustrate the quality of the measurements with a brief exploration of elementary emission line properties of the galaxies in the GAMA sample. We demonstrate the luminosity dependence of the Balmer decrement, consistent with previously published results, and explore further how Balmer decrement varies with galaxy mass and redshift. We also investigate the mass and redshift dependencies of the [NII]/Halpha vs [OIII]/Hbeta spectral diagnostic diagram, commonly used to discriminate between star forming and nuclear activity in galaxies.

We present the first direct measurement of the galaxy-matter bispectrum as a function of galaxy luminosity, stellar mass, and SED type. Our analysis uses a galaxy-galaxy-galaxy lensing technique (G3L), on angular scales between 9 arcsec to 50 arcmin, to quantify (i) the excess surface mass density around galaxy pairs (excess mass hereafter) and (ii) the excess shear-shear correlations around single galaxies, both of which yield a measure of two types of galaxy-matter bispectra. We apply our method to the state-of-the-art Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), spanning 154 square degrees. This survey allows us to detect a significant change of the bispectra with lens properties. Measurements for lens populations with distinct redshift distributions become comparable by a newly devised normalisation technique. That will also aid future comparisons to other surveys or simulations. A significant dependence of the normalised G3L statistics on luminosity within -23<M_r<-18 and stellar mass within 5x10^9 M_sol<M_star<2x10^11 M_sol is found (h=0.73). Both bispectra exhibit a stronger signal for more luminous lenses or those with higher stellar mass (up to a factor 2-3). This is accompanied by a steeper equilateral bispectrum for more luminous or higher stellar mass lenses for the excess mass. Importantly, we find the excess mass to be very sensitive to galaxy type as recently predicted with semi-analytic galaxy models: luminous (M_r<-21) late-type galaxies show no detectable signal, while all excess mass detected for luminous galaxies seems to be associated with early-type galaxies. We also present the first observational constraints on third-order stochastic galaxy biasing parameters.

We present cosmological constraints from 2D weak gravitational lensing by the large-scale structure in the Canada-France Hawaii Telescope Lensing Survey (CFHTLenS) which spans 154 square degrees in five optical bands. Using accurate photometric redshifts and measured shapes for 4.2 million galaxies between redshifts of 0.2 and 1.3, we compute the 2D cosmic shear correlation function over angular scales ranging between 0.8 and 350 arcmin. Using non-linear models of the dark-matter power spectrum, we constrain cosmological parameters by exploring the parameter space with Population Monte Carlo sampling. The best constraints from lensing alone are obtained for the small-scale density-fluctuations amplitude sigma_8 scaled with the total matter density Omega_m. For a flat LambdaCDM model we obtain sigma_8(Omega_m/0.27)^0.6 = 0.79+-0.03. We combine the CFHTLenS data with WMAP7, BOSS and an HST distance-ladder prior on the Hubble constant to get joint constraints. For a flat LambdaCDM model, we find Omega_m = 0.283+-0.010 and sigma_8 = 0.813+-0.014. In the case of a curved wCDM universe, we obtain Omega_m = 0.27+-0.03, sigma_8 = 0.83+-0.04, w_0 = -1.10+-0.15 and Omega_K = 0.006+0.006-0.004. We calculate the Bayesian evidence to compare flat and curved LambdaCDM and dark-energy CDM models. From the combination of all four probes, we find models with curvature to be at moderately disfavoured with respect to the flat case. A simple dark-energy model is indistinguishable from LambdaCDM. Our results therefore do not necessitate any deviations from the standard cosmological model.

The Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS) comprises deep multi-colour (u*g'r'i'z') photometry spanning 154 square degrees, with accurate photometric redshifts and shape measurements. We demonstrate that the redshift probability distribution function summed over galaxies provides an accurate representation of the galaxy redshift distribution accounting for random and catastrophic errors for galaxies with best fitting photometric redshifts z_p < 1.3. We present cosmological constraints using tomographic weak gravitational lensing by large-scale structure. We use two broad redshift bins 0.5 < z_p <= 0.85 and 0.85 < z_p <= 1.3 free of intrinsic alignment contamination, and measure the shear correlation function on angular scales in the range ~1-40 arcmin. We show that the problematic redshift scaling of the shear signal, found in previous CFHTLS data analyses, does not afflict the CFHTLenS data. For a flat Lambda-CDM model and a fixed matter density Omega_m=0.27, we find the normalisation of the matter power spectrum sigma_8=0.771 \pm 0.041. When combined with cosmic microwave background data (WMAP7), baryon acoustic oscillation data (BOSS), and a prior on the Hubble constant from the HST distance ladder, we find that CFHTLenS improves the precision of the fully marginalised parameter estimates by an average factor of 1.5-2. Combining our results with the above cosmological probes, we find Omega_m=0.2762 \pm 0.0074 and sigma_8=0.802 \pm 0.013.

We perform a quantitative morphological comparison between the hosts of Active Galactic Nuclei (AGN) and quiescent galaxies at intermediate redshifts (z~0.7). The imaging data are taken from the large HST/ACS mosaics of the GEMS and STAGES surveys. Our main aim is to test whether nuclear activity at this cosmic epoch is triggered by major mergers. Using images of quiescent galaxies and stars, we create synthetic AGN images to investigate the impact of an optical nucleus on the morphological analysis of AGN hosts. Galaxy morphologies are parameterized using the asymmetry index A, concentration index C, Gini coefficient G and M20 index. A sample of ~200 synthetic AGN is matched to 21 real AGN in terms of redshift, host brightness and host-to-nucleus ratio to ensure a reliable comparison between active and quiescent galaxies. The optical nuclei strongly affect the morphological parameters of the underlying host galaxy. Taking these effects into account, we find that the morphologies of the AGN hosts are clearly distinct from galaxies undergoing violent gravitational interactions. In fact, the host galaxies' distributions in morphological descriptor space are more similar to undisturbed galaxies than major mergers. Intermediate-luminosity (Lx < 10^44 erg/s) AGN hosts at z~0.7 show morphologies similar to the general population of massive galaxies with significant bulges at the same redshifts. If major mergers are the driver of nuclear activity at this epoch, the signatures of gravitational interactions fade rapidly before the optical AGN phase starts, making them undetectable on single-orbit HST images, at least with usual morphological descriptors. This could be investigated in future synthetic observations created from numerical simulations of galaxy-galaxy interactions.

A likelihood-based method for measuring weak gravitational lensing shear in deep galaxy surveys is described and applied to the Canada-France-Hawaii Telescope (CFHT) Lensing Survey (CFHTLenS). CFHTLenS comprises 154 sq deg of multicolour optical data from the CFHT Legacy Survey, with lensing measurements being made in the i' band to a depth i'(AB)<24.7, for galaxies with signal-to-noise ratio greater than about 10. The method is based on the lensfit algorithm described in earlier papers, but here we describe a full analysis pipeline that takes into account the properties of real surveys. The method creates pixel-based models of the varying point spread function (PSF) in individual image exposures. It fits PSF-convolved two-component (disk plus bulge) models, to measure the ellipticity of each galaxy, with bayesian marginalisation over model nuisance parameters of galaxy position, size, brightness and bulge fraction. The method allows optimal joint measurement of multiple, dithered image exposures, taking into account imaging distortion and the alignment of the multiple measurements. We discuss the effects of noise bias on the likelihood distribution of galaxy ellipticity. Two sets of image simulations that mirror the observed properties of CFHTLenS have been created, to establish the method's accuracy and to derive an empirical correction for the effects of noise bias.

We present data products from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). CFHTLenS is based on the Wide component of the Canada-France-Hawaii Telescope Legacy Survey (CFHTLS). It encompasses 154 deg^2 of deep, optical, high-quality, sub-arcsecond imaging data in the five optical filters u^*g'r'i'z'. The article presents our data processing of the complete CFHTLenS data set. We were able to obtain a data set with very good image quality and high-quality astrometric and photometric calibration. Our external astrometric accuracy is between 60-70 mas with respect to SDSS data and the internal alignment in all filters is around 30 mas. Our average photometric calibration shows a dispersion on the order of 0.01 to 0.03 mag for g'r'i'z' and about 0.04 mag for u^* with respect to SDSS sources down to i <= 21. In the spirit of the CFHTLS all our data products are released to the astronomical community via the Canadian Astronomy Data Centre. We give a description and how-to manuals of the public products which include image pixel data, source catalogues with photometric redshift estimates and all relevant quantities to perform weak lensing studies.

We present the results from the first public blind PSF reconstruction challenge, the GRavitational lEnsing Accuracy Testing 2010 (GREAT10) Star Challenge. Reconstruction of a spatially varying PSF, sparsely sampled by stars, at non-star positions is a critical part in the image analysis for weak lensing where inaccuracies in the modelled ellipticity and size-squared can impact the ability to measure the shapes of galaxies. This is of importance because weak lensing is a particularly sensitive probe of dark energy, and can be used to map the mass distribution of large scale structure. Participants in the challenge were presented with 27,500 stars over 1300 images subdivided into 26 sets, where in each set a category change was made in the type or spatial variation of the PSF. Thirty submissions were made by 9 teams. The best methods reconstructed the PSF with an accuracy of ~0.00025 in ellipticity and ~0.00074 in size squared. For a fixed pixel scale narrower PSFs were found to be more difficult to model than larger PSFs, and the PSF reconstruction was severely degraded with the inclusion of an atmospheric turbulence model (although this result is likely to be a strong function of the amplitude of the turbulence power spectrum).

We present the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS) that accurately determines a weak gravitational lensing signal from the full 154 square degrees of deep multi-colour data obtained by the CFHT Legacy Survey. Weak gravitational lensing by large-scale structure is widely recognised as one of the most powerful but technically challenging probes of cosmology. We outline the CFHTLenS analysis pipeline, describing how and why every step of the chain from the raw pixel data to the lensing shear and photometric redshift measurement has been revised and improved compared to previous analyses of a subset of the same data. We present a novel method to identify data which contributes a non-negligible contamination to our sample and quantify the required level of calibration for the survey. Through a series of cosmology-insensitive tests we demonstrate the robustness of the resulting cosmic shear signal, presenting a science-ready shear and photometric redshift catalogue for future exploitation.

In this paper we present results from the Mapping Dark Matter competition that expressed the weak lensing shape measurement task in its simplest form and as a result attracted over 700 submissions in 2 months and a factor of 3 improvement in shape measurement accuracy on high signal to noise galaxies, over previously published results, and a factor 10 improvement over methods tested on constant shear blind simulations. We also review weak lensing shape measurement challenges, including the Shear TEsting Programmes (STEP1 and STEP2) and the GRavitational lEnsing Accuracy Testing competitions (GREAT08 and GREAT10).

Here we present the results of various approaches to measure accurate colours and photometric redshifts (photo-z's) from wide-field imaging data. We use data from the Canada-France-Hawaii-Telescope Legacy Survey (CFHTLS) which have been re- processed by the CFHT Lensing Survey (CFHTLenS) team in order to carry out a number of weak gravitational lensing studies. An emphasis is put on the correction of systematic effects in the photo-z's arising from the different Point Spread Functions (PSF) in the five optical bands. Different ways of correcting these effects are discussed and the resulting photo-z accuracies are quantified by comparing the photo-z's to large spectroscopic redshift (spec-z) data sets. Careful homogenisation of the PSF between bands leads to increased overall accuracy of photo-z's. The gain is particularly pronounced at fainter magnitudes where galaxies are smaller and flux measurements are affected more by PSF-effects. We also study possible re- calibrations of the photometric zeropoints (ZPs) with the help of galaxies with known spec-z's. We find that if PSF-effects are properly taken into account, a re-calibration of the ZPs becomes much less important suggesting that previous such re-calibrations described in the literature could in fact be mostly corrections for PSF-effects rather than corrections for real inaccuracies in the ZPs. The implications of this finding for future surveys like KiDS, DES, LSST, or Euclid are mixed. On the one hand, ZP re-calibrations with spec-z's might not be as accurate as previously thought. On the other hand, careful PSF homogenisation might provide a way out and yield accurate, homogeneous photometry without the need for full spectroscopic coverage. This is the first paper in a series describing the technical aspects of CFHTLenS. (abridged)

High precision cosmology with weak gravitational lensing requires a precise measure of the Point Spread Function across the imaging data where the accuracy to which high spatial frequency variation can be modelled is limited by the stellar number density across the field. We analyse dense stellar fields imaged at the Canada-France-Hawaii Telescope to quantify the degree of high spatial frequency variation in ground-based imaging Point Spread Functions and compare our results to models of atmospheric turbulence. The data shows an anisotropic turbulence pattern with an orientation independent of the wind direction and wind speed. We find the amplitude of the high spatial frequencies to decrease with increasing exposure time as $t^{-1/2}$, and find a negligibly small atmospheric contribution to the Point Spread Function ellipticity variation for exposure times $t>180$ seconds. For future surveys analysing shorter exposure data, this anisotropic turbulence will need to be taken into account as the amplitude of the correlated atmospheric distortions becomes comparable to a cosmological lensing signal on scales less than $\sim 10$ arcminutes. This effect could be mitigated, however, by correlating galaxy shear measured on exposures imaged with a time separation greater than 50 seconds, for which we find the spatial turbulence patterns to be uncorrelated.

GRavitational lEnsing Accuracy Testing 2010 (GREAT10) is a public image analysis challenge aimed at the development of algorithms to analyze astronomical images. Specifically, the challenge is to measure varying image distortions in the presence of a variable convolution kernel, pixelization and noise. This is the second in a series of challenges set to the astronomy, computer science and statistics communities, providing a structured environment in which methods can be improved and tested in preparation for planned astronomical surveys. GREAT10 extends upon previous work by introducing variable fields into the challenge. The "Galaxy Challenge" involves the precise measurement of galaxy shape distortions, quantified locally by two parameters called shear, in the presence of a known convolution kernel. Crucially, the convolution kernel and the simulated gravitational lensing shape distortion both now vary as a function of position within the images, as is the case for real data. In addition, we introduce the "Star Challenge" that concerns the reconstruction of a variable convolution kernel, similar to that in a typical astronomical observation. This document details the GREAT10 Challenge for potential participants. Continually updated information is also available from http://www.greatchallenges.info.

A large fraction of the information collected by cosmological surveys is simply discarded to avoid lengthscales which are difficult to model theoretically. We introduce a new technique which enables the extraction of useful information from the bispectrum of galaxies well beyond the conventional limits of perturbation theory. Our results strongly suggest that this method increases the range of scales where the relation between the bispectrum and power spectrum in tree-level perturbation theory may be applied, from k_max ~ 0.1 h/Mpc to ~ 0.7 h/Mpc. This leads to correspondingly large improvements in the determination of galaxy bias. Since the clipped matter power spectrum closely follows the linear power spectrum, there is the potential to use this technique to probe the growth rate of linear perturbations and confront theories of modified gravity with observation.

We present weak lensing data from the HST/STAGES survey to study the three-dimensional spatial distribution of matter and galaxies in the Abell 901/902 supercluster complex. Our method improves over the existing 3D lensing mapping techniques by calibrating and removing redshift bias and accounting for the effects of the radial elongation of 3D structures. We also include the first detailed noise analysis of a 3D lensing map, showing that even with deep HST quality data, only the most massive structures, for example M200>~10^15 Msun/h at z~0.8, can be resolved in 3D with any reasonable redshift accuracy (\Delta z~0.15). We compare the lensing map to the stellar mass distribution and find luminous counterparts for all mass peaks detected with a peak significance >3\sigma. We see structures in and behind the z=0.165 foreground supercluster, finding structure directly behind the A901b cluster at z~0.6 and also behind the SW group at z~0.7. This 3D structure viewed in projection has no significant impact on recent mass estimates of A901b or the SW group components SWa and SWb.

Euclid is a space-based survey mission from the European Space Agency designed to understand the origin of the Universe's accelerating expansion. It will use cosmological probes to investigate the nature of dark energy, dark matter and gravity by tracking their observational signatures on the geometry of the universe and on the cosmic history of structure formation. The mission is optimised for two independent primary cosmological probes: Weak gravitational Lensing (WL) and Baryonic Acoustic Oscillations (BAO). The Euclid payload consists of a 1.2 m Korsch telescope designed to provide a large field of view. It carries two instruments with a common field-of-view of ~0.54 deg2: the visual imager (VIS) and the near infrared instrument (NISP) which contains a slitless spectrometer and a three bands photometer. The Euclid wide survey will cover 15,000 deg2 of the extragalactic sky and is complemented by two 20 deg2 deep fields. For WL, Euclid measures the shapes of 30-40 resolved galaxies per arcmin2 in one broad visible R+I+Z band (550-920 nm). The photometric redshifts for these galaxies reach a precision of dz/(1+z) < 0.05. They are derived from three additional Euclid NIR bands (Y, J, H in the range 0.92-2.0 micron), complemented by ground based photometry in visible bands derived from public data or through engaged collaborations. The BAO are determined from a spectroscopic survey with a redshift accuracy dz/(1+z) =0.001. The slitless spectrometer, with spectral resolution ~250, predominantly detects Ha emission line galaxies. Euclid is a Medium Class mission of the ESA Cosmic Vision 2015-2025 programme, with a foreseen launch date in 2019. This report (also known as the Euclid Red Book) describes the outcome of the Phase A study.

We present an optimised galaxy cluster finder, 3D-Matched-Filter (3D-MF), which utilises galaxy cluster radial profiles, luminosity functions and redshift information to detect galaxy clusters in optical surveys. This method is an improvement over other matched-filter methods, most notably through implementing redshift slicing of the data to significantly reduce line-of-sight projections and related false positives. We apply our method to the Canada-France-Hawaii Telescope Legacy Survey (CFHTLS) Deep fields, finding ~170 galaxy clusters per square degree in the 0.2 <= z <= 1.0 redshift range. Future surveys such as LSST and JDEM can exploit 3D-MF's automated methodology to produce complete and reliable galaxy cluster catalogues. We determine the reliability and accuracy of the statistical approach of our method through a thorough analysis of mock data from the Millennium Simulation. We detect clusters with 100% completeness for M_200 >= 3.0x10^(14)M_sun, 88% completeness for M_200 >= 1.0x10^(14)M_sun, and 72% completeness well into the 10^(13)M_sun cluster mass range. We show a 36% multiple detection rate for cluster masses >= 1.5x10^(13)M_sun and a 16% false detection rate for galaxy clusters >~ 5x10^(13)M_sun, reporting that for clusters with masses <~ 5x10^(13)M_sun false detections may increase up to ~24%. Utilising these selection functions we conclude that our galaxy cluster catalogue is the most complete CFHTLS Deep cluster catalogue to date.

We investigate the properties of bright (MV <= -18) barred and unbarred disks in the Abell 901/902 cluster system at z~0.165 with the STAGES HST ACS survey. To identify and characterize bars, we use ellipse-fitting. We use visual classification, a Sersic cut, and a color cut to select disk galaxies, and find that the latter two methods miss 31% and 51%, respectively of disk galaxies identified through visual classification. This underscores the importance of carefully selecting the disk sample in cluster environments. However, we find that the global optical bar fraction in the clusters is ~30% regardless of the method of disk selection. We study the relationship of the optical bar fraction to host galaxy properties, and find that the optical bar fraction depends strongly on the luminosity of the galaxy and whether it hosts a prominent bulge or is bulgeless. Within a given absolute magnitude bin, the optical bar fraction increases for galaxies with no significant bulge component. Within each morphological type bin, the optical bar fraction increases for brighter galaxies. We find no strong trend (variations larger than a factor of 1.3) for the optical bar fraction with local density within the cluster between the core and virial radius (R ~ 0.25 to 1.2 Mpc). We discuss the implications of our results for the evolution of bars and disks in dense environments.

We present a non-parametric measure of the ellipticity and the alignment of the dark matter halos in Abell 901/902 supercluster. This super-cluster is a system of four separate peaks in a $0.5^{\circ}\times0.5^{\circ}$ field of view. We map the mass distribution of each individual peak using an improved version of Particle Based Lensing (PBL) and measure the ellipticity of the dark matter halos associated with two of the peaks directly from the mass map and by fitting them to a singular isothermal ellipse. The parametric and non-parametric measurements are consistent for A901b while the position angle for the Southwest Group is different for the two techniques. We account for this discrepancy to substructure present in the Southwest Peak. We estimate an axis ratio of $0.37\pm 0.1$ for A901b and $0.54^{+0.08}_{-0.09}$ for the Southwest Group.

Measurements of X-ray scaling laws are critical for improving cosmological constraints derived with the halo mass function and for understanding the physical processes that govern the heating and cooling of the intracluster medium. In this paper, we use a sample of 206 X-ray selected galaxy groups to investigate the scaling relation between X-ray luminosity (Lx) and halo mass (M00) where M200 is derived via stacked weak gravitational lensing. This work draws upon a broad array of multi-wavelength COSMOS observations including 1.64 square degrees of contiguous imaging with the Advanced Camera for Surveys (ACS) and deep XMM-Newton/Chandra imaging. The combined depth of these two data-sets allows us to probe the lensing signals of X-ray detected structures at both higher redshifts and lower masses than previously explored. Weak lensing profiles and halo masses are derived for nine sub-samples, narrowly binned in luminosity and redshift. The COSMOS data alone are well fit by a power law, M200 ~ Lx^a, with a slope of a=0.66+-0.14. These results significantly extend the dynamic range for which the halo masses of X-ray selected structures have been measured with weak gravitational lensing. As a result, tight constraints are obtained for the slope of the M-Lx relation. The combination of our group data with previously published cluster data demonstrates that the M-Lx relation is well described by a single power law, a=0.64+-0.03, over two decades in mass, 10^13.5-10^15.5 h72^-1 Msun. These results are inconsistent at the 3.7 level with the self-similar prediction of a=0.75. We examine the redshift dependence of the M-Lx relation and find little evidence for evolution beyond the rate predicted by self-similarity from z ~ 0.25 to z ~ 0.8.

We present a study of the population of bright early-type dwarf galaxies in the multiple-cluster system Abell 901/902. We use data from the STAGES survey and COMBO-17 to investigate the relation between the color and structural properties of the dwarfs and their location in the cluster. The definition of the dwarf sample is based on the central surface brightness and includes galaxies in the luminosity range -16 >= M_B >~-19 mag. Using a fit to the color magnitude relation of the dwarfs, our sample is divided into a red and blue subsample. We find a color-density relation in the projected radial distribution of the dwarf sample: at the same luminosity dwarfs with redder colors are located closer to the cluster centers than their bluer counterparts. Furthermore, the redder dwarfs are on average more compact and rounder than the bluer dwarfs. These findings are consistent with theoretical expectations assuming that bright early-type dwarfs are the remnants of transformed late-type disk galaxies involving processes such as ram pressure stripping and galaxy harassment. This indicates that a considerable fraction of dwarf elliptical galaxies in clusters are the results of transformation processes related to interactions with their host cluster.

We present a study of galaxy mergers and the influence of environment in the Abell 901/902 supercluster at z~0.165. We use HST ACS F606W data from the STAGES survey, COMBO-17, Spitzer 24um, and XMM-Newton X-ray data. Our analysis utilizes both a visual classification system, and quantitative CAS parameters to identify systems which show evidence of a recent or ongoing merger of mass ratio >1/10. Our results are: (1) After visual classification and minimizing the contamination from false projection pairs, we find that the merger fraction f_merge is 0.023+/-0.007. The estimated fractions of likely major mergers, likely minor mergers, and ambiguous cases are 0.01+/-0.004, 0.006+/-0.003, and 0.007+/-0.003, respectively. (2) The mergers lie outside the cluster core of radius R < 0.25 Mpc: the lack of mergers in the core is likely due to the large galaxy velocity dispersion in the core. Mergers populate the region (0.25 Mpc < R <= 2 Mpc) between the core and outskirt. In this region, the estimated frequency of mergers is similar to those seen at typical group overdensities. This suggests ongoing growth of the clusters via accretion of group and field galaxies. (3) We compare our observed merger fraction with those reported in other clusters and groups out to z~0.4. Existing data points on the merger fraction for L<= L* galaxies in clusters allow for a range of evolutionary scenarios. (4) The fraction of mergers, which lie on the blue cloud is 80%+/-18% versus 34%+/-7% for non-interacting galaxies, implying that interacting galaxies are preferentially blue. (5) The average SFR, based on UV or UV+IR data, is enhanced by a factor of ~1.5 to 2 in mergers compared to non-interacting galaxies. However, mergers in the clusters contribute only a small fraction (between 10% and 15%) of the total SFR density.(Abridged)

We study the optimal use of third order statistics in the analysis of weak lensing by large-scale structure. These higher order statistics have long been advocated as a powerful tool to break measured degeneracies between cosmological parameters. Using ray-tracing simulations, incorporating important survey features such as a realistic depth-dependent redshift distribution, we find that a joint two- and three-point correlation function analysis is a much stronger probe of cosmology than the skewness statistic. We compare different observing strategies, showing that for a limited survey time there is an optimal depth for the measurement of third-order statistics, which balances statistical noise and cosmic variance against signal amplitude. We find that the chosen CFHTLS observing strategy was optimal and forecast that a joint two- and three-point analysis of the completed CFHTLS-Wide will constrain the amplitude of the matter power spectrum $\sigma_8$ to 10% and the matter density parameter $\Omega_m$ to 17%, a factor of ~2.5 improvement on the two-point analysis alone. Our error analysis includes all non-Gaussian terms, finding that the coupling between cosmic variance and shot noise is a non-negligible contribution which should be included in any future analytical error calculations.

We present the results of the GREAT08 Challenge, a blind analysis challenge to infer weak gravitational lensing shear distortions from images. The primary goal was to stimulate new ideas by presenting the problem to researchers outside the shear measurement community. Six GREAT08 Team methods were presented at the launch of the Challenge and five additional groups submitted results during the 6 month competition. Participants analyzed 30 million simulated galaxies with a range in signal to noise ratio, point-spread function ellipticity, galaxy size, and galaxy type. The large quantity of simulations allowed shear measurement methods to be assessed at a level of accuracy suitable for currently planned future cosmic shear observations for the first time. Different methods perform well in different parts of simulation parameter space and come close to the target level of accuracy in several of these. A number of fresh ideas have emerged as a result of the Challenge including a re-examination of the process of combining information from different galaxies, which reduces the dependence on realistic galaxy modelling. The image simulations will become increasingly sophisticated in future GREAT challenges, meanwhile the GREAT08 simulations remain as a benchmark for additional developments in shear measurement algorithms.

Both observations and simulations show that major tidal interactions or mergers between gas-rich galaxies can lead to intense bursts of starformation. Yet, the average enhancement in star formation rate (SFR) in major mergers and the contribution of such events to the cosmic SFR are not well estimated. Here we use photometric redshifts, stellar masses and UV SFRs from COMBO-17, 24 micron SFRs from Spitzer and morphologies from two deep HST cosmological survey fields (ECDFS/GEMS and A901/STAGES) to study the enhancement in SFR as a function of projected galaxy separation. We apply two-point projected correlation function techniques, which we augment with morphologically-selected very close pairs (separation <2 arcsec) and merger remnants from the HST imaging. Our analysis confirms that the most intensely star-forming systems are indeed interacting or merging. Yet, for massive (M* > 10^10 Msun) star-forming galaxies at 0.4<z<0.8, we find that the SFRs of galaxies undergoing a major interaction (mass ratios <1:4 and separations < 40 kpc) are only 1.80 +/- 0.30 times higher than the SFRs of non-interacting galaxies when averaged over all interactions and all stages of the interaction, in good agreement with other observational works. We demonstrate that these results imply that <10% of star formation at 0.4 < z < 0.8 is triggered directly by major mergers and interactions; these events are not important factors in the build-up of stellar mass since z=1.

The GRavitational lEnsing Accuracy Testing 2008 (GREAT08) Challenge focuses on a problem that is of crucial importance for future observations in cosmology. The shapes of distant galaxies can be used to determine the properties of dark energy and the nature of gravity, because light from those galaxies is bent by gravity from the intervening dark matter. The observed galaxy images appear distorted, although only slightly, and their shapes must be precisely disentangled from the effects of pixelisation, convolution and noise. The worldwide gravitational lensing community has made significant progress in techniques to measure these distortions via the Shear TEsting Program (STEP). Via STEP, we have run challenges within our own community, and come to recognise that this particular image analysis problem is ideally matched to experts in statistical inference, inverse problems and computational learning. Thus, in order to continue the progress seen in recent years, we are seeking an infusion of new ideas from these communities. This document details the GREAT08 Challenge for potential participants. Please visit http://www.great08challenge.info for the latest information.

Galaxies migrate from the blue cloud to the red sequence when their star formation is quenched. Here, we report on galaxies quenched by environmental effects and not by mergers or strong AGN as often invoked: They form stars at a reduced rate which is optically even less conspicuous, and manifest a transition population of blue spirals evolving into S0 galaxies. These 'optically passive' or 'red spirals' are found in large numbers in the STAGES project (and by Galaxy Zoo) in the infall region of clusters and groups.

The precision study of dark matter using weak lensing by large scale structure is strongly constrained by the accuracy with which one can measure galaxy shapes. Several methods have been devised but none have demonstrated the ability to reach the level of precision required by future weak lensing surveys. In this Letter we explore new avenues to the existing Shapelets approach, combining a priori knowledge of the galaxy profile with the power of orthogonal basis function decomposition. This Letter discusses the new issues raised by this matched filter approach and proposes promising alternatives to shape measurement techniques. In particular it appears that the use of a matched filter (e.g. Sersic profile) restricted to elliptical radial fitting functions resolves several well known Shapelet issues.

We present a study of bar and host disk evolution in a dense cluster environment, based on a sample of ~800 bright (MV <= -18) galaxies in the Abell 901/2 supercluster at z~0.165. We use HST ACS F606W imaging from the STAGES survey, and data from Spitzer, XMM-Newton, and COMBO-17. We identify and characterize bars through ellipse-fitting, and other morphological features through visual classification. (1) We explore three commonly used methods for selecting disk galaxies. We find 625, 485, and 353 disk galaxies, respectively, via visual classification, a single component S'ersic cut (n <= 2.5), and a blue-cloud cut. In cluster environments, the latter two methods miss 31% and 51%, respectively, of visually-identified disks. (2) For moderately inclined disks, the three methods of disk selection yield a similar global optical bar fraction (f_bar-opt) of 34% +10%/-3%, 31% +10%/-3%, and 30% +10%/-3%, respectively. (3) f_bar-opt rises in brighter galaxies and those which appear to have no significant bulge component. Within a given absolute magnitude bin, f_bar-opt is higher in visually-selected disk galaxies that have no bulge as opposed to those with bulges. For a given morphological class, f_bar-opt rises at higher luminosities. (4) For bright early-types, as well as faint late-type systems with no evident bulge, the optical bar fraction in the Abell 901/2 clusters is comparable within a factor of 1.1 to 1.4 to that of field galaxies at lower redshifts (5) Between the core and the virial radius of the cluster at intermediate environmental densities, the optical bar fraction does not appear to depend strongly on the local environment density and varies at most by a factor of ~1.3. We discuss the implications of our results for the evolution of bars and disks in dense environments.

We perform a comprehensive estimate of the frequency of galaxy mergers and their impact on star formation over z~0.24--0.80 (lookback time T_b~3--7 Gyr) using 3698 (M*>=1e9 Msun) galaxies with GEMS HST, COMBO-17, and Spitzer data. Our results are: (1) Among 790 high mass (M*>=2.5e10 Msun) galaxies, the visually-based merger fraction over z~0.24--0.80, ranges from 9%+-5% to 8%+-2%. Lower limits on the major and minor merger fractions over this interval range from 1.1% to 3.5%, and 3.6% to 7.5%, respectively. This is the first approximate empirical estimate of the frequency of minor mergers at z<1. For a visibility timescale of ~0.5 Gyr, it follows that over T_b~3--7 Gyr, ~68% of high mass systems have undergone a merger of mass ratio >1/10, with ~16%, 45%, and 7% of these corresponding respectively to major, minor, and ambiguous `major or minor' mergers. The mean merger rate is a few x 1e-4 Gyr-1 Mpc-3. (2) We compare the empirical merger fraction and rate for high mass galaxies to a suite of Lambda CDM-based models: halo occupation distribution models, semi-analytic models, and hydrodynamic SPH simulations. We find qualitative agreement between observations and models such that the (major+minor) merger fraction or rate from different models bracket the observations, and show a factor of five dispersion. Near-future improvements can now start to rule out certain merger scenarios. (3) Among ~3698 M*>=1e9 Msun galaxies, we find that the mean SFR of visibly merging systems is only modestly enhanced compared to non-interacting galaxies over z~0.24--0.80. Visibly merging systems only account for less than 30% of the cosmic SFR density over T_b~3--7 Gyr. This suggests that the behavior of the cosmic SFR density over the last 7 Gyr is predominantly shaped by non-interacting galaxies.

We explore the amount of obscured star-formation as a function of environment in the A901/902 supercluster at z=0.165 in conjunction with a field sample drawn from the A901 and CDFS fields, imaged with HST as part of the STAGES and GEMS surveys. We combine the COMBO-17 near-UV/optical SED with Spitzer 24um photometry to estimate both the unobscured and obscured star formation in galaxies with Mstar>10^{10}Msun. We find that the star formation activity in massive galaxies is suppressed in dense environments, in agreement with previous studies. Yet, nearly 40% of the star-forming galaxies have red optical colors at intermediate and high densities. These red systems are not starbursting; they have star formation rates per unit stellar mass similar to or lower than blue star-forming galaxies. More than half of the red star-forming galaxies have low IR-to-UV luminosity ratios, relatively high Sersic indices and they are equally abundant at all densities. They might be gradually quenching their star-formation, possibly but not necessarily under the influence of gas-removing environmental processes. The other >40% of the red star-forming galaxies have high IR-to-UV luminosity ratios, indicative of high dust obscuration. They have relatively high specific star formation rates and are more abundant at intermediate densities. Our results indicate that while there is an overall suppression in the star-forming galaxy fraction with density, the small amount of star formation surviving the cluster environment is to a large extent obscured, suggesting that environmental interactions trigger a phase of obscured star formation, before complete quenching.

The current methods available to estimate gravitational shear from astronomical images of galaxies introduce systematic errors which can affect the accuracy of weak lensing cosmological constraints. We study the impact of KSB shape measurement bias on the cosmological interpretation of tomographic two-point weak lensing shear statistics. We use a set of realistic image simulations produced by the STEP collaboration to derive shape measurement bias as a function of redshift. We define biased two-point weak lensing statistics and perform a likelihood analysis for two fiducial surveys. We present a derivation of the covariance matrix for tomography in real space and a fitting formula to calibrate it for non-Gaussianity. We find the biased aperture mass dispersion is reduced by ~20% at redshift ~1, and has a shallower scaling with redshift. This effect, if ignored in data analyses, biases sigma_8 and w_0 estimates by a few percent. The power of tomography is significantly reduced when marginalising over a range of realistic shape measurement biases. For a CFHTLS-Wide-like survey, [Omega_m, sigma_8] confidence regions are degraded by a factor of 2, whereas for a KIDS-like survey the factor is 3.5. Our results are strictly valid only for KSB methods but they demonstrate the need to marginalise over a redshift-dependent shape measurement bias in all future cosmological analyses.

We present an overview of the Space Telescope A901/2 Galaxy Evolution Survey (STAGES). STAGES is a multiwavelength project designed to probe physical drivers of galaxy evolution across a wide range of environments and luminosity. A complex multi-cluster system at z~0.165 has been the subject of an 80-orbit F606W HST/ACS mosaic covering the full 0.5x0.5 (~5x5 Mpc^2) span of the supercluster. Extensive multiwavelength observations with XMM-Newton, GALEX, Spitzer, 2dF, GMRT, and the 17-band COMBO-17 photometric redshift survey complement the HST imaging. Our survey goals include simultaneously linking galaxy morphology with other observables such as age, star-formation rate, nuclear activity, and stellar mass. In addition, with the multiwavelength dataset and new high resolution mass maps from gravitational lensing, we are able to disentangle the large-scale structure of the system. By examining all aspects of environment we will be able to evaluate the relative importance of the dark matter halos, the local galaxy density, and the hot X-ray gas in driving galaxy transformation. This paper describes the HST imaging, data reduction, and creation of a master catalogue. We perform Sersic fitting on the HST images and conduct associated simulations to quantify completeness. In addition, we present the COMBO-17 photometric redshift catalogue and estimates of stellar masses and star-formation rates for this field. We define galaxy and cluster sample selection criteria which will be the basis for forthcoming science analyses, and present a compilation of notable objects in the field. Finally, we describe the further multiwavelength observations and announce public access to the data and catalogues.

We investigate the properties of optically passive spirals and dusty red galaxies in the A901/2 cluster complex at redshift ~0.17 using restframe near-UV-optical SEDs, 24 micron IR data and HST morphologies from the STAGES dataset. The cluster sample is based on COMBO-17 redshifts with an rms precision of sigma_cz~2000 km/sec. We find that 'dusty red galaxies' and 'optically passive spirals' in A901/2 are largely the same phenomenon, and that they form stars at a substantial rate, which is only 4x lower than that in blue spirals at fixed mass. This star formation is more obscured than in blue galaxies and its optical signatures are weak. They appear predominantly in the stellar mass range of log M*/Msol=[10,11] where they constitute over half of the star-forming galaxies in the cluster; they are thus a vital ingredient for understanding the overall picture of star formation quenching in clusters. We find that the mean specific SFR of star-forming galaxies in the cluster is clearly lower than in the field, in contrast to the specific SFR properties of blue galaxies alone, which appear similar in cluster and field. Such a rich red spiral population is best explained if quenching is a slow process and morphological transformation is delayed even more. At log M*/Msol<10, such galaxies are rare, suggesting that their quenching is fast and accompanied by morphological change. We note, that edge-on spirals play a minor role; despite being dust-reddened they form only a small fraction of spirals independent of environment.

We investigate the impact of the observed correlation between a galaxies shape and its surrounding density field on the measurement of third order weak lensing shear statistics. Using numerical simulations, we estimate the systematic error contribution to a measurement of the third order moment of the aperture mass statistic (GGG) from three-point intrinsic ellipticity correlations (III), and the three-point coupling between the weak lensing shear experienced by distant galaxies and the shape of foreground galaxies (GGI and GII). We find that third-order weak lensing statistics are typically more strongly contaminated by these physical systematics compared to second-order shear measurements, contaminating the measured three-point signal for moderately deep surveys with a median redshift z_m ~ 0.7 by ~ 15%. It has been shown that accurate photometric redshifts will be crucial to correct for this effect, once a model and the redshift dependence of the effect can be accurately constrained. To this end we provide redshift-dependent fitting functions to our results and propose a new tool for the observational study of intrinsic galaxy alignments. For a shallow survey with z_m ~ 0.4 we find III to be an order of magnitude larger than the expected cosmological GGG shear signal. Compared to the two-point intrinsic ellipticity correlation which is similar in amplitude to the two-point shear signal at these survey depths, third order statistics therefore offer a promising new way to constrain models of intrinsic galaxy alignments. Early shallow data from the next generation of very wide weak lensing surveys will be optimal for this type of study.

We present a study of galaxies in the STAGES survey of the Abell 901/902 supercluster at z~0.165, based on HST ACS F606W, COMBO-17, Spitzer 24um, XMM-Newton X-ray, and gravitational lensing maps. We characterize galaxies with strong, externally-triggered morphological distortions and normal, relatively undisturbed galaxies, using visual classification and quantitative CAS parameters. We compare normal and distorted galaxies in terms of their frequency, distribution within the cluster, star formation properties, and relationship to dark matter (DM) or surface mass density, and intra-cluster medium (ICM) density. We report here our preliminary results.

We explore the history and impact of galaxy mergers and interactions over z~0.24 to 0.80, based on HST ACS, Combo-17, and Spitzer 24 mu data of ~4500 galaxies in the GEMS survey. Using visual and quantitative parameters,we identify galaxies with strong distortions indicative of recent strong interactions and mergers versus normal galaxies (E/S0, Sa, Sb-Sc, Sd/Irr). Our results are: (1) The observed fraction F of strongly disturbed systems among high mass (M>=2.5E10 Msun) galaxies is ~9% to 12% in every Gyr bin over z~0.24 to 0.80. The corresponding merger rate is a few times 10^-4 galaxies Gyr-1 Mpc-3. The fraction F shows fair agreement with the merger fraction of mass ratio >=1:10 predicted by several LCDM-based simulations. (2) For M>=1E9 Msun systems, the average star formation rate (SFR) of strongly disturbed systems is only modestly enhanced with respect to normal galaxies, in agreement with recent simulations. In fact, over z~0.24 to 0.80, strongly disturbed systems only account for a small fraction (<~30%) of the total SFR density. This suggests that the behaviour of the cosmic SFR density over the last seven billion years is predominantly shaped by normal galaxies.

In dense clusters, higher densities at early epochs as well as physical processes, such as ram pressure stripping and tidal interactions become important, and can have direct consequences for the evolution of bars and their host disks. To study bars and disks as a function of environment, we are using the STAGES ACS HST survey of the Abell 901/902 supercluster (z~0.165), along with earlier field studies based the SDSS and the Ohio State University Bright Spiral Galaxy Survey (OSUBSGS). We explore the limitations of traditional methods for characterizing the bar fraction, and in particular highlight uncertainties in disk galaxy selection in cluster environments. We present an alternative approach for exploring the proportion of bars, and investigate the properties of bars as a function of host galaxy color, Sersic index, stellar mass, star formation rate (SFR), specific SFR, and morphology.

We explore galaxy assembly over the last seven billion years by characterizing "normal" galaxies along the Hubble sequence, against strongly disturbed merging/interacting galaxies with the widely used CAS system of concentration (C), asymmetry (A), and 'clumpiness' (S) parameters, as well as visual classification. We analyze Hubble Space Telescope (HST) ACS images of ~4000 intermediate and high mass (> 10^9 solar masses) galaxies from the GEMS survey, one of the largest HST surveys conducted to date in two filters. We explore the effectiveness of the CAS criteria [A>S and A>~0.35] in separating normal and strongly disturbed galaxies at different redshifts, and quantify the recovery and contamination rate. We also compare the average star formation rate and the cosmic star formation rate density as a function of redshift between normal and interacting systems identified by CAS.

We present a study of galaxies in the Abell 901/902 Supercluster at z~0.165, based on HST ACS F606W, COMBO-17, Spitzer 24um, XMM-Newton X-ray, and gravitational lensing maps, as part of the STAGES survey. We characterize galaxies with strong externally-triggered morphological distortions and normal relatively undisturbed galaxies, using visual classification and quantitative CAS parameters. We compare normal and distorted galaxies in terms of their frequency, distribution within the cluster, star formation properties, and relationship to dark matter (DM) or surface mass density, and intra-cluster medium (ICM) density. We revisit the morphology density relation, which postulates a higher fraction of early type galaxies in dense environments, by considering separately galaxies with a low bulge-to-disk (B/D) ratio and a low gas content as these two parameters may not be correlated in clusters. We report here on our preliminary analysis.

We extend the Bayesian model fitting shape measurement method presented in Miller et al. (2007) and use the method to estimate the shear from the Shear TEsting Programme simulations (STEP). The method uses a fast model fitting algorithm which uses realistic galaxy profiles and analytically marginalises over the position and amplitude of the model by doing the model fitting in Fourier space. This is used to find the full posterior probability in ellipticity so that the shear can be estimated in a fully Bayesian way. The Bayesian shear estimation allows measurement bias arising from the presence of random noise to be removed. In this paper we introduce an iterative algorithm that can be used to estimate the intrinsic ellipticity prior and show that this is accurate and stable. By using the method to estimate the shear from the STEP1 simulations we find the method to have a shear bias of m ~ 0.005 and a variation in shear offset with PSF type of sigma_c ~ 0.0002. These values are smaller than for any method presented in the STEP1 publication that behaves linearly with shear. Using the method to estimate the shear from the STEP2 simulations we find than the shear bias and offset are m ~ 0.002 and c ~ -0.0007 respectively. In addition we find that the bias and offset are stable to changes in magnitude and size of the galaxies. Such biases should yield any cosmological constraints from future weak lensing surveys robust to systematic effects in shape measurement.

We present a high resolution dark matter reconstruction of the z=0.165 Abell 901/902 supercluster from a weak lensing analysis of the HST STAGES survey. We detect the four main structures of the supercluster at high significance, resolving substructure within and between the clusters. We find that the distribution of dark matter is well traced by the cluster galaxies, with the brightest cluster galaxies marking out the strongest peaks in the dark matter distribution. We also find a significant extension of the dark matter distribution of Abell 901a in the direction of an infalling X-ray group Abell 901alpha. We present mass, mass-to-light and mass-to-stellar mass ratio measurements of the structures and substructures that we detect. We find no evidence for variation of the mass-to-light and mass-to-stellar mass ratio between the different clusters. We compare our space-based lensing analysis with an earlier ground-based lensing analysis of the supercluster to demonstrate the importance of space-based imaging for future weak lensing dark matter 'observations'.

We present an exploration of weak lensing by large-scale structure in the linear regime, using the third-year (T0003) CFHTLS Wide data release. Our results place tight constraints on the scaling of the amplitude of the matter power spectrum sigma_8 with the matter density Omega_m. Spanning 57 square degrees to i'_AB = 24.5 over three independent fields, the unprecedented contiguous area of this survey permits high signal-to-noise measurements of two-point shear statistics from 1 arcmin to 4 degrees. Understanding systematic errors in our analysis is vital in interpreting the results. We therefore demonstrate the percent-level accuracy of our method using STEP simulations, an E/B-mode decomposition of the data, and the star-galaxy cross correlation function. We also present a thorough analysis of the galaxy redshift distribution using redshift data from the CFHTLS T0003 Deep fields that probe the same spatial regions as the Wide fields. We find sigma_8(Omega_m/0.25)^0.64 = 0.785+-0.043 using the aperture-mass statistic for the full range of angular scales for an assumed flat cosmology, in excellent agreement with WMAP3 constraints. The largest physical scale probed by our analysis is 85 Mpc, assuming a mean redshift of lenses of 0.5 and a LCDM cosmology. This allows for the first time to constrain cosmology using only cosmic shear measurements in the linear regime. Using only angular scales theta> 85 arcmin, we find sigma_8(Omega_m/0.25)_lin^0.53 = 0.837+-0.084, which agree with the results from our full analysis. Combining our results with data from WMAP3, we find Omega_m=0.248+-0.019 and sigma_8 = 0.771+-0.029.

We present a newly developed code that allows simulations of optical observations of galaxy fields with a variety of instruments. The code incorporates gravitational lensing effects and is targetted at simulating lensing by galaxy clusters. Our goal is to create the tools required for comparing theoretical expectations with observations to obtain a better understanding of how observational noise affects lensing applications such as mass estimates, studies on the internal properties of galaxy clusters and arc statistics. Starting from a set of input parameters, characterizing both the instruments and the observational conditions, the simulator provides a virtual observation of a patch of the sky. It includes several sources of noise such as photon-noise, sky background, seeing, and instrumental noise. Ray-tracing through simulated mass distributions accounts for gravitational lensing. Source morphologies are realistically simulated based on shapelet decompositions of galaxy images retrieved from the GOODS-ACS archive. According to their morphological class, spectral-energy-distributions are assigned to the source galaxies in order to reproduce observations of each galaxy in arbitrary photometric bands. We illustrate our techniques showing virtual observations of a galaxy-cluster core as it would be observed with the space telescope DUNE, which was recently proposed to ESA within its "Cosmic vision" programme. (Abridged)

Surveys of distant galaxies with the Hubble Space Telescope and from the ground have shown that there is only mild evolution in the relationship between radial size and stellar mass for galactic disks from z~1 to the present day. Using a sample of nearby disk-dominated galaxies from the Sloan Digital Sky Survey (SDSS), and high redshift data from the GEMS (Galaxy Evolution from Morphology and SEDs) survey, we investigate whether this result is consistent with theoretical expectations within the hierarchical paradigm of structure formation. The relationship between virial radius and mass for dark matter halos in the LCDM model evolves by about a factor of two over this interval. However, N-body simulations have shown that halos of a given mass have less centrally concentrated mass profiles at high redshift. When we compute the expected disk size-stellar mass distribution, accounting for this evolution in the internal structure of dark matter halos and the adiabatic contraction of the dark matter by the self-gravity of the collapsing baryons, we find that the predicted evolution in the mean size at fixed stellar mass since z~1 is about 15-20 percent, in good agreement with the observational constraints from GEMS. At redshift z~2, the model predicts that disks at fixed stellar mass were on average only 60% as large as they are today. Similarly, we predict that the rotation velocity at a given stellar mass (essentially the zero-point of the Tully-Fisher relation) is only about 10 percent larger at z~1 (20 percent at z~2) than at the present day.

We present a cosmic shear analysis of the 100 square degree weak lensing survey, combining data from the CFHTLS-Wide, RCS, VIRMOS-DESCART and GaBoDS surveys. Spanning ~100 square degrees, with a median source redshift z~0.78, this combined survey allows us to place tight joint constraints on the matter density parameter Omega_m, and the amplitude of the matter power spectrum sigma_8, finding sigma_8*(Omega_m/0.24)^0.59 = 0.84+/-0.05. Tables of the measured shear correlation function and the calculated covariance matrix for each survey are included. The accuracy of our results is a marked improvement on previous work owing to three important differences in our analysis; we correctly account for cosmic variance errors by including a non-Gaussian contribution estimated from numerical simulations; we correct the measured shear for a calibration bias as estimated from simulated data; we model the redshift distribution, n(z), of each survey from the largest deep photometric redshift catalogue currently available from the CFHTLS-Deep. This catalogue is randomly sampled to reproduce the magnitude distribution of each survey with the resulting survey dependent n(z) parametrised using two different models. While our results are consistent for the n(z) models tested, we find that our cosmological parameter constraints depend weakly (at the 5% level) on the inclusion or exclusion of galaxies with low confidence photometric redshift estimates (z>1.5). These high redshift galaxies are relatively few in number but contribute a significant weak lensing signal. It will therefore be important for future weak lensing surveys to obtain near-infra-red data to reliably determine the number of high redshift galaxies in cosmic shear analyses.

The principles of measuring the shapes of galaxies by a model-fitting approach are discussed in the context of shape-measurement for surveys of weak gravitational lensing. It is argued that such an approach should be optimal, allowing measurement with maximal signal-to-noise, coupled with estimation of measurement errors. The distinction between likelihood-based and Bayesian methods is discussed. Systematic biases in the Bayesian method may be evaluated as part of the fitting process, and overall such an approach should yield unbiased shear estimation without requiring external calibration from simulations. The principal disadvantage of model-fitting for large surveys is the computational time required, but here an algorithm is presented that enables large surveys to be analysed in feasible computation times. The method and algorithm is tested on simulated galaxies from the Shear TEsting Program (STEP).

This is the first paper of a series describing our measurement of weak lensing by large-scale structure using archival observations from the Advanced Camera for Surveys (ACS) on board the Hubble Space Telescope (HST). In this work we present results from a pilot study testing the capabilities of the ACS for cosmic shear measurements with early parallel observations and presenting a re-analysis of HST/ACS data from the GEMS survey and the GOODS observations of the Chandra Deep Field South (CDFS). We describe our new correction scheme for the time-dependent ACS PSF based on observations of stellar fields. This is currently the only technique which takes the full time variation of the PSF between individual ACS exposures into account. We estimate that our PSF correction scheme reduces the systematic contribution to the shear correlation functions due to PSF distortions to < 2*10^{-6} for galaxy fields containing at least 10 stars. We perform a number of diagnostic tests indicating that the remaining level of systematics is consistent with zero for the GEMS and GOODS data confirming the success of our PSF correction scheme. For the parallel data we detect a low level of remaining systematics which we interpret to be caused by a lack of sufficient dithering of the data. Combining the shear estimate of the GEMS and GOODS observations using 96 galaxies arcmin^{-2} with the photometric redshift catalogue of the GOODS-MUSIC sample, we determine a local single field estimate for the mass power spectrum normalisation sigma_{8,CDFS}=0.52^{+0.11}_{-0.15} (stat) +/- 0.07 (sys) (68% confidence assuming Gaussian cosmic variance) at fixed Omega_m=0.3 for a LambdaCDM cosmology. We interpret this exceptionally low estimate to be due to a local under-density of the foreground structures in the CDFS.

In the context of measuring structure and morphology of intermediate redshift galaxies with recent HST/ACS surveys, we tune, test, and compare two widely used fitting codes (GALFIT and GIM2D) for fitting single-component Sersic models to the light profiles of both simulated and real galaxy data. We find that fitting accuracy depends sensitively on galaxy profile shape. Exponential disks are well fit with Sersic models and have small measurement errors, whereas fits to de Vaucouleurs profiles show larger uncertainties owing to the large amount of light at large radii. We find that both codes provide reliable fits and little systematic error, when the effective surface brightness is above that of the sky. Moreover, both codes return errors that significantly underestimate the true fitting uncertainties, which are best estimated with simulations. We find that GIM2D suffers significant systematic errors for spheroids with close companions owing to the difficulty of effectively masking out neighboring galaxy light; there appears to be no work around to this important systematic in GIM2D's current implementation. While this crowding error affects only a small fraction of galaxies in GEMS, it must be accounted for in the analysis of deeper cosmological images or of more crowded fields with GIM2D. In contrast, GALFIT results are robust to the presence of neighbors because it can simultaneously fit the profiles of multiple companions thereby deblending their effect on the fit to the galaxy of interest. We find GALFIT's robustness to nearby companions and factor of >~20 faster runtime speed are important advantages over GIM2D for analyzing large HST/ACS datasets. Finally we include our final catalog of fit results for all 41,495 objects detected in GEMS.

With a primary goal of conducting precision weak lensing measurements from space, the COSMOS survey has imaged the largest contiguous area observed by the Hubble Space Telescope (HST) to date using the Advanced Camera for Surveys (ACS). This is the first paper in a series where we describe our strategy for addressing the various technical challenges in the production of weak lensing measurements from the COSMOS data. The COSMOS ACS catalog is constructed from 575 ACS/WFC tiles (1.64 deg^2) and contains a total 1.2x10^6 objects to a limiting magnitude of F814W=26.5. This catalog is made publicly available. The shapes of galaxies have been measured and corrected for the distortion induced by the time varying ACS Point Spread Function and for Charge Transfer Efficiency effects. Next, simulated images are used to derive the shear susceptibility factors that are necessary in order to transform shape measurements into unbiased shear estimators. Finally, for each galaxy, we derive a shape measurement error and utilize this quantity to extract the intrinsic shape noise of the galaxy sample. Interestingly, our results indicate that the intrinsic shape noise varies little with either size, magnitude or redshift. Representing a number density of 66 galaxies per arcmin^2, the final COSMOS weak lensing catalog contains 3.9x10^5 galaxies with accurate shape measurements. The properties of the COSMOS weak lensing catalog described throughout this paper will provide key input numbers for the preparation and design of next-generation wide field space missions.

(abridged) We examine the spatial and temporal stability of the HST ACS Wide Field Camera (WFC) point spread function (PSF) using the two square degree COSMOS survey. We show that stochastic aliasing of the PSF necessarily occurs during `drizzling'. This aliasing is maximal if the output pixel scale is equal to the input pixel scale of 0.05''. We show that this source of PSF variation can be significantly reduced by choosing a Gaussian drizzle kernel and by setting the output pixel size to 0.03''. We show that the PSF is temporally unstable, most likely due to thermal fluctuations in the telescope's focus. We find that the primary manifestation of this thermal drift in COSMOS images is an overall slow periodic focus change. Using a modified version of TinyTim, we create undistorted stars in a 30x30 grid across the ACS WFC CCDs. These PSF models are created for telescope focus values in the range -10 microns to +5 microns, thus spanning the allowed range of telescope focus values. We then use the approximately ten well measured stars in each COSMOS field to pick the best-fit focus value for each field. The TinyTim model stars are then used to perform PSF corrections for weak lensing allowing systematics due to incorrectly modeled PSFs to be greatly reduced. We have made the software for PSF modeling using our modified version of TinyTim available to the astronomical community. We show the effects of Charge Transfer Efficiency (CTE) degradation, which distorts the object in the readout direction, mimicking a weak lensing signal. We derive a parametric correction for the effect of CTE on the shapes of objects in the COSMOS field as a function of observation date, position within the ACS WFC field, and object flux.

We present a three dimensional cosmic shear analysis of the Hubble Space Telescope COSMOS survey, the largest ever optical imaging program performed in space. We have measured the shapes of galaxies for the tell-tale distortions caused by weak gravitational lensing, and traced the growth of that signal as a function of redshift. Using both 2D and 3D analyses, we measure cosmological parameters Omega_m, the density of matter in the universe, and sigma_8, the normalization of the matter power spectrum. The introduction of redshift information tightens the constraints by a factor of three, and also reduces the relative sampling (or "cosmic") variance compared to recent surveys that may be larger but are only two dimensional. From the 3D analysis, we find sigma_8*(Omega_m/0.3)^0.44=0.866+^0.085_-0.068 at 68% confidence limits, including both statistical and potential systematic sources of error in the total budget. Indeed, the absolute calibration of shear measurement methods is now the dominant source of uncertainty. Assuming instead a baseline cosmology to fix the geometry of the universe, we have measured the growth of structure on both linear and non-linear physical scales. Our results thus demonstrate a proof of concept for tomographic analysis techniques that have been proposed for future weak lensing surveys by a dedicated wide-field telescope in space.

The results from weak gravitational lensing analyses are subject to a cosmic variance error term that has previously been estimated assuming Gaussian statistics. In this letter we address the issue of estimating cosmic variance errors for weak lensing surveys in the non-Gaussian regime. Using standard cold dark matter model ray-tracing simulations characterized by Omega_m=0.3, Omega_Lambda=0.7, h=0.7, sigma_8=1.0 for different survey redshifts z_s, we determine the variance of the two-point shear correlation function measured across 64 independent lines of sight. We compare the measured variance to the variance expected from a random Gaussian field and derive a redshift-dependent non-Gaussian calibration relation. We find that the ratio can be as high as ~30 for a survey with source redshift z_s ~ 0.5 and ~10 for z_s ~ 1. The transition scale theta_c above which the ratio is consistent with unity, is found to be theta_c ~ 20 arcmin for z_s ~ 0.5 and theta_c ~ 10 arcmin for z_s ~ 1. We provide fitting formula to our results permitting the estimation of non-Gaussian cosmic variance errors for any weak lensing analysis, and discuss the impact on current and future surveys. A more extensive set of simulations will however be required to investigate the dependence of our results on cosmology, specifically on the amplitude of clustering.

The Shear TEsting Programme (STEP) is a collaborative project to improve the accuracy and reliability of weak lensing measurement, in preparation for the next generation of wide-field surveys. We review sixteen current and emerging shear measurement methods in a common language, and assess their performance by running them (blindly) on simulated images that contain a known shear signal. We determine the common features of algorithms that most successfully recover the input parameters. We achieve previously unattained discriminatory precision in our analysis, via a combination of more extensive simulations, and pairs of galaxy images that have been rotated with respect to each other, thus removing noise from their intrinsic ellipticities. The robustness of our simulation approach is also confirmed by testing the relative calibration of methods on real data. Weak lensing measurement has improved since the first STEP paper. Several methods now consistently achieve better than 2% precision, and are still being developed. However, the simulations can now distinguish all methods from perfect performance. Our main concern continues to be the potential for a multiplicative shear calibration bias: not least because this can not be internally calibrated with real data. We determine which galaxy populations are responsible and, by adjusting the simulated observing conditions, we also investigate the effects of instrumental and atmospheric parameters. We have isolated several previously unrecognised aspects of galaxy shape measurement, in which focussed development could provide further progress towards the sub-percent level of precision desired for future surveys. [ABRIDGED]

The cosmological interpretation of weak lensing by large-scale structures requires knowledge of the redshift distribution of the source galaxies. Current lensing surveys are often calibrated using external redshift samples which span a significantly smaller sky area in comparison to the lensing survey, and are thus subject to sample variance. Some future lensing surveys are expected to be calibrated in the same way, in particular the fainter galaxy populations where the entire color coverage, and hence photometric redshift estimate, could be challenging to obtain. Using N-body simulations, we study the impact of this sample variance on cosmic shear analysis and show that, to first approximation, it behaves like a shear calibration error 1+/-epsilon. Using the Hubble Deep Field as a redshift calibration survey could therefore be a problem for current lensing surveys. We discuss the impact of the redshift distribution sampling error and a shear calibration error on the design of future lensing surveys, and find that a lensing survey of area Theta square degrees and limiting magnitude m_lim}, has a minimum shear and redshift calibration accuracy requirements given by epsilon = epsilon_0 10^{beta(m_lim-24.5)} / sqrt(Theta/ 200). Above that limit, lensing surveys would not reach their full potential. Using the galaxy number counts from the Hubble Ultra-Deep Field, we find (epsilon_0,beta)=(0.015,-0.18) and (epsilon_0,beta)=(0.011,-0.23) for ground and space based surveys respectively. Lensing surveys with no or limited redshift information and/or poor shear calibration accuracy will loose their potential to analyse the cosmic shear signal in the sub-degree angular scales, and therefore complete photometric redshift coverage should be a top priority for future lensing surveys.

We present constraints on the evolution of the virial to stellar mass ratio of galaxies with high stellar masses in the redshift range 0.2<z<0.8, by comparing weak lensing measurements of virial mass M_vir to estimates of stellar mass M_star from COMBO-17. For a complete sample of galaxies with log(M_star/ M_\odot) > 10.5, where the majority show an early-type morphology, we find that the virial mass to stellar mass ratio is given by M_vir/M_star = 53^{+13}_{-16}. Assuming a baryon fraction from the concordance cosmology, this corresponds to a stellar fraction of baryons in massive galaxies of Omega_b^*/\Omega_b = 0.10 +/- 0.03. Analysing the galaxy sample in different redshift slices, we find little or no evolution in the virial to stellar mass ratio, and place an upper limit of ~2.5 on the growth of massive galaxies through the conversion of gas into stars from z=0.8 to the present day.

We investigate the expected correlation between the weak gravitational shear of distant galaxies and the orientation of foreground galaxies, through the use of numerical simulations. This shear-ellipticity correlation can mimic a cosmological weak lensing signal, and is potentially the limiting physical systematic effect for cosmology with future high-precision weak lensing surveys. We find that, if uncorrected, the shear-ellipticity correlation could contribute up to 10% of the weak lensing signal on scales up to 20 arcminutes, for lensing surveys with a median depth z=1. The most massive foreground galaxies are expected to cause the largest correlations, a result also seen in the Sloan Digital Sky Survey. We find that the redshift dependence of the effect is proportional to the lensing efficiency of the foreground, and this offers prospects for removal to high precision, although with some model dependence. The contamination is characterised by a weakly negative B-mode, which can be used as a diagnostic of systematic errors. We also provide more accurate predictions for a second potential source of error, the intrinsic alignment of nearby galaxies. This source of contamination is less important, however, as it can be easily removed with distance information.

The Shear TEsting Programme, STEP, is a collaborative project to improve the accuracy and reliability of all weak lensing measurements in preparation for the next generation of wide-field surveys. In this first STEP paper we present the results of a blind analysis of simulated ground-based observations of relatively simple galaxy morphologies. The most successful methods are shown to achieve percent level accuracy. From the cosmic shear pipelines that have been used to constrain cosmology, we find weak lensing shear measured to an accuracy that is within the statistical errors of current weak lensing analyses, with shear measurements accurate to better than 7%. The dominant source of measurement error is shown to arise from calibration uncertainties where the measured shear is over or under-estimated by a constant multiplicative factor. This is of concern as calibration errors cannot be detected through standard diagnostic tests. The measured calibration errors appear to result from stellar contamination, false object detection, the shear measurement method itself, selection bias and/or the use of biased weights. Additive systematics (false detections of shear) resulting from residual point-spread function anisotropy are, in most cases, reduced to below an equivalent shear of 0.001, an order of magnitude below cosmic shear distortions on the scales probed by current surveys. Our results provide a snapshot view of the accuracy of current ground-based weak lensing methods and a benchmark upon which we can improve. To this end we provide descriptions of each method tested and include details of the eight different implementations of the commonly used Kaiser, Squires and Broadhurst (1995) method (KSB+) to aid the improvement of future KSB+ analyses.

We have used the 28'x 28' HST image mosaic from the GEMS (Galaxy Evolution from Morphology and SEDs) survey in conjunction with the COMBO-17 photometric redshift survey to constrain the incidence of major mergers between spheroid-dominated galaxies with little cold gas (dry mergers) since z = 0.7. A set of N-body merger simulations was used to explore the morphological signatures of such interactions: they are recognizable either as < 5kpc separation close pairs or because of broad, low surface brightness tidal features and asymmetries. Data with the depth and resolution of GEMS are sensitive to dry mergers between galaxies with M_V < -20.5 for z < 0.7; dry mergers at higher redshifts are not easily recovered in single-orbit HST imaging. Six dry mergers (12 galaxies) with luminosity ratios between 1:1 and 4:1 were found from a sample of 379 red early-type galaxies with M_V < -20.5 and 0.1 < z < 0.7. The simulations suggest that the morphological signatures of dry merging are visible for ~250Myr and we use this timescale to convert the observed merger incidence into a rate. On this basis we find that present day spheroidal galaxies with M_V < -20.5 on average have undergone between 0.5 and 2 major dry mergers since z ~ 0.7. We have compared this result with the predictions of a Cold Dark Matter based semi-analytic galaxy formation model. The model reproduces the observed declining major merger fraction of bright galaxies and the space density of luminous early-type galaxies reasonably well. The predicted dry merger fraction is consistent with our observational result. Hence, hierarchical models predict and observations now show that major dry mergers are an important driver of the evolution of massive early-type galaxies in recent epochs.

We describe the data reduction and object cataloging for the GEMS survey, a large-area (800 arcmin^2) two-band (F606W and F850LP) imaging survey with the Advanced Camera for Surveys on HST, centered on the Chandra Deep Field South.

We present our cosmic shear analysis of GEMS, one of the largest wide-field surveys ever undertaken by the Hubble Space Telescope. Imaged with the Advanced Camera for Surveys (ACS), GEMS spans 795 square arcmin in the Chandra Deep Field South. We detect weak lensing by large-scale structure in high resolution F606W GEMS data from ~60 resolved galaxies per square arcminute. We measure the two-point shear correlation function, the top-hat shear variance and the shear power spectrum, performing an E/B mode decomposition for each statistic. We show that we are not limited by systematic errors and use our results to place joint constraints on the matter density parameter Omega_m and the amplitude of the matter power spectrum sigma_8. We find sigma_8(Omega_m/0.3)^{0.65}=0.68 +/- 0.13 where the 1sigma error includes both our uncertainty on the median redshift of the survey and sampling variance. Removing image and point spread function (PSF) distortions are crucial to all weak lensing analyses. We therefore include a thorough discussion on the degree of ACS PSF distortion and anisotropy which we characterise directly from GEMS data. Consecutively imaged over 20 days, GEMS data also allows us to investigate PSF instability over time. We find that, even in the relatively short GEMS observing period, the ACS PSF ellipticity varies at the level of a few percent which we account for with a semi-time dependent PSF model. Our correction for the temporal and spatial variability of the PSF is shown to be successful through a series of diagnostic tests.

We present measurements of the clustering properties of bright ($L>L_{*}$) z$\sim$4 Lyman Break Galaxies (LBGs) selected from the Oxford-Dartmouth Thirty Degree Survey (ODT). We describe techniques used to select and evaluate our candidates and calculate the angular correlation function which we find best fitted by a power law, $\omega(\theta)=A_{w}\theta^{-\beta}$ with $A_{w}=15.4$ (with $\theta$ in arcseconds), using a constrained slope of $\beta=0.8$. Using a redshift distribution consistent with photometric models, we deproject this correlation function and find a comoving $r_{0}=11.4_{-1.9}^{+1.7}$ h$_{100}^{-1}$ Mpc in a $\Omega_m=0.3$ flat $\Lambda$ cosmology for $i_{AB}\leq24.5$. This corresponds to a linear bias value of $b=8.1_{-2.6}^{+2.0}$ (assuming $\sigma_{8}=0.9$). These data show a significantly larger $r_{0}$ and $b$ than previous studies at $z\sim4$. We interpret this as evidence that the brightest LBGs have a larger bias than fainter ones, indicating a strong luminosity dependence for the measured bias of an LBG sample. Comparing this against recent results in the literature at fainter (sub-$L_{*}$) limiting magnitudes, and with simple models describing the relationship between LBGs and dark matter haloes, we discuss the implications on the implied environments and nature of LBGs. It seems that the brightest LBGs (in contrast with the majority sub-$L_{*}$ population), have clustering properties, and host dark matter halo masses, that are consistent with them being progenitors of the most massive galaxies today.

We combine high-resolution images from GEMS with redshifts and spectral energy distributions from COMBO-17 to explore the morphological types of galaxies that dominate the z~0.7 UV luminosity density. We analysed rest-frame 280 nm and V-band luminosities of 1483 galaxies with 0.65<z< 0.75, combining these with visual morphological classifications from F850LP images (approximately rest-frame V-band) taken with HST/ACS on the Extended Chandra Deep Field South. We derive UV luminosity functions and j_280 luminosity densities for spheroid-dominated galaxies, spiral galaxies, Magellanic irregulars, and clearly-interacting galaxies with morphologies suggestive of ongoing major mergers. We check the reliability of GEMS morphologies against the deeper GOODS images and quantify an incompleteness of the GEMS merger identification at the faint end. We derive the fractions of the global UV luminosity density j_280 originating from the galaxy types, and find that spiral galaxies and Magellanic irregulars dominate with about 40% each. Interacting and merging galaxies account for roughly 20% of j_280, while the contribution of early types is negligible. These results imply that the strong decline in the UV luminosity density of the Universe observed from z~1 until today is dominated by the decreasing UV luminosities of normal spiral galaxies, accompanied by the migration of UV-luminous star formation in irregular galaxies to systems of progressively lower mass and luminosity. (abridged)

We combine HST imaging from the GEMS survey with photometric redshifts from COMBO-17 to explore the evolution of disk-dominated galaxies since z<1.1. The sample is comprised of all GEMS galaxies with Sersic indices n<2.5, derived from fits to the galaxy images. We account fully for selection effects through careful analysis of image simulations; we are limited by the depth of the redshift and HST data to the study of galaxies with absolute magnitudes M(V)<-20, or equivalently stellar masses log(M)>10. We find strong evolution in the magnitude-size scaling relation for galaxies with M(V)<-20, corresponding to a brightening of 1 mag per sqarcsec in rest-frame V-band by z=1. Yet, disks at a given absolute magnitude are bluer and have lower stellar mass-to-light ratios at z=1 than at the present day. As a result, our findings indicate weak or no evolution in the relation between stellar mass and effective disk size for galaxies with log(M)>10 over the same time interval. This is strongly inconsistent with the most naive theoretical expectation, in which disk size scales in proportion to the halo virial radius, which would predict that disks are a factor of two denser at fixed mass at z=1. The lack of evolution in the stellar mass-size relation is consistent with an ``inside-out'' growth of galaxy disks on average (galaxies increasing in size as they grow more massive), although we cannot rule out more complex evolutionary scenarios.

We combine HST/ACS imaging from the GEMS survey with redshifts and rest-frame quantities from COMBO-17 to study the evolution of morphologically early-type galaxies with red colors since z=1. We use a new large sample of 728 galaxies with centrally-concentrated radial profiles (Sersic n>2.5) and rest-frame U-V colors on the red sequence. By appropriate comparison with the local relations from SDSS, we find that the luminosity-size (L-R) and stellar mass-size (M-R) relations evolve in a manner that is consistent with the passive aging of ancient stars. By itself, this result is consistent with a completely passive evolution of the red early-type galaxy population. If instead, as demonstrated by a number of recent surveys, the early-type galaxy population builds up in mass by a factor of 2 since z=1, our results imply that new additions to the early-type galaxy population follow similar L-R and M-R correlations, compared to the older subset of early-type galaxies. Adding early-type galaxies to the red sequence through disk fading appears to be consistent with the data. Through comparison with models, the role of dissipationless merging is limited to <1 major merger on average since z=1 for the most massive galaxies. Predictions from models of gas-rich mergers are not yet mature enough to allow a detailed comparison to our observations. We find tentative evidence that the amount of luminosity evolution depends on galaxy stellar mass, such that the least massive galaxies show stronger luminosity evolution compared to more massive early types. This could reflect a different origin of low-mass early-type galaxies and/or younger stellar populations; the present data is insufficient to discriminate between these possibilities. (abridged)

One third of present-day spirals host optically visible strong bars that drive their dynamical evolution. However, the fundamental question of how bars evolve over cosmological times has yet to be addressed, and even the frequency of bars at intermediate redshifts remains controversial. We investigate the frequency of bars out to z~1.0 drawing on a sample of 1590 galaxies from the GEMS survey, which provides morphologies from HST ACS two-color images, and highly accurate redshifts from the COMBO-17 survey. We identify spiral galaxies using the Sersic index, concentration parameter, and rest-frame color. We characterize bars and disks by fitting ellipses to F606W and F850LP images, taking advantage of the two bands to minimize bandpass shifting. We exclude highly inclined (i>60 deg) galaxies to ensure reliable morphological classifications, and apply completeness cuts of M_v <= -19.3 and -20.6. More than 40% of the bars that we detect have semi major axes a<0.5" and would be easily missed in earlier surveys without the small PSF of ACS. The bars that we can reliably detect are fairly strong (with ellipticities e>=0.4) and have a in the range ~1.2-13 kpc. We find that the optical fraction of such strong bars remains at ~(30% +- 6%) from the present-day out to look-back times of 2-6 Gyr (z~0.2-0.7) and 6-8 Gyr (z~0.7-1.0); it certainly shows no sign of a drastic decline at z>0.7. Our findings of a large and similar bar fraction at these three epochs favor scenarios in which cold gravitationally unstable disks are already in place by z~1, and where on average bars have a long lifetime (well above 2 Gyr). The distributions of structural bar properties in the two slices are, however, not statistically identical and therefore allow for the possibility that the bar strengths and sizes may evolve over time.

Bars drive the dynamical evolution of disk galaxies by redistributing mass and angular momentum, and they are ubiquitous in present-day spirals. Early studies of the Hubble Deep Field reported a dramatic decline in the rest-frame optical bar fraction f_opt to below 5% at redshifts z>0.7, implying that disks at these epochs are fundamentally different from present-day spirals. The GEMS bar project, based on ~8300 galaxies with HST-based morphologies and accurate redshifts over the range 0.2-1.1, aims at constraining the evolution and impact of bars over the last 8 Gyr. We present early results indicating that f_opt remains nearly constant at ~30% over the range z=0.2-1.1,corresponding to lookback times of ~2.5-8 Gyr. The bars detected at z>0.6 are primarily strong with ellipticities of 0.4-0.8. Remarkably, the bar fraction and range of bar sizes observed at z>0.6 appear to be comparable to the values measured in the local Universe for bars of corresponding strengths. Implications for bar evolution models are discussed.

The Oxford Dartmouth Thirty Degree Survey (ODTS) is a deep, wide, multi-band imaging survey designed to cover a total of 30 square degrees in BVRi'Z, with a subset of U and K band data, in four separate fields of 5-10 deg^2 centred at 00:18:24 +34:52, 09:09:45 +40:50, 13:40:00 +02:30 and 16:39:30 +45:24. Observations have been made using the Wide Field Camera on the 2.5-m Isaac Newton Telescope in La Palma to average limiting depths (5 sigma Vega, aperture magnitudes) of U=24.8, B=25.6, V=25.0, R=24.6, and i'=23.5, with observations taken in ideal conditions reaching the target depths of U=25.3, B=26.2, V=25.7, R=25.4, and i'=24.6. The INT Z band data was found to be severely effected by fringing and, consequently, is now being obtained at the MDM observatory in Arizona. A complementary K-band survey has also been carried out at MDM, reaching an average depth of K_{5\sigma}~18.5. At present, approximately 23 deg^2 of the ODTS have been observed, with 3.5 deg^2 of the K band survey completed. This paper details the survey goals, field selection, observation strategy and data reduction procedure, focusing on the photometric calibration and catalogue construction. Preliminary photometric redshifts have been obtained for a subsample of the objects with R <= 23. These results are presented alongside a brief description of the photometric redshift determination technique used. The median redshift of the survey is estimated to be z~0.7 from a combination of the ODTS photometric redshifts and comparison with the redshift distributions of other surveys. Finally, galaxy number counts for the ODTS are presented which are found to be in excellent agreement with previous studies.

We present a method for removal of the contaminating effects of intrinsic galaxy alignments, in measurements of cosmic shear from multi-colour weak lensing surveys. The method down-weights pairs which are likely to be close in three dimensions, based on spectroscopic or photometric redshifts. Results are dramatic: the intrinsic contamination of the low-redshift Sloan photometric redshift survey could be 80 times the lensing signal, but this can be almost completely removed, leaving random shot noise errors of the order of 10%. Intrinsic galaxy alignments, although an annoying contaminant for cosmic shear studies, are interesting in their own right, and we therefore present a new observational constraint for their amplitude from the aperture mass B-mode in the Red-Sequence Cluster Survey. The small measured B-mode rules out the published intrinsic alignment models from numerical simulations, which assume no evolution in galaxy clustering.

We estimate and remove the contamination of weak gravitational lensing measurements by the intrinsic alignment of close pairs of galaxies. We do this by investigating both the aperture mass B mode statistic, and the shear correlations of close and distant pairs of galaxies. We re-analyse the COMBO-17 survey, and study published results from the RCS and the VIRMOS-DESCART survey, concluding that the intrinsic alignment effect is at the lower end of the range of theoretical predictions. We also revisit this theoretical issue, and show that misalignment of baryon and halo angular momenta may be an important effect which can reduce the intrinsic ellipticity correlations estimated from numerical simulations to the level that we and the SuperCOSMOS survey observe. We re-examine the cosmological parameter estimation from the COMBO-17 survey, now marginalising over the Hubble constant. Assuming no evolution in galaxy clustering, and marginalising over the intrinsic alignment signal, we find the mass clustering amplitude is reduced by 0.03 to sigma_8(Omega_m / 0.27)^0.6 = 0.71 +/- 0.11. We consider the forthcoming SNAP wide weak lensing survey, and the CFHTLS wide synoptic survey, and expect them to be contaminated on scales >1 arcmin by intrinsic alignments at the level of ~1% and ~2% respectively. Division of the SNAP survey for lensing tomography significantly increases the contamination in the lowest redshift bin to ~7% and possibly higher. Removal of the intrinsic alignment effect by the downweighting of nearby galaxy pairs will therefore be vital for SNAP, (abridged).

Intrinsic alignments of galaxies can mimic to an extent the effects of shear caused by weak gravitational lensing. Previous studies have shown that for shallow surveys with median redshifts z_m = 0.1, the intrinsic alignment dominates the lensing signal. For deep surveys with z_m = 1, intrinsic alignments are believed to be a significant contaminant of the lensing signal, preventing high-precision measurements of the matter power spectrum. In this paper we show how distance information, either spectroscopic or photometric redshifts, can be used to down-weight nearby pairs in an optimised way, to reduce the errors in the shear signal arising from intrinsic alignments. Provided a conservatively large intrinsic alignment is assumed, the optimised weights will essentially remove all traces of contamination. For the Sloan spectroscopic galaxy sample, residual shot noise continues to render it unsuitable for weak lensing studies. However, a dramatic improvement for the slightly deeper Sloan photometric survey is found, whereby the intrinsic contribution, at angular scales greater than 1 arcminute, is reduced from about 80 times the lensing signal to a 10% effect. For deeper surveys such as the COMBO-17 survey with z_m = 0.6, the optimisation reduces the error from a largely systematic 220% error at small angular scales to a much smaller and largely statistical error of only 17% of the expected lensing signal. We therefore propose that future weak lensing surveys be accompanied by the acquisition of photometric redshifts, in order to remove fully the unknown intrinsic alignment errors from weak lensing detections.

Weak gravitational lensing is now established as a powerful method to measure mass fluctuations in the universe. It relies on the measurement of small coherent distortions of the images of background galaxies. Even low-level correlations in the intrinsic shapes of galaxies could however produce a significant spurious lensing signal. These correlations are also interesting in their own right, since their detection would constrain models of galaxy formation. Using $3\times 10^{4} - 10^5$ halos found in N-body simulations, we compute the correlation functions of the intrinsic ellipticity of spiral galaxies assuming that the disk is perpendicular to the angular momentum of the dark matter halo. We also consider a simple model for elliptical galaxies, in which the shape of the dark matter halo is assumed to be the same as that of the light. For deep lensing surveys with median redshifts $\sim 1$, we find that intrinsic correlations of $\sim 10^{-4}$ on angular scales $\theta \sim 0.1-10'$ are generally below the expected lensing signal, and contribute only a small fraction of the excess signals reported on these scales. On larger scales we find limits to the intrinsic correlation function at a level $\sim 10^{-5}$, which gives a (model-dependent) range of separations for which the intrinsic signal is about an order of magnitude below the ellipticity correlation function expected from weak lensing. Intrinsic correlations are thus negligible on these scales for dedicated weak lensing surveys. For wider but shallower surveys such as SuperCOSMOS, APM and SDSS, we cannot exclude the possibility that intrinsic correlations could dominate the lensing signal. We discuss how such surveys could be used to calibrate the importance of this effect, as well as study spin-spin correlations of spiral galaxies.

We present a determination of the optical QSO luminosity function and its cosmological evolution with redshift for a sample of over 6000 QSOs identified primarily from the first observations of the 2dF QSO Redshift Survey (2QZ). For QSOs with -26 < M_B < -23 and 0.35 < z < 2.3, we find that pure luminosity evolution (PLE) models provide an acceptable fit to the observed redshift dependence of the luminosity function. The luminosity function is best fit by a two-power-law function of the form. Exponential luminosity evolution models, both as a function of look-back time, and as a general second-order polynomial with redshift, were found to provide acceptable fits to the dataset comprising the 2QZ and the Large Bright Quasar Survey. Exponential evolution with look-back time is prefered for q_0=0.05, while the polynomial evolution model is prefered for q_0=0.5. The shape and evolution of the LF at low redshifts (z<0.5) and/or high luminosities, not currently well sampled by the 2dF QSO survey, may show departures from pure luminosity evolution, but the results presented here show that over a significant range of redshift, PLE is a good description of QSO evolution.