The Complete Calibration of the Colour-Redshift Relation survey (C3R2) is a spectroscopic effort involving ESO and Keck facilities designed to empirically calibrate the galaxy colour-redshift relation - P(z|C) to the Euclid depth (i_AB=24.5) and is intimately linked to upcoming Stage IV dark energy missions based on weak lensing cosmology. The aim is to build a spectroscopic calibration sample that is as representative as possible of the galaxies of the Euclid weak lensing sample. In order to minimise the number of spectroscopic observations to fill the gaps in current knowledge of the P(z|C), self-organising map (SOM) representations of the galaxy colour space have been constructed. Here we present the first results of an ESO@ VLT Large Programme approved in the context of C3R2, which makes use of the two VLT optical and near-infrared multi-object spectrographs, FORS2 and KMOS. This paper focuses on high-quality spectroscopic redshifts of high-z galaxies observed with the KMOS spectrograph in the H- and K-bands. A total of 424 highly-reliable z are measured in the 1.3<=z<=2.5 range, with total success rates of 60.7% in the H-band and 32.8% in the K-band. The newly determined z fill 55% of high and 35% of lower priority empty SOM grid cells. We measured Halpha fluxes in a 1."2 radius aperture from the spectra of the spectroscopically confirmed galaxies and converted them into star formation rates. In addition, we performed an SED fitting analysis on the same sample in order to derive stellar masses, E(B-V), total magnitudes, and SFRs. We combine the results obtained from the spectra with those derived via SED fitting, and we show that the spectroscopic failures come from either weakly star-forming galaxies (at z<1.7, i.e. in the H-band) or low S/N spectra (in the K-band) of z>2 galaxies.

We present the methodology for a joint cosmological analysis of weak gravitational lensing from the fourth data release of the ESO Kilo-Degree Survey (KiDS-1000) and galaxy clustering from the partially overlapping BOSS and 2dFLenS surveys. Cross-correlations between galaxy positions and ellipticities are incorporated into the analysis, developing a hybrid model of non-linear scales that blends perturbative and non-perturbative approaches, and assessing signal contributions by astrophysical effects. All weak lensing signals are measured consistently via Fourier-space statistics that are insensitive to the survey mask and display low levels of mode mixing. The calibration of photometric redshift distributions and multiplicative gravitational shear bias is updated, and a more complete tally of residual calibration uncertainties is propagated into the likelihood. A dedicated suite of more than 20000 mocks is used to assess the performance of covariance models and to quantify the impact of survey geometry and spatial variations of survey depth on signals and their errors. The sampling distributions for the likelihood and the $\chi^2$ goodness-of-fit statistic are validated, with proposed changes to calculating the effective number of degrees of freedom. Standard weak lensing point estimates on $S_8=\sigma_8\,(\Omega_{\rm m}/0.3)^{1/2}$ derived from its marginal posterior are easily misinterpreted to be biased low, and an alternative estimator and associated credible interval are proposed. Known systematic effects pertaining to weak lensing modelling and inference are shown to bias $S_8$ by no more than 0.1 standard deviations, with the caveat that no conclusive validation data exist for models of intrinsic galaxy alignments. Compared to the previous KiDS analyses, $S_8$ constraints are expected to improve by 20% for weak lensing alone and by 29% for the joint analysis. [abridged]

Measuring cosmic shear in wide-field imaging surveys requires accurate knowledge of the redshift distribution of all sources. The clustering-redshift technique exploits the angular cross-correlation of a target galaxy sample with unknown redshifts and a reference sample with known redshifts, and is an attractive alternative to colour-based methods of redshift calibration. We test the performance of such clustering redshift measurements using mock catalogues that resemble the Kilo-Degree Survey (KiDS). These mocks are created from the MICE simulation and closely mimic the properties of the KiDS source sample and the overlapping spectroscopic reference samples. We quantify the performance of the clustering redshifts by comparing the cross-correlation results with the true redshift distributions in each of the five KiDS photometric redshift bins. Such a comparison to an informative model is necessary due to the incompleteness of the reference samples at high redshifts. Clustering mean redshifts are unbiased at $|\Delta z|<0.006$ under these conditions. The redshift evolution of the galaxy bias can be reliably mitigated at this level of precision using auto-correlation measurements and self-consistency relations, and will not become a dominant source of systematic error until the arrival of Stage-IV cosmic shear surveys. Using redshift distributions from a direct colour-based estimate instead of the true redshift distributions as a model for comparison with the clustering redshifts increases the biases in the mean to up to $|\Delta z|\sim0.04$. This indicates that the interpretation of clustering redshifts in real-world applications will require more sophisticated (parameterised) models of the redshift distribution in the future. If such better models are available, the clustering-redshift technique promises to be a highly complementary alternative to other methods of redshift calibration.

We present weak lensing shear catalogues from the fourth data release of the Kilo-Degree Survey, KiDS-1000, spanning 1006 square degrees of deep and high-resolution imaging. Our `gold-sample' of galaxies, with well calibrated photometric redshift distributions, consists of 21 million galaxies with an effective number density of $\sim 6.2$ galaxies per square arcminute. We quantify the accuracy of the spatial, temporal and flux-dependent point-spread function (PSF) model, verifying that the model meets our requirements to induce less than a $0.1\sigma$ change in the inferred cosmic shear constraints on the clustering cosmological parameter $S_8 = \sigma_8\sqrt{\Omega_{\rm m}/0.3}$. Through a series of two-point null-tests we validate the shear estimates, finding no evidence for significant non-lensing B-mode distortions in the data. PSF residuals are detected in the highest-redshift bins, originating from object selection and/or weight bias. The amplitude is however shown to be sufficiently low and within our stringent requirements. With a shear-ratio null-test we verify the expected redshift scaling of the galaxy-galaxy lensing signal around luminous red galaxies. We conclude that the joint KiDS-1000 shear and photometric redshift calibration is sufficiently robust for combined-probe gravitational lensing and spectroscopic clustering analyses.

Accurate weak lensing mass estimates of clusters are needed in order to calibrate mass proxies for the cosmological exploitation of galaxy cluster surveys. Such measurements require accurate knowledge of the redshift distribution of the weak lensing source galaxies. In this context, we investigate the accuracy of photometric redshifts (photo-$z$s) computed by the 3D-HST team for the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey fields, which provide a relevant photometric reference data set for deep weak lensing studies. Through the comparison to spectroscopic redshifts and photo-$z$s based on very deep data from the Hubble Ultra Deep Field, we identify catastrophic redshift outliers in the 3D-HST/CANDELS catalogue. These would significantly bias weak lensing results if not accounted for. We investigate the cause of these outliers and demonstrate that the interpolation of spectral energy distribution (SED) templates and a well-selected combination of photometric data can reduce the net impact for weak lensing studies.

Context. The density split statistics in weak gravitational lensing analyses probes the correlation between regions of different (foreground) galaxy number densities and their weak lensing signal, measured by the shape distortion of background galaxies. Aims. In this paper, we reconsider density split statistics, by constructing a new angular filter function that is adapted to the expected relation between galaxy number density and shear pattern, in a way that the filter weighting the galaxy number density is matched to the filter that is used to quantify the shear signal. Methods. We use the results of numerical ray-tracing simulations, specifically through Millennium Simulation supplemented by a galaxy distribution based on a semi-analytic model, to construct a matched pair of adapted filter functions for the galaxy density and the tangential shear signal. We compare the performance of our new filter to the previously used top-hat filter, applying both to a different and independent set of numerical simulations (SLICS, cosmo-SLICS). Results. We show that the adapted filter yields a better correlation between the total matter and the galaxy distribution. Furthermore, the adapted filter provides a larger signal-to-noise ratio to constrain the bias between the total matter and the galaxy distribution, and we show that it is, in general, a more sensitive discriminator between different cosmologies, with the exception of cosmologies with very large $\sigma_8$ values. All analyses lead to the conclusion that our adapted filter should be favored in future density split statistic works.

We simultaneously present constraints on the stellar-to-halo mass relation for central and satellite galaxies through a weak lensing analysis of spectroscopically classified galaxies. Using overlapping data from the 4th data release of the Kilo-Degree Survey (KiDS), and the Galaxy And Mass Assembly survey (GAMA), we find that satellite galaxies are hosted by halo masses that are $0.53 \pm 0.39$ dex (68% confidence, $3\sigma$ detection) smaller than those of central galaxies of the same stellar mass (for a stellar mass of $\log(M_{\star}/M_{\odot}) = 10.6$). This is consistent with galaxy formation models, whereby infalling satellite galaxies are preferentially stripped of their dark matter. We find consistent results with similar uncertainties when comparing constraints from a standard azimuthally averaged galaxy-galaxy lensing analysis and a two-dimensional likelihood analysis of the full shear field. As the latter approach is somewhat biased due to the lens incompleteness, and does not provide any improvement in precision when applied to actual data, we conclude that stacked tangential shear measurements are best-suited for studies of the galaxy-halo connection.

We use KiDS weak lensing data to measure variations in mean halo mass as a function of several key galaxy properties (namely: stellar colour, specific star formation rate, Sersic index, and effective radius) for a volume-limited sample of GAMA galaxies in a narrow stellar mass range ($M_* \sim 2$-$5 \times 10^{10}$ Msol). This mass range is particularly interesting, inasmuch as it is where bimodalities in galaxy properties are most pronounced, and near to the break in both the galaxy stellar mass function and the stellar-to-halo mass relation (SHMR). In this narrow mass range, we find that both size and Sersic index are better predictors of halo mass than colour or SSFR, with the data showing a slight preference for Sersic index. In other words, we find that mean halo mass is more strongly correlated with galaxy structure than either stellar populations or star formation rate. These results lead to an approximate lower bound on the dispersion in halo masses among $ \log M_* \approx {10.5}$ galaxies: we find that the dispersion is $\gtrsim 0.3$ dex. This would imply either that offsets from the mean SHMR are very tightly coupled to size/structure, or that the dispersion in the SHMR is larger than past results have suggested. Our results thus provide new empirical constraints on the relationship between stellar and halo mass assembly at this particularly interesting mass range.

Several semi-analytic models (SAMs) try to explain how galaxies form, evolve and interact inside the dark matter large-scale structure. These SAMs can be tested by comparing their predictions for galaxy-galaxy-galaxy-lensing (G3L), which is weak gravitational lensing around galaxy pairs, with observations. We evaluate the SAMs by Henriques et al. (2015; H15) and by Lagos et al. (2012; L12), implemented in the Millennium Run, by comparing their predictions for G3L to observations at smaller scales than previous studies and also for pairs of lens galaxies from different populations. We compare the G3L signal predicted by the SAMs to measurements in the overlap of the Galaxy And Mass Assembly survey (GAMA), the Kilo-Degree Survey (KiDS), and the VISTA Kilo-degree Infrared Galaxy survey (VIKING), splitting lens galaxies into two colour and five stellar-mass samples. Using an improved G3L estimator, we measure the three-point correlation of the matter distribution for mixed lens pairs with galaxies from different samples, and unmixed lens pairs with galaxies from the same sample. Predictions by the H15 SAM agree with the observations for all colour-selected and all but one stellar-mass-selected sample with 95% confidence. Deviations occur for lenses with stellar masses below $9.5h^{-2}\mathrm{M}_\odot$ at scales below $0.2h^{-1}\mathrm{Mpc}$. Predictions by the L12 SAM for stellar-mass selected samples and red galaxies are significantly higher than observed, while the predicted signal for blue galaxy pairs is too low. The L12 SAM predicts more pairs of small stellar-mass and red galaxies than the H15 SAM and the observations, as well as fewer pairs of blue galaxies. Likely explanations are different treatments of environmental effects by the SAMs and different models of the initial mass function. We conclude that G3L provides a stringent test for models of galaxy formation and evolution.

The physics of gravity on cosmological scales affects both the rate of assembly of large-scale structure, and the gravitational lensing of background light through this cosmic web. By comparing the amplitude of these different observational signatures, we can construct tests that can distinguish General Relativity from its potential modifications. We use the latest weak gravitational lensing dataset from the Kilo-Degree Survey, KiDS-1000, in conjunction with overlapping galaxy spectroscopic redshift surveys BOSS and 2dFLenS, to perform the most precise existing amplitude-ratio test. We measure the associated E_G statistic with 15-20% errors, in five dz = 0.1 tomographic redshift bins in the range 0.2 < z < 0.7, on projected scales up to 100 Mpc/h. The scale-independence and redshift-dependence of these measurements are consistent with the theoretical expectation of General Relativity in a Universe with matter density Omega_m = 0.27 +/- 0.04. We demonstrate that our results are robust against different analysis choices, including schemes for correcting the effects of source photometric redshift errors, and compare the performance of angular and projected galaxy-galaxy lensing statistics.

We present updated cosmological constraints for the KiDS+VIKING-450 cosmic shear dataset (KV450), estimated using redshift distributions and photometric samples defined using self organising maps (SOMs). Our fiducial analysis finds marginal posterior constraints of $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.716^{+0.043}_{-0.038}$; smaller than, but fully consistent with, previous work using this dataset ($|\Delta S_8| = 0.023$). We analyse additional samples and redshift distributions constructed in three ways: excluding certain spectroscopic surveys during redshift calibration, excluding lower-confidence spectroscopic redshifts in redshift calibration, and considering only photometric sources which are jointly calibrated by at least three spectroscopic surveys. In all cases, the method utilised here proves robust: we find a maximal deviation from our fiducial analysis of $|\Delta S_8| \leq 0.009$ for all samples defined and analysed using our SOM. Our largest shift in $S_8$ is found when calibrating redshift distributions without the DEEP2 spectroscopic subset, where we find $S_8=0.707_{-0.042}^{+0.046}$. This difference with respect to the fiducial is both significantly smaller than, and in the opposite direction to, the equivalent shift from previous work. No sample analysed in this work results in a meaningful positive shift in $S_8$ with respect to our fiducial constraints. These results suggest that our improved cosmological parameter estimates are insensitive to pathological misrepresentation of photometric sources by the spectroscopy used for direct redshift calibration, and therefore that this systematic effect cannot be responsible for the observed difference between $S_8$ estimates made with KV450 and Planck CMB probes.

We present a combined tomographic weak gravitational lensing analysis of the Kilo Degree Survey (KV450) and the Dark Energy Survey (DES-Y1). We homogenize the analysis of these two public cosmic shear datasets by adopting consistent priors and modeling of nonlinear scales, and determine new redshift distributions for DES-Y1 based on deep public spectroscopic surveys. Adopting these revised redshifts results in a $0.8\sigma$ reduction in the DES-inferred value for $S_8$, which decreases to a $0.5\sigma$ reduction when including a systematic redshift calibration error model from mock DES data based on the MICE2 simulation. The combined KV450 + DES-Y1 constraint on $S_8 = 0.762^{+0.025}_{-0.024}$ is in tension with the Planck 2018 constraint from the cosmic microwave background at the level of $2.5\sigma$. This result highlights the importance of developing methods to provide accurate redshift calibration for current and future weak lensing surveys.

In this paper we introduce the \textsc{Deepz} deep learning photometric redshift (photo-$z$) code. As a test case, we apply the code to the PAU survey (PAUS) data in the COSMOS field. \textsc{Deepz} reduces the $\sigma_{68}$ scatter statistic by 50\% at $i_{\rm AB}=22.5$ compared to existing algorithms. This improvement is achieved through various methods, including transfer learning from simulations where the training set consists of simulations as well as observations, which reduces the need for training data. The redshift probability distribution is estimated with a mixture density network (MDN), which produces accurate redshift distributions. Our code includes an autoencoder to reduce noise and extract features from the galaxy SEDs. It also benefits from combining multiple networks, which lowers the photo-$z$ scatter by 10 percent. Furthermore, training with randomly constructed coadded fluxes adds information about individual exposures, reducing the impact of photometric outliers. In addition to opening up the route for higher redshift precision with narrow bands, these machine learning techniques can also be valuable for broad-band surveys.

Accurate photometric redshift calibration is central to the robustness of all cosmology constraints from cosmic shear surveys. Analyses of the KiDS re-weighted training samples from all overlapping spectroscopic surveys to provide a direct redshift calibration. Using self-organising maps (SOMs) we demonstrate that this spectroscopic compilation is sufficiently complete for KiDS, representing $99\%$ of the effective 2D cosmic shear sample. We use the SOM to define a $100\%$ represented `gold' cosmic shear sample, per tomographic bin. Using mock simulations of KiDS and the spectroscopic training set, we estimate the uncertainty on the SOM redshift calibration, and find that photometric noise, sample variance, and spectroscopic selection effects (including redshift and magnitude incompleteness) induce a combined maximal scatter on the bias of the redshift distribution reconstruction ($\Delta \langle z \rangle=\langle z \rangle_{\rm est}-\langle z \rangle_{\rm true}$) of $\sigma_{\Delta \langle z \rangle} \leq 0.006$ in all tomographic bins. We show that the SOM calibration is unbiased in the cases of noiseless photometry and perfectly representative spectroscopic datasets, as expected from theory. The inclusion of both photometric noise and spectroscopic selection effects in our mock data introduces a maximal bias of $\Delta \langle z \rangle =0.013\pm0.006$, or $\Delta \langle z \rangle \leq 0.025$ at $97.5\%$ confidence, once quality flags have been applied to the SOM. The method presented here represents a significant improvement over the previously adopted direct redshift calibration implementation for KiDS, owing to its diagnostic and quality assurance capabilities. The implementation of this method in future cosmic shear studies will allow better diagnosis, examination, and mitigation of systematic biases in photometric redshift calibration.

We present cosmological constraints from a joint cosmic shear analysis of the Kilo-Degree Survey (KV450) and the Dark Energy Survey (DES-Y1), conducted using Complete Orthogonal Sets of E/B-Integrals (COSEBIs). With COSEBIs we isolate any B-modes which have a non-cosmic shear origin and demonstrate the robustness of our cosmological E-mode analysis as no significant B-modes are detected. We highlight how COSEBIs are fairly insensitive to the amplitude of the non-linear matter power spectrum at high $k$-scales, mitigating the uncertain impact of baryon feedback in our analysis. COSEBIs, therefore, allow us to utilise additional small-scale information, improving the DES-Y1 joint constraints on $S_8=\sigma_8(\Omega_{\rm m}/0.3)^{0.5}$ and $\Omega_{\rm m}$ by $20\%$. Adopting a flat $\Lambda$CDM model we find $S_8=0.755^{+0.019}_{-0.021}$, which is in $3.2\sigma$ tension with the Planck Legacy analysis of the cosmic microwave background.

We reanalyse the anisotropic galaxy clustering measurement from the Baryon Oscillation Spectroscopic Survey (BOSS), demonstrating that using the full shape information provides cosmological constraints that are comparable to other low-redshift probes. We find $\Omega_\mathrm{m} = 0.317^{+0.015}_{-0.019}$, $\sigma_8 = 0.710\pm 0.049$, and $h = 0.704\pm 0.024$ for flat $\Lambda$CDM cosmologies using uninformative priors on $\Omega_\mathrm{c}h^2$, $100\theta_\mathrm{MC}$, $\ln 10^{10} A_{s}$, and $n_{s}$, and a prior on $\Omega_\mathrm{b}h^2$ that is much wider than current constraints. We quantify the agreement between the Planck 2018 constraints from the cosmic microwave background and BOSS, finding the two data sets to be consistent within a flat $\Lambda$CDM cosmology using the Bayes factor as well as the prior-insensitive suspiciousness statistic. Combining two low-redshift probes, we jointly analyse the clustering of BOSS galaxies with weak lensing measurements from the Kilo-Degree Survey (KV450). The combination of BOSS and KV450 improves the measurement by up to 45%, constraining $\sigma_8 = 0.702\pm 0.029$ and $S_8 = \sigma_8\sqrt{\Omega_\mathrm{m}/0.3} = 0.728\pm 0.026$. Over the full 5D parameter space, the odds in favour of single cosmology describing galaxy clustering, lensing, and the cosmic microwave background are $7\pm2$. The suspiciousness statistic signals a $2.1\pm0.3\sigma$ tension between the combined low-redshift probes and measurements from the cosmic microwave background.

We present a semi-analytic model for the shear two-point correlation function of a cosmic shear survey with non-uniform depth. Ground-based surveys are subject to depth variations that primarily arise through varying atmospheric conditions. For a survey like the Kilo-Degree Survey (KiDS), we find that the measured depth variation increases the amplitude of the observed shear correlation function at the level of a few percent out to degree-scales, relative to the assumed uniform-depth case. The impact on the inferred cosmological parameters is shown to be insignificant for a KiDS-like survey. For next-generation cosmic shear experiments, however, we conclude that variable depth should be accounted for.

We present a weak lensing detection of filamentary structures in the cosmic web, combining data from the Kilo-Degree Survey, the Red Cluster Sequence Lensing Survey and the Canada-France-Hawaii Telescope Lensing Survey. The line connecting luminous red galaxies with a separation of $3 - 5\, h^{-1}\text{Mpc}$ is chosen as a proxy for the location of filaments. We measure the average weak lensing shear around $\sim$11,000 candidate filaments selected in this way from the Sloan Digital Sky Survey. After nulling the shear induced by the dark matter haloes around each galaxy, we report a $3.4\,\sigma$ detection of an anisotropic shear signal from the matter that connects them. Adopting a filament density profile, motivated from $N$-body simulations, the average density at the centre of these filamentary structures is found to be $15 \pm 4$ times the critical density.

We present a tomographic cosmic shear analysis of the Kilo-Degree Survey (KiDS) combined with the VISTA Kilo-Degree Infrared Galaxy Survey (VIKING). This is the first time that a full optical to near-infrared data set has been used for a wide-field cosmological weak lensing experiment. This unprecedented data, spanning $450~$deg$^2$, allows us to improve significantly the estimation of photometric redshifts, such that we are able to include robustly higher-redshift sources for the lensing measurement, and - most importantly - solidify our knowledge of the redshift distributions of the sources. Based on a flat $\Lambda$CDM model we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.737_{-0.036}^{+0.040}$ in a blind analysis from cosmic shear alone. The tension between KiDS cosmic shear and the Planck-Legacy CMB measurements remains in this systematically more robust analysis, with $S_8$ differing by $2.3\sigma$. This result is insensitive to changes in the priors on nuisance parameters for intrinsic alignment, baryon feedback, and neutrino mass. KiDS shear measurements are calibrated with a new, more realistic set of image simulations and no significant B-modes are detected in the survey, indicating that systematic errors are under control. When calibrating our redshift distributions by assuming the 30-band COSMOS-2015 photometric redshifts are correct (following the Dark Energy Survey and the Hyper Suprime-Cam Survey), we find the tension with Planck is alleviated. The robust determination of source redshift distributions remains one of the most challenging aspects for future cosmic shear surveys.

The PAU Survey (PAUS) is an innovative photometric survey with 40 narrow bands at the William Herschel Telescope (WHT). The narrow bands are spaced at 100\AA\ intervals covering the range 4500\AA\ to 8500\AA\ and, in combination with standard broad bands, enable excellent redshift precision. This paper describes the technique, galaxy templates and additional photometric calibration used to determine early photometric redshifts from PAUS. Using BCNz2, a new photometric redshift code developed for this purpose, we characterise the photometric redshift performance using PAUS data on the COSMOS field. Comparison to secure spectra from zCOSMOS DR3 shows that PAUS achieves $\sigma_{68} /(1+z) = 0.0037$ to $i_{\mathrm{AB}} < 22.5$ when selecting the best 50% of the sources based on a photometric redshift quality cut. Furthermore, a higher photo-z precision ($\sigma_{68}/(1+z) \sim 0.001$) is obtained for a bright and high quality selection, which is driven by the identification of emission lines. We conclude that PAUS meets its design goals, opening up a hitherto uncharted regime of deep, wide, and dense galaxy survey with precise redshifts that will provide unique insights into the formation, evolution and clustering of galaxies, as well as their intrinsic alignments.

We present the curation and verification of a new combined optical and near infrared dataset for cosmology and astrophysics, derived from the combination of $ugri$-band imaging from the Kilo Degree Survey (KiDS) and $ZY\!J\!H\!K_{\rm s}$-band imaging from the VISTA Kilo degree Infrared Galaxy (VIKING) survey. This dataset is unrivaled in cosmological imaging surveys due to its combination of area ($458$ deg$^2$ before masking), depth ($r\le25$), and wavelength coverage ($ugriZY\!J\!H\!K_{\rm s}$). The combination of survey depth, area, and (most importantly) wavelength coverage allows significant reductions in systematic uncertainties (i.e. reductions of between 10 and 60\% in bias, outlier rate, and scatter) in photometric-to-spectroscopic redshift comparisons, compared to the optical-only case at photo-$z$ above $0.7$. The complementarity between our optical and NIR surveys means that over $80\%$ of our sources, across all photo-$z$, have significant detections (i.e. not upper limits) in our $8$ reddest bands. We derive photometry, photo-$z$, and stellar masses for all sources in the survey, and verify these data products against existing spectroscopic galaxy samples. We demonstrate the fidelity of our higher-level data products by constructing the survey stellar mass functions in 8 volume-complete redshift bins. We find that these photometrically derived mass functions provide excellent agreement with previous mass evolution studies derived using spectroscopic surveys. The primary data products presented in this paper are publicly available at http://kids.strw.leidenuniv.nl/.

The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope, specifically designed for measuring weak gravitational lensing by galaxies and large-scale structure. When completed it will consist of 1350 square degrees imaged in four filters (ugri). Here we present the fourth public data release which more than doubles the area of sky covered by data release 3. We also include aperture-matched ZYJHKs photometry from our partner VIKING survey on the VISTA telescope in the photometry catalogue. We illustrate the data quality and describe the catalogue content. Two dedicated pipelines are used for the production of the optical data. The Astro-WISE information system is used for the production of co-added images in the four survey bands, while a separate reduction of the r-band images using the theli pipeline is used to provide a source catalogue suitable for the core weak lensing science case. All data have been re-reduced for this data release using the latest versions of the pipelines. The VIKING photometry is obtained as forced photometry on the theli sources, using a re-reduction of the VIKING data that starts from the VISTA pawprints. Modifications to the pipelines with respect to earlier releases are described in detail. The photometry is calibrated to the Gaia DR2 G band using stellar locus regression. In this data release a total of 1006 square-degree survey tiles with stacked ugri images are made available, accompanied by weight maps, masks, and single-band source lists. We also provide a multi-band catalogue based on r-band detections, including homogenized photometry and photometric redshifts, for the whole dataset. Mean limiting magnitudes (5 sigma in a 2" aperture) are 24.23, 25.12, 25.02, 23.68 in ugri, respectively, and the mean r-band seeing is 0.70".

We analyse three public cosmic shear surveys; the Kilo-Degree Survey (KiDS-450), the Dark Energy Survey (DES-SV) and the Canada France Hawaii Telescope Lensing Survey (CFHTLenS). Adopting the COSEBIs statistic to cleanly and completely separate the lensing E-modes from the non-lensing B-modes, we detect B-modes in KiDS-450 and CFHTLenS at the level of about 2.7 $\sigma$. For DES- SV we detect B-modes at the level of 2.8 $\sigma$ in a non-tomographic analysis, increasing to a 5.5 $\sigma$ B-mode detection in a tomographic analysis. In order to understand the origin of these detected B-modes we measure the B-mode signature of a range of different simulated systematics including PSF leakage, random but correlated PSF modelling errors, camera-based additive shear bias and photometric redshift selection bias. We show that any correlation between photometric-noise and the relative orientation of the galaxy to the point-spread-function leads to an ellipticity selection bias in tomographic analyses. This work therefore introduces a new systematic for future lensing surveys to consider. We find that the B-modes in DES-SV appear similar to a superposition of the B-mode signatures from all of the systematics simulated. The KiDS-450 and CFHTLenS B-mode measurements show features that are consistent with a repeating additive shear bias.

Exploiting the full statistical power of future cosmic shear surveys will necessitate improvements to the accuracy with which the gravitational lensing signal is measured. We present a framework for calibrating shear with image simulations that demonstrates the importance of including realistic correlations between galaxy morphology, size and more importantly, photometric redshifts. This realism is essential so that selection and shape measurement biases can be calibrated accurately for a tomographic cosmic shear analysis. We emulate Kilo-Degree Survey (KiDS) observations of the COSMOS field using morphological information from {\it Hubble} Space Telescope imaging, faithfully reproducing the measured galaxy properties from KiDS observations of the same field. We calibrate our shear measurements from lensfit, and find through a range of sensitivity tests that lensfit is robust and unbiased within the allowed 2 per cent tolerance of our study. Our results show that the calibration has to be performed by selecting the tomographic samples in the simulations, consistent with the actual cosmic shear analysis, because the joint distributions of galaxy properties are found to vary with redshift. Ignoring this redshift variation could result in misestimating the shear bias by an amount that exceeds the allowed tolerance. To improve the calibration for future cosmic shear analyses, it will be essential to also correctly account for the measurement of photometric redshifts, which requires simulating multi-band observations.

We study projected underdensities in the cosmic galaxy density field known as 'troughs', and their overdense counterparts, which we call 'ridges'. We identify these regions using a bright sample of foreground galaxies from the photometric Kilo-Degree Survey (KiDS), specifically selected to mimic the spectroscopic Galaxy And Mass Assembly survey (GAMA). Using background galaxies from KiDS, we measure the weak gravitational lensing profiles of the troughs/ridges. We quantify the amplitude of their lensing strength $A$ as a function of galaxy density percentile rank $P$ and galaxy overdensity $\delta$, and find that the skewness in the galaxy density distribution is reflected in the total mass distribution measured by weak lensing. We interpret our results using the mock galaxy catalogue from the Marenostrum Institut de Ci\`encies de l'Espai (MICE) simulation, and find a good agreement with our observations. Using signal-to-noise weights derived from the Scinet LIghtCone Simulations (SLICS) mock catalogue we optimally stack the lensing signal of KiDS troughs with an angular radius $\theta_A$ = {5,10,15,20} arcmin, resulting in {16.8,14.9,10.13,7.55} $\sigma$ detections. Finally, we select troughs using a volume-limited sample of galaxies, split into two redshift bins between 0.1 < z < 0.3. For troughs/ridges with transverse comoving radius $R_A$ = 1.9 Mpc/h, we find no significant difference in the comoving Excess Surface Density as a function of $P$ and $\delta$ between the low- and high-redshift sample. Using the MICE and SLICS mocks we predict that trough and ridge evolution could be detected with gravitational lensing using deeper and wider lensing surveys, such as those from the Large Synoptic Survey Telescope and Euclid.

Cosmic shear is a primary cosmological probe for several present and upcoming surveys investigating dark matter and dark energy, such as Euclid or WFIRST. The probe requires an extremely accurate measurement of the shapes of millions of galaxies based on imaging data. Crucially, the shear measurement must address and compensate for a range of interwoven nuisance effects related to the instrument optics and detector, noise, unknown galaxy morphologies, colors, blending of sources, and selection effects. This paper explores the use of supervised machine learning (ML) as a tool to solve this inverse problem. We present a simple architecture that learns to regress shear point estimates and weights via shallow artificial neural networks. The networks are trained on simulations of the forward observing process, and take combinations of moments of the galaxy images as inputs. A challenging peculiarity of this ML application is the combination of the noisiness of the input features and the requirements on the accuracy of the inverse regression. To address this issue, the proposed training algorithm minimizes bias over multiple realizations of individual source galaxies, reducing the sensitivity to properties of the overall sample of source galaxies. Importantly, an observational selection function of these source galaxies can be straightforwardly taken into account via the weights. We first introduce key aspects of our approach using toy-model simulations, and then demonstrate its potential on images mimicking Euclid data. Finally, we analyze images from the GREAT3 challenge, obtaining competitively low shear biases despite the use of a simple training set. We conclude that the further development of ML approaches is of high interest to meet the stringent requirements on the shear measurement in current and future surveys. A demonstration implementation of our technique is publicly available.

The outer regions of galaxies are more susceptible to the tidal interactions that lead to intrinsic alignments of galaxies. The resulting alignment signal may therefore depend on the passband if the colours of galaxies vary spatially. To quantify this, we measured the shapes of galaxies with spectroscopic redshifts from the GAMA survey using deep gri imaging data from the KiloDegree Survey. The performance of the moment-based shape measurement algorithm DEIMOS was assessed using dedicated image simulations, which showed that the ellipticities could be determined with an accuracy better than 1% in all bands. Additional tests for potential systematic errors did not reveal any issues. We measure a significant difference of the alignment signal between the g,r and i-band observations. This difference exceeds the amplitude of the linear alignment model on scales below 2 Mpc/h. Separating the sample into central/satellite and red/blue galaxies, we find that that the difference is dominated by red satellite galaxies.

We present the mass calibration for galaxy clusters detected with the AMICO code in KiDS DR3 data. The cluster sample comprises $\sim$ 7000 objects and covers the redshift range 0.1 < $z$ < 0.6. We perform a weak lensing stacked analysis by binning the clusters according to redshift and two different mass proxies provided by AMICO, namely the amplitude $A$ (measure of galaxy abundance through an optimal filter) and the richness $\lambda^*$ (sum of membership probabilities in a consistent radial and magnitude range across redshift). For each bin, we model the data as a truncated NFW profile plus a 2-halo term, taking into account uncertainties related to concentration and miscentring. From the retrieved estimates of the mean halo masses, we construct the $A$-$M_{200}$ and the $\lambda^*$-$M_{200}$ relations. The relations extend over more than one order of magnitude in mass, down to $M_{200} \sim 2 (5) \times 10^{13} M_\odot/h$ at $z$ = 0.2 (0.5), with small evolution in redshift. The logarithmic slope is $\sim 2.0$ for the $A$-mass relation, and $\sim 1.7$ for the $\lambda^*$-mass relation, consistent with previous estimations on mock catalogues and coherent with the different nature of the two observables.

We present a weak gravitational lensing analysis of 815 square degree of $i$-band imaging from the Kilo-Degree Survey (KiDS-$i$-800). In contrast to the deep $r$-band observations, which take priority during excellent seeing conditions and form the primary KiDS dataset (KiDS-$r$-450), the complementary yet shallower KiDS-$i$-800 spans a wide range of observing conditions. The overlapping KiDS-$i$-800 and KiDS-$r$-450 imaging therefore provides a unique opportunity to assess the robustness of weak lensing measurements. In our analysis, we introduce two new `null' tests. The `nulled' two-point shear correlation function uses a matched catalogue to show that the calibrated KiDS-$i$-800 and KiDS-$r$-450 shear measurements agree at the level of $1 \pm 4$\%. We use five galaxy lens samples to determine a `nulled' galaxy-galaxy lensing signal from the full KiDS-$i$-800 and KiDS-$r$-450 surveys and find that the measurements agree to $7 \pm 5$\% when the KiDS-$i$-800 source redshift distribution is calibrated using either spectroscopic redshifts, or the 30-band photometric redshifts from the COSMOS survey.

We present a new measurement of $E_{\rm G}$, which combines measurements of weak gravitational lensing, real-space galaxy clustering and redshift space distortions. This statistic was proposed as a consistency test of General Relativity (GR) that is insensitive to linear, deterministic galaxy bias and the matter clustering amplitude. We combine deep imaging data from KiDS with overlapping spectroscopy from 2dFLenS, BOSS DR12 and GAMA and find $E_{\rm G}(\overline{z}=0.267)=0.43 \pm 0.13$ (GAMA), $E_{\rm G}(\overline{z}=0.305)=0.27 \pm 0.08$ (LOWZ+2dFLOZ) and $E_{\rm G}(\overline{z}=0.554)=0.26 \pm 0.07$ (CMASS+2dFHIZ). We demonstrate that the existing tension in the value of the matter density parameter hinders the robustness of this statistic as solely a test of GR. We find that our $E_{\rm G}$ measurements, as well as existing ones in the literature, favour a lower matter density cosmology than the Cosmic Microwave Background. For a flat $\Lambda$CDM Universe and assuming GR, we find $\Omega_{\rm m}(z=0)=0.25\pm0.03$. With this paper we publicly release the 2dFLenS dataset at: \url{http://2dflens.swin.edu.au}.

We present a public suite of weak lensing mock data, extending the Scinet Light Cone Simulations (SLICS) to simulate cross-correlation analyses with different cosmological probes. These mocks include KiDS-450- and LSST-like lensing data, cosmic microwave background lensing maps and simulated spectroscopic surveys that emulate the GAMA, BOSS and 2dFLenS galaxy surveys. With 844 independent realisations, our mocks are optimised for combined-probe covariance estimation, which we illustrate for the case of a joint measurement involving cosmic shear, galaxy-galaxy lensing and galaxy clustering from KiDS-450 and BOSS data. With their high spatial resolution, the SLICS are also optimal for predicting the signal for novel lensing estimators, for the validation of analysis pipelines, and for testing a range of systematic effects such as the impact of neighbour-exclusion bias on the measured tomographic cosmic shear signal. For surveys like KiDS and DES, where the rejection of neighbouring galaxies occurs within ~2 arcseconds, we show that the measured cosmic shear signal will be biased low, but by less than a percent on the angular scales that are typically used in cosmic shear analyses. The amplitude of the neighbour-exclusion bias doubles in deeper, LSST-like data. The simulation products described in this paper are made available at http://slics.roe.ac.uk/.

We present the first "clipped" cosmic shear measurement using data from the Kilo-Degree Survey (KiDS-450). "Clipping" transformations suppress the signal from the highest density, non-linear regions of cosmological fields. We demonstrate that these transformations improve constraints on $S_8=\sigma_8(\Omega_{\rm{m}}/0.3)^{0.5}$ when used in combination with conventional two-point statistics. For the KiDS-450 data, we find that the combined measurements improve the constraints on $S_8$ by 17%, compared to shear correlation functions alone. We determine the expectation value of the clipped shear correlation function using a suite of numerical simulations, and develop methodology to mitigate the impact of masking and shot noise. Future improvements in numerical simulations and mass reconstruction methodology will permit the precise calibration of clipped cosmic shear statistics such that clipping can become a standard tool in weak lensing analyses.

We measure the projected galaxy clustering and galaxy-galaxy lensing signals using the Galaxy And Mass Assembly (GAMA) survey and Kilo-Degree Survey (KiDS) to study galaxy bias. We use the concept of non-linear and stochastic galaxy biasing in the framework of halo occupation statistics to constrain the parameters of the halo occupation statistics and to unveil the origin of galaxy biasing. The bias function $\Gamma_{\text{gm}}(r_{\text{p}})$, where $r_{\text{p}}$ is the projected comoving separation, is evaluated using the analytical halo model from which the scale dependence of $\Gamma_{\text{gm}}(r_{\text{p}})$, and the origin of the non-linearity and stochasticity in halo occupation models can be inferred. Our observations unveil the physical reason for the non-linearity and stochasticity, further explored using hydrodynamical simulations, with the stochasticity mostly originating from the non-Poissonian behaviour of satellite galaxies in the dark matter haloes and their spatial distribution, which does not follow the spatial distribution of dark matter in the halo. The observed non-linearity is mostly due to the presence of the central galaxies, as was noted from previous theoretical work on the same topic. We also see that overall, more massive galaxies reveal a stronger scale dependence, and out to a larger radius. Our results show that a wealth of information about galaxy bias is hidden in halo occupation models. These models should therefore be used to determine the influence of galaxy bias in cosmological studies.

We present a machine-learning photometric redshift analysis of the Kilo-Degree Survey Data Release 3, using two neural-network based techniques: ANNz2 and MLPQNA. Despite limited coverage of spectroscopic training sets, these ML codes provide photo-zs of quality comparable to, if not better than, those from the BPZ code, at least up to zphot<0.9 and r<23.5. At the bright end of r<20, where very complete spectroscopic data overlapping with KiDS are available, the performance of the ML photo-zs clearly surpasses that of BPZ, currently the primary photo-z method for KiDS. Using the Galaxy And Mass Assembly (GAMA) spectroscopic survey as calibration, we furthermore study how photo-zs improve for bright sources when photometric parameters additional to magnitudes are included in the photo-z derivation, as well as when VIKING and WISE infrared bands are added. While the fiducial four-band ugri setup gives a photo-z bias $\delta z=-2e-4$ and scatter $\sigma_z<0.022$ at mean z = 0.23, combining magnitudes, colours, and galaxy sizes reduces the scatter by ~7% and the bias by an order of magnitude. Once the ugri and IR magnitudes are joined into 12-band photometry spanning up to 12 $\mu$, the scatter decreases by more than 10% over the fiducial case. Finally, using the 12 bands together with optical colours and linear sizes gives $\delta z<4e-5$ and $\sigma_z<0.019$. This paper also serves as a reference for two public photo-z catalogues accompanying KiDS DR3, both obtained using the ANNz2 code. The first one, of general purpose, includes all the 39 million KiDS sources with four-band ugri measurements in DR3. The second dataset, optimized for low-redshift studies such as galaxy-galaxy lensing, is limited to r<20, and provides photo-zs of much better quality than in the full-depth case thanks to incorporating optical magnitudes, colours, and sizes in the GAMA-calibrated photo-z derivation.

We present cosmological parameter constraints from a joint analysis of three cosmological probes: the tomographic cosmic shear signal in $\sim$450 deg$^2$ of data from the Kilo Degree Survey (KiDS), the galaxy-matter cross-correlation signal of galaxies from the Galaxies And Mass Assembly (GAMA) survey determined with KiDS weak lensing, and the angular correlation function of the same GAMA galaxies. We use fast power spectrum estimators that are based on simple integrals over the real-space correlation functions, and show that they are practically unbiased over relevant angular frequency ranges. We test our full pipeline on numerical simulations that are tailored to KiDS and retrieve the input cosmology. By fitting different combinations of power spectra, we demonstrate that the three probes are internally consistent. For all probes combined, we obtain $S_8\equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3}=0.800_{-0.027}^{+0.029}$, consistent with Planck and the fiducial KiDS-450 cosmic shear correlation function results. Marginalising over wide priors on the mean of the tomographic redshift distributions yields consistent results for $S_8$ with an increase of $28\%$ in the error. The combination of probes results in a $26\%$ reduction in uncertainties of $S_8$ over using the cosmic shear power spectra alone. The main gain from these additional probes comes through their constraining power on nuisance parameters, such as the galaxy intrinsic alignment amplitude or potential shifts in the redshift distributions, which are up to a factor of two better constrained compared to using cosmic shear alone, demonstrating the value of large-scale structure probe combination.

We demonstrate that deep good-seeing VLT/HAWK-I $K_\mathrm{s}$ images complemented with $g$+$z$-band photometry can yield a sensitivity for weak lensing studies of massive galaxy clusters at redshifts \mbox{$0.7\lesssim z \lesssim 1.1$}, which is almost identical to the sensitivity of HST/ACS mosaics of single-orbit depth. Key reasons for this good performance are the excellent image quality frequently achievable for $K_\mathrm{s}$ imaging from the ground, a highly effective photometric selection of background galaxies, and a galaxy ellipticity dispersion that is noticeably lower than for optically observed high-redshift galaxy samples. Incorporating results from the 3D-HST and UltraVISTA surveys we also obtained a more accurate calibration of the source redshift distribution than previously achieved for similar optical weak lensing data sets. Here we studied the extremely massive galaxy cluster RCS2$J$232727.7$-$020437 (\mbox{$z=0.699$}), combining deep VLT/\mbox{HAWK-I} $K_\mathrm{s}$ images (point spread function with a 0\farcs35 full width at half maximum) with LBT/LBC photometry. The resulting weak lensing mass reconstruction suggests that the cluster consists of a single overdensity, which is detected with a peak significance of $10.1\sigma$. We constrained the cluster mass to \mbox{$M_\mathrm{200c}/(10^{15} \mathrm{M}_\odot) =2.06^{+0.28}_{-0.26}(\mathrm{stat.})\pm 0.12 (\mathrm{sys.})$} assuming a spherical Navarro, Frenk \& White model and simulation-based priors on the concentration, making it one of the most massive galaxy clusters known in the \mbox{$z\gtrsim 0.7$} Universe. We also cross-checked the HAWK-I measurements through an analysis of overlapping HST/ACS images, yielding fully consistent estimates of the lensing signal.

We compute the spherical-sky weak-lensing power spectrum of the shear and convergence. We discuss various approximations, such as flat-sky, and first- and second- order Limber equations for the projection. We find that the impact of adopting these approximations is negligible when constraining cosmological parameters from current weak lensing surveys. This is demonstrated using data from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). We find that the reported tension with Planck Cosmic Microwave Background (CMB) temperature anisotropy results cannot be alleviated. For future large-scale surveys with unprecedented precision, we show that the spherical second-order Limber approximation will provide sufficient accuracy. In this case, the cosmic-shear power spectrum is shown to be in agreement with the full projection at the sub-percent level for l > 3, with the corresponding errors an order of magnitude below cosmic variance for all l. When computing the two-point shear correlation function, we show that the flat-sky fast Hankel transformation results in errors below two percent compared to the full spherical transformation. In the spirit of reproducible research, our numerical implementation of all approximations and the full projection are publicly available within the package nicaea at http://www.cosmostat.org/software/nicaea.

We study the scaling relations between the baryonic content and total mass of groups of galaxies, as these systems provide a unique way to examine the role of non-gravitational processes in structure formation. Using Planck and ROSAT data, we conduct detailed comparisons of the stacked thermal Sunyaev-Zel'dovich (tSZ) effect and X-ray scaling relations of galaxy groups found in the the Galaxy And Mass Assembly (GAMA) survey and the BAHAMAS hydrodynamical simulation. We use weak gravitational lensing data from the Kilo Degree Survey (KiDS) to determine the average halo mass of the studied systems. We analyse the simulation in the same way, using realistic weak lensing, X-ray, and tSZ synthetic observations. Furthermore, to keep selection biases under control, we employ exactly the same galaxy selection and group identification procedures to the observations and simulation. Applying this careful comparison, we find that the simulations are in agreement with the observations, particularly with regards to the scaling relations of the lensing and tSZ results. This finding demonstrates that hydrodynamical simulation have reached the level of realism that is required to interpret observational survey data and study the baryon physics within dark matter haloes, where analytical modelling is challenging. Finally, using simulated data, we demonstrate that our observational processing of the X-ray and tSZ signals is free of significant biases. We find that our optical group selection procedure has, however, some room for improvement.

Euclid and the Large Synoptic Survey Telescope (LSST) are poised to dramatically change the astronomy landscape early in the next decade. The combination of high cadence, deep, wide-field optical photometry from LSST with high resolution, wide-field optical photometry and near-infrared photometry and spectroscopy from Euclid will be powerful for addressing a wide range of astrophysical questions. We explore Euclid/LSST synergy, ignoring the political issues associated with data access to focus on the scientific, technical, and financial benefits of coordination. We focus primarily on dark energy cosmology, but also discuss galaxy evolution, transient objects, solar system science, and galaxy cluster studies. We concentrate on synergies that require coordination in cadence or survey overlap, or would benefit from pixel-level co-processing that is beyond the scope of what is currently planned, rather than scientific programs that could be accomplished only at the catalog level without coordination in data processing or survey strategies. We provide two quantitative examples of scientific synergies: the decrease in photo-z errors (benefitting many science cases) when high resolution Euclid data are used for LSST photo-z determination, and the resulting increase in weak lensing signal-to-noise ratio from smaller photo-z errors. We briefly discuss other areas of coordination, including high performance computing resources and calibration data. Finally, we address concerns about the loss of independence and potential cross-checks between the two missions and potential consequences of not collaborating.

We describe data release 3 (DR3) of the Galaxy And Mass Assembly (GAMA) survey. The GAMA survey is a spectroscopic redshift and multi-wavelength photometric survey in three equatorial regions each of 60.0 deg^2 (G09, G12, G15), and two southern regions of 55.7 deg^2 (G02) and 50.6 deg^2 (G23). DR3 consists of: the first release of data covering the G02 region and of data on H-ATLAS sources in the equatorial regions; and updates to data on sources released in DR2. DR3 includes 154809 sources with secure redshifts across four regions. A subset of the G02 region is 95.5% redshift complete to r<19.8 over an area of 19.5 deg^2, with 20086 galaxy redshifts, that overlaps substantially with the XXL survey (X-ray) and VIPERS (redshift survey). In the equatorial regions, the main survey has even higher completeness (98.5%), and spectra for about 75% of H-ATLAS filler targets were also obtained. This filler sample extends spectroscopic redshifts, for probable optical counterparts to H-ATLAS sub-mm sources, to 0.8 mag deeper (r<20.6) than the GAMA main survey. There are 25814 galaxy redshifts for H-ATLAS sources from the GAMA main or filler surveys. GAMA DR3 is available at the survey website (www.gama-survey.org/dr3/).

We perform a combined analysis of cosmic shear tomography, galaxy-galaxy lensing tomography, and redshift-space multipole power spectra (monopole and quadrupole) using 450 deg$^2$ of imaging data by the Kilo Degree Survey (KiDS) overlapping with two spectroscopic surveys: the 2-degree Field Lensing Survey (2dFLenS) and the Baryon Oscillation Spectroscopic Survey (BOSS). We restrict the galaxy-galaxy lensing and multipole power spectrum measurements to the overlapping regions with KiDS, and self-consistently compute the full covariance between the different observables using a large suite of $N$-body simulations. We methodically analyze different combinations of the observables, finding that galaxy-galaxy lensing measurements are particularly useful in improving the constraint on the intrinsic alignment amplitude (by 30%, positive at $3.5\sigma$ in the fiducial data analysis), while the multipole power spectra are useful in tightening the constraints along the lensing degeneracy direction (e.g. factor of two stronger matter density constraint in the fiducial analysis). The fully combined constraint on $S_8 \equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3} = 0.742 \pm 0.035$, which is an improvement by 20% compared to KiDS alone, corresponds to a $2.6\sigma$ discordance with Planck, and is not significantly affected by fitting to a more conservative set of scales. Given the tightening of the parameter space, we are unable to resolve the discordance with an extended cosmology that is simultaneously favored in a model selection sense, including the sum of neutrino masses, curvature, evolving dark energy, and modified gravity. The complementarity of our observables allows for constraints on modified gravity degrees of freedom that are not simultaneously bounded with either probe alone, and up to a factor of three improvement in the $S_8$ constraint in the extended cosmology compared to KiDS alone.

We measured stacked weak lensing cluster masses for a sample of 1325 galaxy clusters detected by the RedGOLD algorithm in the Canada-France-Hawaii Telescope Legacy Survey W1 and the Next Generation Virgo Cluster Survey at $0.2<z<0.5$, in the optical richness range $10<\lambda<70$. After a selection of our best richness subsample ($20<\lambda<50$), this is the most comprehensive lensing study of a $\sim 100\%$ complete and $\sim 90\%$ pure optical cluster catalogue in this redshift range, with a total of 346 clusters in $\sim164~deg^2$. We test three different mass models, and our best model includes a basic halo model, with a Navarro Frenk and White profile, and correction terms that take into account cluster miscentering, non-weak shear, the two-halo term, the contribution of the Brightest Cluster Galaxy, and an a posteriori correction for the intrinsic scatter in the mass-richness relation. With this model, we obtain a mass-richness relation of $\log{M_{\rm 200}/M_{\odot}}=(14.48\pm0.04)+(1.14\pm0.23)\log{(\lambda/40)}$ (statistical uncertainties). This result is consistent with other published lensing mass-richness relations. When compared to X-ray masses and mass proxies, we find that on average weak lensing masses are $\sim 10\%$ higher than those derived in the X-ray in the range $2\times10^{13}M_{\rm \odot}<E(z) M^{X}_{\rm 200}<2\times10^{14}M_{\rm \odot}$, in agreement with most previous results and simulations. We also give the coefficients of the scaling relations between the lensing mass and X-ray mass proxies, $L_X$ and $T_X$, and compare them with previous results.

This paper is the first of a series of papers constraining cosmological parameters with weak lensing peak statistics using $\sim 450~\rm deg^2$ of imaging data from the Kilo Degree Survey (KiDS-450). We measure high signal-to-noise ratio (SNR: $\nu$) weak lensing convergence peaks in the range of $3<\nu<5$, and employ theoretical models to derive expected values. These models are validated using a suite of simulations. We take into account two major systematic effects, the boost factor and the effect of baryons on the mass-concentration relation of dark matter haloes. In addition, we investigate the impacts of other potential astrophysical systematics including the projection effects of large scale structures, intrinsic galaxy alignments, as well as residual measurement uncertainties in the shear and redshift calibration. Assuming a flat $\Lambda$CDM model, we find constraints for $S_{\rm 8}=\sigma_{\rm 8}(\Omega_{\rm m}/0.3)^{0.5}=0.746^{+0.046}_{-0.107}$ according to the degeneracy direction of the cosmic shear analysis and $\Sigma_{\rm 8}=\sigma_{\rm 8}(\Omega_{\rm m}/0.3)^{0.38}=0.696^{+0.048}_{-0.050}$ based on the derived degeneracy direction of our high-SNR peak statistics. The difference between the power index of $S_{\rm 8}$ and in $\Sigma_{\rm 8}$ indicates that combining the two probes has the potential to break the degeneracy in $\sigma_{\rm 8}$ and $\Omega_{\rm m}$. Our results are consistent with the cosmic shear tomographic correlation analysis of the same dataset and $\sim 2\sigma$ lower than the Planck 2016 results.

We study the statistics of peaks in a weak lensing reconstructed mass map of the first 450 square degrees of the Kilo Degree Survey. The map is computed with aperture masses directly applied to the shear field with an NFW-like compensated filter. We compare the peak statistics in the observations with that of simulations for various cosmologies to constrain the cosmological parameter $S_8 = \sigma_8 \sqrt{\Omega_{\rm m}/0.3}$, which probes the ($\Omega_{\rm m}, \sigma_8$) plane perpendicularly to its main degeneracy. We estimate $S_8=0.750\pm0.059$, using peaks in the signal-to-noise range $0 \leq {\rm S/N} \leq 4$, and accounting for various systematics, such as multiplicative shear bias, mean redshift bias, baryon feedback, intrinsic alignment, and shear-position coupling. These constraints are $\sim25\%$ tighter than the constraints from the high significance peaks alone ($3 \leq {\rm S/N} \leq 4$) which typically trace single-massive halos. This demonstrates the gain of information from low-S/N peaks. However we find that including ${\rm S/N} < 0$ peaks does not add further information. Our results are in good agreement with the tomographic shear two-point correlation function measurement in KiDS-450. Combining shear peaks with non-tomographic measurements of the shear two-point correlation functions yields a $\sim20\%$ improvement in the uncertainty on $S_8$ compared to the shear two-point correlation functions alone, highlighting the great potential of peaks as a cosmological probe.

Measuring and calibrating relations between cluster observables is critical for resource-limited studies. The mass-richness relation of clusters offers an observationally inexpensive way of estimating masses. Its calibration is essential for cluster and cosmological studies, especially for high-redshift clusters. Weak gravitational lensing magnification is a promising and complementary method to shear studies, that can be applied at higher redshifts. We employed the weak lensing magnification method to calibrate the mass-richness relation up to a redshift of 1.4. We used the Spitzer Adaptation of the Red-Sequence Cluster Survey (SpARCS) galaxy cluster candidates ($0.2<z<1.4$) and optical data from the Canada France Hawaii Telescope (CFHT) to test whether magnification can be effectively used to constrain the mass of high-redshift clusters. Lyman-Break Galaxies (LBGs) selected using the $u$-band dropout technique and their colours were used as a background sample of sources. LBG positions were cross-correlated with the centres of the sample of SpARCS clusters to estimate the magnification signal measured for cluster sub-samples, binned in both redshift and richness. We detected a weak lensing magnification signal for all bins at a detection significance of 2.6-5.5$\sigma$. In particular, the significance of the measurement for clusters with $z>1.0$ is 4.1$\sigma$; for the entire cluster sample we obtained an average M$_{200}$ of $1.28^{+0.23}_{-0.21}$ $\times 10^{14} \, \textrm{M}_{\odot}$. Our measurements demonstrated the feasibility of using weak lensing magnification as a viable tool for determining the average halo masses for samples of high redshift galaxy clusters. The results also established the success of using galaxy over-densities to select massive clusters at $z > 1$. Additional studies are necessary for further modelling of the various systematic effects we discussed.

In recent years, multiple studies have reported substantial populations of large, low-surface-brightness galaxies in local galaxy clusters. Various theories that aim to explain the presence of such ultra-diffuse galaxies (UDGs) have since been proposed. A key question that will help to differentiate between models is whether UDGs have counterparts in lower-mass host haloes, and what their abundance as a function of halo mass is. In this study we extend our previous study of UDGs in galaxy clusters to galaxy groups. We measure the abundance of UDGs in 325 spectroscopically-selected groups from the Galaxy And Mass Assembly (GAMA) survey. We make use of the overlapping imaging from the ESO Kilo-Degree Survey (KiDS), from which we can identify galaxies with mean surface brightnesses within their effective radii down to ~25.5 mag arcsec$^{-2}$ in the r-band. We are able to measure a significant overdensity of UDGs (with sizes r_eff > 1.5 kpc) in galaxy groups down to M200=10^12 Msun, a regime where approximately only 1 in 10 groups contains a UDG that we can detect. We combine measurements of the abundance of UDGs in haloes that cover three orders of magnitude in halo mass, finding that their numbers scale quite steeply with halo mass; N_UDG (R<R200) $\propto$ M200^(1.11+/-0.07). To better interpret this, we also measure the mass-richness relation for brighter galaxies down to $M^*_r$+2.5 in the same GAMA groups, and find a much shallower relation of N_Bright (R<R200) $\propto$ M200^(0.78+/-0.05). This shows that UDGs are relatively more abundant, compared to bright galaxies, in massive clusters than in groups. We discuss implications, but whether this difference is related to a higher destruction rate of UDGs in groups, or whether massive haloes have a positive effect on their formation, is not yet clear.

We present measurements of the weak gravitational lensing shear power spectrum based on $450$ sq. deg. of imaging data from the Kilo Degree Survey. We employ a quadratic estimator in two and three redshift bins and extract band powers of redshift auto-correlation and cross-correlation spectra in the multipole range $76 \leq \ell \leq 1310$. The cosmological interpretation of the measured shear power spectra is performed in a Bayesian framework assuming a $\Lambda$CDM model with spatially flat geometry, while accounting for small residual uncertainties in the shear calibration and redshift distributions as well as marginalising over intrinsic alignments, baryon feedback and an excess-noise power model. Moreover, massive neutrinos are included in the modelling. The cosmological main result is expressed in terms of the parameter combination $S_8 \equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3}$ yielding $S_8 = \ 0.651 \pm 0.058$ (3 z-bins), confirming the recently reported tension in this parameter with constraints from Planck at $3.2\sigma$ (3 z-bins). We cross-check the results of the 3 z-bin analysis with the weaker constraints from the 2 z-bin analysis and find them to be consistent. The high-level data products of this analysis, such as the band power measurements, covariance matrices, redshift distributions, and likelihood evaluation chains are available at http://kids.strw.leidenuniv.nl/

We build a background cluster candidate catalog from the Next Generation Virgo Cluster Survey, using our detection algorithm RedGOLD. The NGVS covers 104$deg^2$ of the Virgo cluster in the $u^*,g,r,i,z$-bandpasses to a depth of $ g \sim 25.7$~mag (5$\sigma$). Part of the survey was not covered or has shallow observations in the $r$--band. We build two cluster catalogs: one using all bandpasses, for the fields with deep $r$--band observations ($\sim 20 \ deg^2$), and the other using four bandpasses ($u^*,g,i,z$) for the entire NGVS area. Based on our previous CFHT-LS W1 studies, we estimate that both of our catalogs are $\sim100\%$($\sim70\%$) complete and $\sim80\%$ pure, at $z\le 0.6$($z\lesssim1$), for galaxy clusters with masses of $M\gtrsim10^{14}\ M_{\odot}$. We show that when using four bandpasses, though the photometric redshift accuracy is lower, RedGOLD detects massive galaxy clusters up to $z\sim 1$ with completeness and purity similar to the five-band case. This is achieved when taking into account the bias in the richness estimation, which is $\sim40\%$ lower at $0.5\le z<0.6$ and $\sim20\%$ higher at $0.6<z< 0.8$, with respect to the five-band case. RedGOLD recovers all the X-ray clusters in the area with mass $M_{500} > 1.4 \times 10^{14} \rm M_{\odot}$ and $0.08<z<0.5$. Because of our different cluster richness limits and the NGVS depth, our catalogs reach to lower masses than the published redMaPPer cluster catalog over the area, and we recover $\sim 90-100\%$ of its detections.

The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products, adding further area, homogenized photometric calibration, photometric redshifts and weak lensing shear measurements to the first two releases. A dedicated pipeline embedded in the Astro-WISE information system is used for the production of the main release. Modifications with respect to earlier releases are described in detail. Photometric redshifts have been derived using both Bayesian template fitting, and machine-learning techniques. For the weak lensing measurements, optimized procedures based on the THELI data reduction and lensfit shear measurement packages are used. In DR3 stacked ugri images, weight maps, masks, and source lists for 292 new survey tiles (~300 sq.deg) are made available. The multi-band catalogue, including homogenized photometry and photometric redshifts, covers the combined DR1, DR2 and DR3 footprint of 440 survey tiles (447 sq.deg). Limiting magnitudes are typically 24.3, 25.1, 24.9, 23.8 (5 sigma in a 2 arcsec aperture) in ugri, respectively, and the typical r-band PSF size is less than 0.7 arcsec. The photometric homogenization scheme ensures accurate colors and an absolute calibration stable to ~2% for gri and ~3% in u. Separately released are a weak lensing shear catalogue and photometric redshifts based on two different machine-learning techniques.

We set out to quantify the number density of quiescent massive compact galaxies at intermediate redshifts. We determine structural parameters based on i-band imaging using the CFHT equatorial SDSS Stripe 82 (CS82) survey (~170 sq. degrees) taking advantage of an exquisite median seeing of ~0.6''. We select compact massive (M > 5x10^10 M_sun) galaxies within the redshift range of 0.2<z<0.6. The large volume sampled allows to decrease the effect of cosmic variance that has hampered the calculation of the number density for this enigmatic population in many previous studies. We undertake an exhaustive analysis in an effort to untangle the various findings inherent to the diverse definition of compactness present in the literature. We find that the absolute number of compact galaxies is very dependent on the adopted definition and can change up to a factor of >10. We systematically measure a factor of ~5 more compacts at the same redshift than what was previously reported on smaller fields with HST imaging, which are more affected by cosmic variance. This means that the decrease in number density from z ~ 1.5 to z ~ 0.2 might be only of a factor of ~2-5, significantly smaller than what previously reported. This supports progenitor bias as the main contributor to the size evolution. This milder decrease is roughly compatible with the predictions from recent numerical simulations. Only the most extreme compact galaxies, with Reff < 1.5x( M/10^11 M_sun)^0.75 and M > 10^10.7 M_sun, appear to drop in number by a factor of ~20 and hence likely experience a noticeable size evolution.

RedGOLD searches for red-sequence galaxy overdensities while minimizing contamination from dusty star-forming galaxies. It imposes an NFW profile and calculates cluster detection significance and richness. We optimize these latter two parameters using both simulations and X-ray detected cluster catalogs, and obtain a catalog $\sim 80\%$ pure up to $z \sim 1$, and $\sim 100\%$ ($\sim 70\%$) complete at $z\le 0.6$ ( $z\lesssim1$) for galaxy clusters with $M \gtrsim 10^{14}\ {\rm M_{\odot}}$ at the CFHT-LS Wide depth. In the CFHT-LS W1, we detect 11 cluster candidates per $\rm deg^2$ out to $z\sim1.1$. When we optimize both completeness and purity, RedGOLD obtains a cluster catalog with higher completeness and purity than other public catalogs, obtained using CFHT-LS W1 observations, for $M \gtrsim 10^{14}\ {\rm M_{\odot}}$. We use X-ray detected cluster samples to extend the study of the X-ray temperature-optical richness relation to a lower mass threshold, and find a mass scatter at fixed richness of $\sigma_{lnM|\lambda}=0.39\pm0.07$ and $\sigma_{lnM|\lambda}=0.30\pm0.13$ for the Gozaliasl et al. (2014) and Mehrtens et al. (2012) samples. When considering similar mass ranges as previous work, we recover a smaller scatter in mass at fixed richness. We recover $93\%$ of the redMaPPer detections, and find that its richness estimates is on average $\sim 40-50\%$ larger than ours at $z>0.3$. RedGOLD recovers X-ray cluster spectroscopic redshifts at better than $5\%$ up to $z\sim1$, and the centers within a few tens of arcseconds.

We test extensions to the standard cosmological model with weak gravitational lensing tomography using 450 deg$^2$ of imaging data from the Kilo Degree Survey (KiDS). In these extended cosmologies, which include massive neutrinos, nonzero curvature, evolving dark energy, modified gravity, and running of the scalar spectral index, we also examine the discordance between KiDS and cosmic microwave background measurements from Planck. The discordance between the two datasets is largely unaffected by a more conservative treatment of the lensing systematics and the removal of angular scales most sensitive to nonlinear physics. The only extended cosmology that simultaneously alleviates the discordance with Planck and is at least moderately favored by the data includes evolving dark energy with a time-dependent equation of state (in the form of the $w_0-w_a$ parameterization). In this model, the respective $S_8 = \sigma_8 \sqrt{\Omega_{\rm m}/0.3}$ constraints agree at the $1\sigma$ level, and there is `substantial concordance' between the KiDS and Planck datasets when accounting for the full parameter space. Moreover, the Planck constraint on the Hubble constant is wider than in LCDM and in agreement with the Riess et al. (2016) direct measurement of $H_0$. The dark energy model is moderately favored as compared to LCDM when combining the KiDS and Planck measurements, and remains moderately favored after including an informative prior on the Hubble constant. In both of these scenarios, marginalized constraints in the $w_0-w_a$ plane are discrepant with a cosmological constant at the $3\sigma$ level. Moreover, KiDS constrains the sum of neutrino masses to 4.0 eV (95% CL), finds no preference for time or scale dependent modifications to the metric potentials, and is consistent with flatness and no running of the spectral index. The analysis code is public at https://github.com/sjoudaki/kids450

We study the stellar-to-halo mass relation of central galaxies in the range 9.7<log_10(M_*/h^-2 M_sun)<11.7 and z<0.4, obtained from a combined analysis of the Kilo Degree Survey (KiDS) and the Galaxy And Mass Assembly (GAMA) survey. We use ~100 deg^2 of KiDS data to study the lensing signal around galaxies for which spectroscopic redshifts and stellar masses were determined by GAMA. We show that lensing alone results in poor constraints on the stellar-to-halo mass relation due to a degeneracy between the satellite fraction and the halo mass, which is lifted when we simultaneously fit the stellar mass function. At M_sun>5x10^10 h^-2 M_sun, the stellar mass increases with halo mass as ~M_h^0.25. The ratio of dark matter to stellar mass has a minimum at a halo mass of 8x10^11 h^-1 M_sun with a value of M_h/M_*=56_-10^+16 [h]. We also use the GAMA group catalogue to select centrals and satellites in groups with five or more members, which trace regions in space where the local matter density is higher than average, and determine for the first time the stellar-to-halo mass relation in these denser environments. We find no significant differences compared to the relation from the full sample, which suggests that the stellar-to-halo mass relation does not vary strongly with local density. Furthermore, we find that the stellar-to-halo mass relation of central galaxies can also be obtained by modelling the lensing signal and stellar mass function of satellite galaxies only, which shows that the assumptions to model the satellite contribution in the halo model do not significantly bias the stellar-to-halo mass relation. Finally, we show that the combination of weak lensing with the stellar mass function can be used to test the purity of group catalogues.

We constrain the average halo ellipticity of ~2 600 galaxy groups from the Galaxy And Mass Assembly (GAMA) survey, using the weak gravitational lensing signal measured from the overlapping Kilo Degree Survey (KiDS). To do so, we quantify the azimuthal dependence of the stacked lensing signal around seven different proxies for the orientation of the dark matter distribution, as it is a priori unknown which one traces the orientation best. On small scales, the major axis of the brightest group/cluster member (BCG) provides the best proxy, leading to a clear detection of an anisotropic signal. In order to relate that to a halo ellipticity, we have to adopt a model density profile. We derive new expressions for the quadrupole moments of the shear field given an elliptical model surface mass density profile. Modeling the signal with an elliptical Navarro-Frenk-White (NFW) profile on scales < 250 kpc, which roughly corresponds to half the virial radius, and assuming that the BCG is perfectly aligned with the dark matter, we find an average halo ellipticity of e_h=0.38 +/- 0.12. This agrees well with results from cold-dark-matter-only simulations, which typically report values of e_h ~ 0.3. On larger scales, the lensing signal around the BCGs does not trace the dark matter distribution well, and the distribution of group satellites provides a better proxy for the halo's orientation instead, leading to a 3--4 sigma detection of a non-zero halo ellipticity at scales between 250 kpc and 750 kpc. Our results suggest that the distribution of stars enclosed within a certain radius forms a good proxy for the orientation of the dark matter within that radius, which has also been observed in hydrodynamical simulations.

We investigate possible signatures of halo assembly bias for spectroscopically selected galaxy groups from the GAMA survey using weak lensing measurements from the spatially overlapping regions of the deeper, high-imaging-quality photometric KiDS survey. We use GAMA groups with an apparent richness larger than 4 to identify samples with comparable mean host halo masses but with a different radial distribution of satellite galaxies, which is a proxy for the formation time of the haloes. We measure the weak lensing signal for groups with a steeper than average and with a shallower than average satellite distribution and find no sign of halo assembly bias, with the bias ratio of $0.85^{+0.37}_{-0.25}$, which is consistent with the $\Lambda$CDM prediction. Our galaxy groups have typical masses of $10^{13} M_{\odot}/h$, naturally complementing previous studies of halo assembly bias on galaxy cluster scales.

We present the tomographic cross-correlation between galaxy lensing measured in the Kilo Degree Survey (KiDS-450) with overlapping lensing measurements of the cosmic microwave background (CMB), as detected by Planck 2015. We compare our joint probe measurement to the theoretical expectation for a flat $\Lambda$CDM cosmology, assuming the best-fitting cosmological parameters from the KiDS-450 cosmic shear and Planck CMB analyses. We find that our results are consistent within $1\sigma$ with the KiDS-450 cosmology, with an amplitude re-scaling parameter $A_{\rm KiDS} = 0.86 \pm 0.19$. Adopting a Planck cosmology, we find our results are consistent within $2\sigma$, with $A_{\it Planck} = 0.68 \pm 0.15$. We show that the agreement is improved in both cases when the contamination to the signal by intrinsic galaxy alignments is accounted for, increasing $A$ by $\sim 0.1$. This is the first tomographic analysis of the galaxy lensing -- CMB lensing cross-correlation signal, and is based on five photometric redshift bins. We use this measurement as an independent validation of the multiplicative shear calibration and of the calibrated source redshift distribution at high redshifts. We find that constraints on these two quantities are strongly correlated when obtained from this technique, which should therefore not be considered as a stand-alone competitive calibration tool.

We measure the cross-correlation between Fermi-LAT gamma-ray photons and over 1000 deg$^2$ of weak lensing data from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), the Red Cluster Sequence Lensing Survey (RCSLenS), and the Kilo Degree Survey (KiDS). We present the first measurement of tomographic weak lensing cross-correlations and the first application of spectral binning to cross-correlations between gamma rays and weak lensing. The measurements are performed using an angular power spectrum estimator while the covariance is estimated using an analytical prescription. We verify the accuracy of our covariance estimate by comparing it to two internal covariance estimators. Based on the non-detection of a cross-correlation signal, we derive constraints on weakly interacting massive particle (WIMP) dark matter. We compute exclusion limits on the dark matter annihilation cross-section $\langle\sigma_\rm{ann} v \rangle$, decay rate $\Gamma_\rm{dec}$, and particle mass $m_\rm{DM}$. We find that in the absence of a cross-correlation signal, tomography does not significantly improve the constraining power of the analysis. Assuming a strong contribution to the gamma-ray flux due to small-scale clustering of dark matter and accounting for known astrophysical sources of gamma rays, we exclude the thermal relic cross-section for particle masses of $m_\rm{DM}\lesssim 20$ GeV.

We present The-wiZZ, an open source and user-friendly software for estimating the redshift distributions of photometric galaxies with unknown redshifts by spatially cross-correlating them against a reference sample with known redshifts. The main benefit of The-wiZZ is in separating the angular pair finding and correlation estimation from the computation of the output clustering redshifts allowing anyone to create a clustering redshift for their sample without the intervention of an "expert". It allows the end user of a given survey to select any sub-sample of photometric galaxies with unknown redshifts, match this sample's catalog indices into a value-added data file, and produce a clustering redshift estimation for this sample in a fraction of the time it would take to run all the angular correlations needed to produce a clustering redshift. We show results with this software using photometric data from the Kilo-Degree Survey (KiDS) and spectroscopic redshifts from the Galaxy and Mass Assembly (GAMA) survey and the Sloan Digital Sky Survey (SDSS). The results we present for KiDS are consistent with the redshift distributions used in a recent cosmic shear analysis from the survey. We also present results using a hybrid machine learning-clustering redshift analysis that enables the estimation of clustering redshifts for individual galaxies. The-wiZZ can be downloaded at http://github.com/morriscb/The-wiZZ/.

We determine the accuracy of galaxy redshift distributions as estimated from photometric redshift probability distributions $p(z)$. Our method utilises measurements of the angular cross-correlation between photometric galaxies and an overlapping sample of galaxies with spectroscopic redshifts. We describe the redshift leakage from a galaxy photometric redshift bin $j$ into a spectroscopic redshift bin $i$ using the sum of the $p(z)$ for the galaxies residing in bin $j$. We can then predict the angular cross-correlation between photometric and spectroscopic galaxies due to intrinsic galaxy clustering when $i \neq j$ as a function of the measured angular cross-correlation when $i=j$. We also account for enhanced clustering arising from lensing magnification using a halo model. The comparison of this prediction with the measured signal provides a consistency check on the validity of using the summed $p(z)$ to determine galaxy redshift distributions in cosmological analyses, as advocated by the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). We present an analysis of the photometric redshifts measured by CFHTLenS, which overlaps the Baryon Oscillation Spectroscopic Survey (BOSS). We also analyse the Red-sequence Cluster Lensing Survey (RCSLenS), which overlaps both BOSS and the WiggleZ Dark Energy Survey. We find that the summed $p(z)$ from both surveys are generally biased with respect to the true underlying distributions. If unaccounted for, this bias would lead to errors in cosmological parameter estimation from CFHTLenS by less than $\sim 4\%$. For photometric redshift bins which spatially overlap in 3-D with our spectroscopic sample, we determine redshift bias corrections which can be used in future cosmological analyses that rely on accurate galaxy redshift distributions.

Verlinde (2016) proposed that the observed excess gravity in galaxies and clusters is the consequence of Emergent Gravity (EG). In this theory the standard gravitational laws are modified on galactic and larger scales due to the displacement of dark energy by baryonic matter. EG gives an estimate of the excess gravity (described as an apparent dark matter density) in terms of the baryonic mass distribution and the Hubble parameter. In this work we present the first test of EG using weak gravitational lensing, within the regime of validity of the current model. Although there is no direct description of lensing and cosmology in EG yet, we can make a reasonable estimate of the expected lensing signal of low redshift galaxies by assuming a background LambdaCDM cosmology. We measure the (apparent) average surface mass density profiles of 33,613 isolated central galaxies, and compare them to those predicted by EG based on the galaxies' baryonic masses. To this end we employ the ~180 square degrees overlap of the Kilo-Degree Survey (KiDS) with the spectroscopic Galaxy And Mass Assembly (GAMA) survey. We find that the prediction from EG, despite requiring no free parameters, is in good agreement with the observed galaxy-galaxy lensing profiles in four different stellar mass bins. Although this performance is remarkable, this study is only a first step. Further advancements on both the theoretical framework and observational tests of EG are needed before it can be considered a fully developed and solidly tested theory.

We present predictions for the galaxy-galaxy lensing profile from the EAGLE hydrodynamical cosmological simulation at redshift z=0.18, in the spatial range 0.02 < R/(Mpc/h) < 2, and for five logarithmically equi-spaced stellar mass bins in the range 10.3 < $\log_{10}$(Mstar/ $M_{\odot}$) < 11.8. We compare these excess surface density profiles to the observed signal from background galaxies imaged by the Kilo Degree Survey around spectroscopically confirmed foreground galaxies from the GAMA survey. Exploiting the GAMA galaxy group catalogue, the profiles of central and satellite galaxies are computed separately for groups with at least five members to minimise contamination. EAGLE predictions are in broad agreement with the observed profiles for both central and satellite galaxies, although the signal is underestimated at R$\approx$0.5-2 Mpc/h for the highest stellar mass bins. When central and satellite galaxies are considered simultaneously, agreement is found only when the selection function of lens galaxies is taken into account in detail. Specifically, in the case of GAMA galaxies, it is crucial to account for the variation of the fraction of satellite galaxies in bins of stellar mass induced by the flux-limited nature of the survey. We report the inferred stellar-to-halo mass relation and we find good agreement with recent published results. We note how the precision of the galaxy-galaxy lensing profiles in the simulation holds the potential to constrain fine-grained aspects of the galaxy-dark matter connection.

We present a new training set for estimating empirical photometric redshifts of galaxies, which was created as part of the 2dFLenS project. This training set is located in a 700 sq deg area of the KiDS South field and is randomly selected and nearly complete at r<19.5. We investigate the photometric redshift performance obtained with ugriz photometry from VST-ATLAS and W1/W2 from WISE, based on several empirical and template methods. The best redshift errors are obtained with kernel-density estimation, as are the lowest biases, which are consistent with zero within statistical noise. The 68th percentiles of the redshift scatter for magnitude-limited samples at r<(15.5, 17.5, 19.5) are (0.014, 0.017, 0.028). In this magnitude range, there are no known ambiguities in the colour-redshift map, consistent with a small rate of redshift outliers. In the fainter regime, the KDE method produces p(z) estimates per galaxy that represent unbiased and accurate redshift frequency expectations. The p(z) sum over any subsample is consistent with the true redshift frequency plus Poisson noise. Further improvements in redshift precision at r<20 would mostly be expected from filter sets with narrower passbands to increase the sensitivity of colours to small changes in redshift.

We present high signal-to-noise galaxy-galaxy lensing measurements of the BOSS CMASS sample using 250 square degrees of weak lensing data from CFHTLenS and CS82. We compare this signal with predictions from mock catalogs trained to match observables including the stellar mass function and the projected and two dimensional clustering of CMASS. We show that the clustering of CMASS, together with standard models of the galaxy-halo connection, robustly predicts a lensing signal that is 20-40% larger than observed. Detailed tests show that our results are robust to a variety of systematic effects. Lowering the value of $S_{\rm 8}=\sigma_{\rm 8} \sqrt{\Omega_{\rm m}/0.3}$ compared to Planck2015 reconciles the lensing with clustering. However, given the scale of our measurement ($r<10$ $h^{-1}$ Mpc), other effects may also be at play and need to be taken into consideration. We explore the impact of baryon physics, assembly bias, massive neutrinos, and modifications to general relativity on $\Delta\Sigma$ and show that several of these effects may be non-negligible given the precision of our measurement. Disentangling cosmological effects from the details of the galaxy-halo connection, the effects of baryons, and massive neutrinos, is the next challenge facing joint lensing and clustering analyses. This is especially true in the context of large galaxy samples from Baryon Acoustic Oscillation surveys with precise measurements but complex selection functions.

We develop a statistical estimator to infer the redshift probability distribution of a photometric sample of galaxies from its angular cross-correlation in redshift bins with an overlapping spectroscopic sample. This estimator is a minimum variance weighted quadratic function of the data: a quadratic estimator. This extends and modifies the methodology presented by McQuinn & White (2013). The derived source redshift distribution is degenerate with the source galaxy bias, which must be constrained via additional assumptions. We apply this estimator to constrain source galaxy redshift distributions in the Kilo-Degree imaging survey through cross-correlation with the spectroscopic 2-degree Field Lensing Survey, presenting results first as a binned step-wise distribution in the range z < 0.8, and then building a continuous distribution using a Gaussian process model. We demonstrate the robustness of our methodology using mock catalogues constructed from N-body simulations, and comparisons with other techniques for inferring the redshift distribution.

We investigate the impact of astrophysical systematics on cosmic shear cosmological parameter constraints from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), and the concordance with cosmic microwave background measurements by Planck. We present updated CFHTLenS cosmic shear tomography measurements extended to degree scales using a covariance calibrated by a new suite of N-body simulations. We analyze these measurements with a new model fitting pipeline, accounting for key systematic uncertainties arising from intrinsic galaxy alignments, baryonic effects in the nonlinear matter power spectrum, and photometric redshift uncertainties. We examine the impact of the systematic degrees of freedom on the cosmological parameter constraints, both independently and jointly. When the systematic uncertainties are considered independently, the intrinsic alignment amplitude is the only degree of freedom that is substantially preferred by the data. When the systematic uncertainties are considered jointly, there is no consistently strong preference in favor of the more complex models. We quantify the level of concordance between the CFHTLenS and Planck datasets by employing two distinct data concordance tests, grounded in Bayesian evidence and information theory. We find that the two data concordance tests largely agree with one another, and that the level of concordance between the CFHTLenS and Planck datasets is sensitive to the exact details of the systematic uncertainties included in our analysis, ranging from decisive discordance to substantial concordance as the treatment of the systematic uncertainties becomes more conservative. The least conservative scenario is the one most favored by the cosmic shear data, but it is also the one that shows the greatest degree of discordance with Planck. The data and analysis code are public at https://github.com/sjoudaki/cfhtlens_revisited

We present cosmological parameter constraints from a tomographic weak gravitational lensing analysis of ~450deg$^2$ of imaging data from the Kilo Degree Survey (KiDS). For a flat $\Lambda$CDM cosmology with a prior on $H_0$ that encompasses the most recent direct measurements, we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.745\pm0.039$. This result is in good agreement with other low redshift probes of large scale structure, including recent cosmic shear results, along with pre-Planck cosmic microwave background constraints. A $2.3$-$\sigma$ tension in $S_8$ and `substantial discordance' in the full parameter space is found with respect to the Planck 2015 results. We use shear measurements for nearly 15 million galaxies, determined with a new improved `self-calibrating' version of $lens$fit validated using an extensive suite of image simulations. Four-band $ugri$ photometric redshifts are calibrated directly with deep spectroscopic surveys. The redshift calibration is confirmed using two independent techniques based on angular cross-correlations and the properties of the photometric redshift probability distributions. Our covariance matrix is determined using an analytical approach, verified numerically with large mock galaxy catalogues. We account for uncertainties in the modelling of intrinsic galaxy alignments and the impact of baryon feedback on the shape of the non-linear matter power spectrum, in addition to the small residual uncertainties in the shear and redshift calibration. The cosmology analysis was performed blind. Our high-level data products, including shear correlation functions, covariance matrices, redshift distributions, and Monte Carlo Markov Chains are available at http://kids.strw.leidenuniv.nl.

We present the Red-sequence Cluster Lensing Survey (RCSLenS), an application of the methods developed for the Canada France Hawaii Telescope Lensing Survey (CFHTLenS) to the ~785deg$^2$, multi-band imaging data of the Red-sequence Cluster Survey 2 (RCS2). This project represents the largest public, sub-arcsecond seeing, multi-band survey to date that is suited for weak gravitational lensing measurements. With a careful assessment of systematic errors in shape measurements and photometric redshifts we extend the use of this data set to allow cross-correlation analyses between weak lensing observables and other data sets. We describe the imaging data, the data reduction, masking, multi-colour photometry, photometric redshifts, shape measurements, tests for systematic errors, and a blinding scheme to allow for more objective measurements. In total we analyse 761 pointings with r-band coverage, which constitutes our lensing sample. Residual large-scale B-mode systematics prevent the use of this shear catalogue for cosmic shear science. The effective number density of lensing sources over an unmasked area of 571.7deg$^2$ and down to a magnitude limit of r~24.5 is 8.1 galaxies per arcmin$^2$ (weighted: 5.5 arcmin$^{-2}$) distributed over 14 patches on the sky. Photometric redshifts based on 4-band griz data are available for 513 pointings covering an unmasked area of 383.5 deg$^2$ We present weak lensing mass reconstructions of some example clusters as well as the full survey representing the largest areas that have been mapped in this way. All our data products are publicly available through CADC at http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/en/community/rcslens/query.html in a format very similar to the CFHTLenS data release.

We present measurements of the spatial mapping between (hot) baryons and the total matter in the Universe, via the cross-correlation between the thermal Sunyaev-Zeldovich (tSZ) map from Planck and the weak gravitational lensing maps from the Red Sequence Cluster Survey (RCSLenS). The cross-correlations are performed on the map level where all the sources (including diffuse intergalactic gas) contribute to the signal. We consider two configuration-space correlation function estimators, $\xi^{ y-\kappa}$ and $\xi^ {y-\gamma_{t}}$, and a Fourier space estimator, $C_{\ell}^{y-\kappa}$, in our analysis. We detect a significant correlation out to three degrees of angular separation on the sky. Based on statistical noise only, we can report 13$\sigma$ and 17$\sigma$ detections of the cross-correlation using the configuration-space $y-\kappa$ and $y-\gamma_{t}$ estimators, respectively. Including a heuristic estimate of the sampling variance yields a detection significance of 6$\sigma$ and 8$\sigma$, respectively. A similar level of detection is obtained from the Fourier-space estimator, $C_{\ell}^{y-\kappa}$. As each estimator probes different dynamical ranges, their combination improves the significance of the detection. We compare our measurements with predictions from the cosmo-OWLS suite of cosmological hydrodynamical simulations, where different galactic feedback models are implemented. We find that a model with considerable AGN feedback that removes large quantities of hot gas from galaxy groups and WMAP-7yr best-fit cosmological parameters provides the best match to the measurements. All baryonic models in the context of a Planck cosmology over-predict the observed signal. Similar cosmological conclusions are drawn when we employ a halo model with the observed `universal' pressure profile.

We present the 2-degree Field Lensing Survey (2dFLenS), a new galaxy redshift survey performed at the Anglo-Australian Telescope. 2dFLenS is the first wide-area spectroscopic survey specifically targeting the area mapped by deep-imaging gravitational lensing fields, in this case the Kilo-Degree Survey. 2dFLenS obtained 70,079 redshifts in the range z < 0.9 over an area of 731 sq deg, and is designed to extend the datasets available for testing gravitational physics and promote the development of relevant algorithms for joint imaging and spectroscopic analysis. The redshift sample consists first of 40,531 Luminous Red Galaxies (LRGs), which enable analyses of galaxy-galaxy lensing, redshift-space distortion, and the overlapping source redshift distribution by cross-correlation. An additional 28,269 redshifts form a magnitude-limited (r < 19.5) nearly-complete sub-sample, allowing direct source classification and photometric-redshift calibration. In this paper, we describe the motivation, target selection, spectroscopic observations, and clustering analysis of 2dFLenS. We use power spectrum multipole measurements to fit the redshift-space distortion parameter of the LRG sample in two redshift ranges 0.15 < z < 0.43 and 0.43 < z < 0.7 as beta = 0.49 +/- 0.15 and beta = 0.26 +/- 0.09, respectively. These values are consistent with those obtained from LRGs in the Baryon Oscillation Spectroscopic Survey. 2dFLenS data products will be released via our website http://2dflens.swin.edu.au.

We measure the cross-correlation signature between the Planck CMB lensing map and the weak lensing observations from both the Red-sequence Cluster Lensing Survey (RCSLenS) and the Canada-France-Hawai Telescope Lensing Survey (CFHTLenS). In addition to a Fourier analysis, we include the first configuration-space detection, based on the estimators $\langle \kappa_{\rm CMB} \kappa_{\rm gal} \rangle$ and $\langle \kappa_{\rm CMB} \gamma_{t} \rangle$. Combining 747.2 deg$^2$ from both surveys, we find a detection significance that exceeds $4.2\sigma$ in both Fourier- and configuration-space analyses. Scaling the predictions by a free parameter $A$, we obtain $A^{\rm Planck}_{\rm CFHT}= 0.68\pm 0.31 $ and $A^{\rm Planck}_{\rm RCS}= 1.31\pm 0.33$. In preparation for the next generation of measurements similar to these, we quantify the impact of different analysis choices on these results. First, since none of these estimators probes the exact same dynamical range, we improve our detection by combining them. Second, we carry out a detailed investigation on the effect of apodization, zero-padding and mask multiplication, validated on a suite of high-resolution simulations, and find that the latter produces the largest systematic bias in the cosmological interpretation. Finally, we show that residual contamination from intrinsic alignment and the effect of photometric redshift error are both largely degenerate with the characteristic signal from massive neutrinos, however the signature of baryon feedback might be easier to distinguish. The three lensing datasets are now publicly available.

Galaxies and their dark matter haloes are part of a complex network of mass structures, collectively called the cosmic web. Using the tidal tensor prescription these structures can be classified into four cosmic environments: voids, sheets, filaments and knots. As the cosmic web may influence the formation and evolution of dark matter haloes and the galaxies they host, we aim to study the effect of these cosmic environments on the average mass of galactic haloes. To this end we measure the galaxy-galaxy lensing profile of 91,195 galaxies, within 0.039 < z < 0.263, from the spectroscopic Galaxy And Mass Assembly (GAMA) survey, using ~100 square degrees of overlapping data from the Kilo-Degree Survey (KiDS). In each of the four cosmic environments we model the contributions from group centrals, satellites and neighbouring groups to the stacked galaxy-galaxy lensing profiles. After correcting the lens samples for differences in the stellar mass distribution, we find no dependence of the average halo mass of central galaxies on their cosmic environment. We do find a significant increase in the average contribution of neighbouring groups to the lensing profile in increasingly dense cosmic environments. We show, however, that the observed effect can be entirely attributed to the galaxy density at much smaller scales (within 4 Mpc/h), which is correlated with the density of the cosmic environments. Within our current uncertainties we find no direct dependence of galaxy halo mass on their cosmic environment.

We present measurements of the galaxy bias $b$ and the galaxy-matter cross-correlation coefficient $r$ for the BOSS LOWZ luminous red galaxy sample. Using a new statistical weak lensing analysis of the Red Sequence Cluster Lensing Survey (RCSLenS) we find the bias properties of this sample to be higher than previously reported with $b=2.45^{+0.05}_{-0.05}$ and $r=1.64^{+0.17}_{-0.16}$ on scales between $3'$ and $20'$. We repeat the measurement for angular scales of $20'\leq \vartheta \leq70'$, which yields $b=2.39^{+0.07}_{-0.07}$ and $r=1.24^{+0.26}_{-0.25}$. This is the first application of a data compression analysis using a complete set of discrete estimators for galaxy-galaxy lensing and galaxy clustering. As cosmological data sets grow, our new method of data compression will become increasingly important in order to interpret joint weak lensing and galaxy clustering measurements and to estimate the data covariance. In future studies this formalism can be used as a tool to study the large-scale structure of the Universe to yield a precise determination of cosmological parameters.

The unknown nature of dark energy motivates continued cosmological tests of large-scale gravitational physics. We present a new consistency check based on the relative amplitude of non-relativistic galaxy peculiar motions, measured via redshift-space distortion, and the relativistic deflection of light by those same galaxies traced by galaxy-galaxy lensing. We take advantage of the latest generation of deep, overlapping imaging and spectroscopic datasets, combining the Red Cluster Sequence Lensing Survey (RCSLenS), the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), the WiggleZ Dark Energy Survey and the Baryon Oscillation Spectroscopic Survey (BOSS). We quantify the results using the "gravitational slip" statistic E_G, which we estimate as 0.48 +/- 0.10 at z=0.32 and 0.30 +/- 0.07 at z=0.57, the latter constituting the highest redshift at which this quantity has been determined. These measurements are consistent with the predictions of General Relativity, for a perturbed Friedmann-Robertson-Walker metric in a Universe dominated by a cosmological constant, which are E_G = 0.41 and 0.36 at these respective redshifts. The combination of redshift-space distortion and gravitational lensing data from current and future galaxy surveys will offer increasingly stringent tests of fundamental cosmology.

In this paper we present results of applying the shear-ratio method to the RCSLenS data. The method takes the ratio of the mean of the weak lensing tangential shear signal about galaxy clusters, averaged over all clusters of the same redshift, in multiple background redshift bins. In taking a ratio the mass-dependency of the shear signal is cancelled-out leaving a statistic that is dependent on the geometric part of the lensing kernel only. We apply this method to 535 clusters and measure a cosmology-independent distance-redshift relation to redshifts z~1. In combination with Planck data the method lifts the degeneracies in the CMB measurements, resulting in cosmological parameter constraints of OmegaM=0.31 +/- 0.10 and w0 = -1.02 +/- 0.37, for a flat wCDM cosmology.

Flux magnification is an interesting complement to shear-based lensing measurements, especially at high redshift where sources are harder to resolve. One measures either changes in the source density (magnification bias) or in the shape of the flux distribution (e.g. magnitude-shift). The interpretation of these measurements relies on theoretical estimates of how the observables change under magnification. Here we present simulations to create multi-band photometric mock catalogues of Lyman-break galaxies in a CFHTLenS-like survey that include several observational effects that can change these relations, making simple theoretical estimates unusable. In particular, we show how the magnification bias can be affected by photometric noise, colour selection, and dust extinction. We find that a simple measurement of the slope of the number-counts is not sufficient for the precise interpretation of virtually all observations of magnification bias. We also explore how sensitive the shift in the mean magnitude of a source sample in different photometric bands is to magnification including the same observational effects. Again we find significant deviations from simple analytical estimates. We also discover a wavelength-dependence of the magnitude-shift effect when applied to a colour-selected noisy source sample. Such an effect can mimic the reddening by dust in the lens. It has to be disentangled from the dust extinction before the magnitude-shift/colour-excess can be used to measure the distribution of either dark matter or extragalactic dust. Using simulations like the ones presented here these observational effects can be studied and eventually removed from observations making precise measurements of flux magnification possible.

We present weak lensing constraints on the ellipticity of galaxy-scale matter haloes and the galaxy-halo misalignment. Using data from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), we measure the weighted-average ratio of the aligned projected ellipticity components of galaxy matter haloes and their embedded galaxies, $f_\mathrm{h}$, split by galaxy type. We then compare our observations to measurements taken from the Millennium Simulation, assuming different models of galaxy-halo misalignment. Using the Millennium Simulation we verify that the statistical estimator used removes contamination from cosmic shear. We also detect an additional signal in the simulation, which we interpret as the impact of intrinsic shape-shear alignments between the lenses and their large-scale structure environment. These alignments are likely to have caused some of the previous observational constraints on $f_\mathrm{h}$ to be biased high. From CFHTLenS we find $f_\mathrm{h}=-0.04 \pm 0.25$ for early-type galaxies, which is consistent with current models for the galaxy-halo misalignment predicting $f_\mathrm{h}\simeq 0.20$. For late-type galaxies we measure $f_\mathrm{h}=0.69_{-0.36}^{+0.37}$ from CFHTLenS. This can be compared to the simulated results which yield $f_\mathrm{h}\simeq 0.02$ for misaligned late-type models.

Dark energy may be the first sign of new fundamental physics in the Universe, taking either a physical form or revealing a correction to Einsteinian gravity. Weak gravitational lensing and galaxy peculiar velocities provide complementary probes of General Relativity, and in combination allow us to test modified theories of gravity in a unique way. We perform such an analysis by combining measurements of cosmic shear tomography from the Canada-France Hawaii Telescope Lensing Survey (CFHTLenS) with the growth of structure from the WiggleZ Dark Energy Survey and the Six-degree-Field Galaxy Survey (6dFGS), producing the strongest existing joint constraints on the metric potentials that describe general theories of gravity. For scale-independent modifications to the metric potentials which evolve linearly with the effective dark energy density, we find present-day cosmological deviations in the Newtonian potential and curvature potential from the prediction of General Relativity to be (Delta Psi)/Psi = 0.05 \pm 0.25 and (Delta Phi)/Phi = -0.05 \pm 0.3 respectively (68 per cent CL).

The Kilo-Degree Survey (KiDS) is a multi-band imaging survey designed for cosmological studies from weak lensing and photometric redshifts. It uses the ESO VLT Survey Telescope with its wide-field camera OmegaCAM. KiDS images are taken in four filters similar to the SDSS ugri bands. The best-seeing time is reserved for deep r-band observations that reach a median 5-sigma limiting AB magnitude of 24.9 with a median seeing that is better than 0.7arcsec. Initial KiDS observations have concentrated on the GAMA regions near the celestial equator, where extensive, highly complete redshift catalogues are available. A total of 109 survey tiles, one square degree each, form the basis of the first set of lensing analyses, which focus on measurements of halo properties of GAMA galaxies. 9 galaxies per square arcminute enter the lensing analysis, for an effective inverse shear variance of 69 per square arcminute. Accounting for the shape measurement weight, the median redshift of the sources is 0.53. KiDS data processing follows two parallel tracks, one optimized for galaxy shape measurement (for weak lensing), and one for accurate matched-aperture photometry in four bands (for photometric redshifts). This technical paper describes how the lensing and photometric redshift catalogues have been produced (including an extensive description of the Gaussian Aperture and Photometry pipeline), summarizes the data quality, and presents extensive tests for systematic errors that might affect the lensing analyses. We also provide first demonstrations of the suitability of the data for cosmological measurements, and explain how the shear catalogues were blinded to prevent confirmation bias in the scientific analyses. The KiDS shear and photometric redshift catalogues, presented in this paper, are released to the community through http://kids.strw.leidenuniv.nl .

We present an investigation into the effects of survey systematics such as varying depth, point spread function (PSF) size, and extinction on the galaxy selection and correlation in photometric, multi-epoch, wide area surveys. We take the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS) as an example. Variations in galaxy selection due to systematics are found to cause density fluctuations of up to 10% for some small fraction of the area for most galaxy redshift slices and as much as 50% for some extreme cases of faint high-redshift samples. This results in correlations of galaxies against survey systematics of order $\sim$1% when averaged over the survey area. We present an empirical method for mitigating these systematic correlations from measurements of angular correlation functions using weighted random points. These weighted random catalogs are estimated from the observed galaxy over densities by mapping these to survey parameters. We are able to model and mitigate the effect of systematic correlations allowing for non-linear dependencies of density on systematics. Applied to CFHTLenS we find that the method reduces spurious correlations in the data by a factor two for most galaxy samples and as much as an order of magnitude in others. Such a treatment is particularly important for an unbiased estimation of very small correlation signals, as e.g. from weak gravitational lensing magnification bias. We impose a criterion for using a galaxy sample in a magnification measurement of the majority of the systematic correlations show improvement and are less than 10% of the expected magnification signal when combined in the galaxy cross correlation. After correction the galaxy samples in CFHTLenS satisfy this criterion for $z_{\rm phot}<0.9$ and will be used in a future analysis of magnification.

Calibrating the photometric redshifts of >10^9 galaxies for upcoming weak lensing cosmology experiments is a major challenge for the astrophysics community. The path to obtaining the required spectroscopic redshifts for training and calibration is daunting, given the anticipated depths of the surveys and the difficulty in obtaining secure redshifts for some faint galaxy populations. Here we present an analysis of the problem based on the self-organizing map, a method of mapping the distribution of data in a high-dimensional space and projecting it onto a lower-dimensional representation. We apply this method to existing photometric data from the COSMOS survey selected to approximate the anticipated Euclid weak lensing sample, enabling us to robustly map the empirical distribution of galaxies in the multidimensional color space defined by the expected Euclid filters. Mapping this multicolor distribution lets us determine where - in galaxy color space - redshifts from current spectroscopic surveys exist and where they are systematically missing. Crucially, the method lets us determine whether a spectroscopic training sample is representative of the full photometric space occupied by the galaxies in a survey. We explore optimal sampling techniques and estimate the additional spectroscopy needed to map out the color-redshift relation, finding that sampling the galaxy distribution in color space in a systematic way can efficiently meet the calibration requirements. While the analysis presented here focuses on the Euclid survey, similar analysis can be applied to other surveys facing the same calibration challenge, such as DES, LSST, and WFIRST.

We use the first 100 sq. deg. of overlap between the Kilo-Degree Survey (KiDS) and the Galaxy And Mass Assembly (GAMA) survey to determine the galaxy halo mass of ~10,000 spectroscopically-confirmed satellite galaxies in massive ($M > 10^{13}h^{-1}{\rm M}_\odot$) galaxy groups. Separating the sample as a function of projected distance to the group centre, we jointly model the satellites and their host groups with Navarro-Frenk-White (NFW) density profiles, fully accounting for the data covariance. The probed satellite galaxies in these groups have total masses $\log M_{\rm sub} /(h^{-1}{\rm M}_\odot) \approx 11.7 - 12.2$ consistent across group-centric distance within the errorbars. Given their typical stellar masses, $\log M_{\rm \star,sat}/(h^{-2}{\rm M}_\odot) \sim 10.5$, such total masses imply stellar mass fractions of $M_{\rm \star,sat} /M_{\rm sub} \approx 0.04 h^{-1}$ . The average subhalo hosting these satellite galaxies has a mass $M_{\rm sub} \sim 0.015M_{\rm host}$ independent of host halo mass, in broad agreement with the expectations of structure formation in a $\Lambda$CDM universe.

We have discovered an optically rich galaxy cluster at z=1.7089 with star formation occurring in close proximity to the central galaxy. The system, SpARCS104922.6+564032.5, was detected within the Spitzer Adaptation of the red-sequence Cluster Survey, (SpARCS), and confirmed through Keck-MOSFIRE spectroscopy. The rest-frame optical richness of Ngal(500kpc) = 30+/-8 implies a total halo mass, within 500kpc, of ~3.8+/-1.2 x 10^14 Msun, comparable to other clusters at or above this redshift. There is a wealth of ancillary data available, including Canada-France-Hawaii Telescope optical, UKIRT-K, Spitzer-IRAC/MIPS, and Herschel-SPIRE. This work adds submillimeter imaging with the SCUBA2 camera on the James Clerk Maxwell Telescope and near-infrared imaging with the Hubble Space Telescope (HST). The mid/far-infrared (M/FIR) data detect an Ultra-luminous Infrared Galaxy spatially coincident with the central galaxy, with LIR = 6.2+/-0.9 x 10^12 Lsun. The detection of polycyclic aromatic hydrocarbons (PAHs) at z=1.7 in a Spitzer-IRS spectrum of the source implies the FIR luminosity is dominated by star formation (an Active Galactic Nucleus contribution of 20%) with a rate of ~860+/-30 Msun/yr. The optical source corresponding to the IR emission is likely a chain of of > 10 individual clumps arranged as "beads on a string" over a linear scale of 66 kpc. Its morphology and proximity to the Brightest Cluster Galaxy imply a gas-rich interaction at the center of the cluster triggered the star formation. This system indicates that wet mergers may be an important process in forming the stellar mass of BCGs at early times.

The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the survey is mapping the large-scale matter distribution in the Universe back to a redshift of ~0.5. Secondary science cases are manifold, covering topics such as galaxy evolution, Milky Way structure, and the detection of high-redshift clusters and quasars. KiDS is an ESO Public Survey and dedicated to serving the astronomical community with high-quality data products derived from the survey data, as well as with calibration data. Public data releases will be made on a yearly basis, the first two of which are presented here. For a total of 148 survey tiles (~160 sq.deg.) astrometrically and photometrically calibrated, coadded ugri images have been released, accompanied by weight maps, masks, source lists, and a multi-band source catalog. A dedicated pipeline and data management system based on the Astro-WISE software system, combined with newly developed masking and source classification software, is used for the data production of the data products described here. The achieved data quality and early science projects based on the data products in the first two data releases are reviewed in order to validate the survey data. Early scientific results include the detection of nine high-z QSOs, fifteen candidate strong gravitational lenses, high-quality photometric redshifts and galaxy structural parameters for hundreds of thousands of galaxies. (Abridged)

The Kilo-Degree Survey (KiDS) is an optical wide-field survey designed to map the matter distribution in the Universe using weak gravitational lensing. In this paper, we use these data to measure the density profiles and masses of a sample of $\sim \mathrm{1400}$ spectroscopically identified galaxy groups and clusters from the Galaxy And Mass Assembly (GAMA) survey. We detect a highly significant signal (signal-to-noise-ratio $\sim$ 120), allowing us to study the properties of dark matter haloes over one and a half order of magnitude in mass, from $M \sim 10^{13}-10^{14.5} h^{-1}\mathrm{M_{\odot}}$. We interpret the results for various subsamples of groups using a halo model framework which accounts for the mis-centring of the Brightest Cluster Galaxy (used as the tracer of the group centre) with respect to the centre of the group's dark matter halo. We find that the density profiles of the haloes are well described by an NFW profile with concentrations that agree with predictions from numerical simulations. In addition, we constrain scaling relations between the mass and a number of observable group properties. We find that the mass scales with the total r-band luminosity as a power-law with slope $1.16 \pm 0.13$ (1-sigma) and with the group velocity dispersion as a power-law with slope $1.89 \pm 0.27$ (1-sigma). Finally, we demonstrate the potential of weak lensing studies of groups to discriminate between models of baryonic feedback at group scales by comparing our results with the predictions from the Cosmo-OverWhelmingly Large Simulations (Cosmo-OWLS) project, ruling out models without AGN feedback.

Finding a sample of the most massive clusters with redshifts $z>0.6$ can provide an interesting consistency check of the $\Lambda$ cold dark matter ($\Lambda$CDM) model. Here we present results from our search for clusters with $0.6\lesssim z\lesssim1.0$ where the initial candidates were selected by cross-correlating the RASS faint and bright source catalogues with red galaxies from the Sloan Digital Sky Survey DR8. Our survey thus covers $\approx10,000\,\rm{deg^2}$, much larger than previous studies of this kind. Deeper follow-up observations in three bands using the William Herschel Telescope and the Large Binocular Telescope were performed to confirm the candidates, resulting in a sample of 44 clusters for which we present richnesses and red sequence redshifts, as well as spectroscopic redshifts for a subset. At least two of the clusters in our sample are comparable in richness to RCS2-$J$232727.7$-$020437, one of the richest systems discovered to date. We also obtained new observations with the Combined Array for Research in Millimeter Astronomy for a subsample of 21 clusters. For 11 of those we detect the Sunyaev-Zel'dovich effect signature. The Sunyaev-Zel'dovich signal allows us to estimate $M_{200}$ and check for tension with the cosmological standard model. We find no tension between our cluster masses and the $\Lambda$CDM model.

We present weak lensing and X-ray analysis of 12 low mass clusters from the CFHTLenS and XMM-CFHTLS surveys. We combine these systems with high-mass systems from CCCP and low-mass systems from COSMOS to obtain a sample of 70 systems, spanning over two orders of magnitude in mass. We measure core-excised Lx-Tx, M-Lx and M-Tx scaling relations and include corrections for observational biases. By providing fully bias corrected relations, we give the current limitations for Lx and Tx as cluster mass proxies. We demonstrate that Tx benefits from a significantly lower intrinsic scatter at fixed mass than Lx. By studying the residuals of the bias corrected relations, we show for the first time using weak lensing masses that galaxy groups seem more luminous and warmer for their mass than clusters. This implies a steepening of the M-Lx and M-Tx relations at low masses. We verify the inferred steepening using a different high mass sample from the literature and show that variance between samples is the dominant effect leading to discrepant scaling relations. We divide our sample into subsamples of merging and relaxed systems, and find that mergers may have enhanced scatter in lensing measurements, most likely due to stronger triaxiality and more substructure. For the Lx-Tx relation, which is unaffected by lensing measurements, we find the opposite trend in scatter. We also explore the effects of X-ray cross-calibration and find that Chandra calibration leads to flatter Lx-Tx and M-Tx relations than XMM-Newton.

We present a detailed gravitational lens model of the galaxy cluster RCS2 J232727.6-020437. Due to cosmological dimming of cluster members and ICL, its high redshift ($z=0.6986$) makes it ideal for studying background galaxies. Using new ACS and WFC3/IR HST data, we identify 16 multiple images. From MOSFIRE follow up, we identify a strong emission line in the spectrum of one multiple image, likely confirming the redshift of that system to $z=2.083$. With a highly magnified ($\mu\gtrsim2$) source plane area of $\sim0.7$ arcmin$^2$ at $z=7$, RCS2 J232727.6-020437 has a lensing efficiency comparable to the Hubble Frontier Fields clusters. We discover four highly magnified $z\sim7$ candidate Lyman-break galaxies behind the cluster, one of which may be multiply-imaged. Correcting for magnification, we find that all four galaxies are fainter than $0.5 L_{\star}$. One candidate is detected at ${>10\sigma}$ in both Spitzer/IRAC [3.6] and [4.5] channels. A spectroscopic follow-up with MOSFIRE does not result in the detection of the Lyman-alpha emission line from any of the four candidates. From the MOSFIRE spectra we place median upper limits on the Lyman-alpha flux of $5-14 \times 10^{-19}\, \mathrm{erg \,\, s^{-1} cm^{-2}}$ ($5\sigma$).

We study the correlations of the shear signal between triplets of sources in the Canada-France-Hawaii Lensing Survey (CFHTLenS) to probe cosmological parameters via the matter bispectrum. In contrast to previous studies, we adopted a non-Gaussian model of the data likelihood which is supported by our simulations of the survey. We find that for state-of-the-art surveys, similar to CFHTLenS, a Gaussian likelihood analysis is a reasonable approximation, albeit small differences in the parameter constraints are already visible. For future surveys we expect that a Gaussian model becomes inaccurate. Our algorithm for a refined non-Gaussian analysis and data compression is then of great utility especially because it is not much more elaborate if simulated data are available. Applying this algorithm to the third-order correlations of shear alone in a blind analysis, we find a good agreement with the standard cosmological model: $\Sigma_8$=$\sigma_8$ $(\Omega_{\rm m}/0.27)^{0.64}$=$0.79^{+0.08}_{-0.11}$ for a flat $\Lambda\rm CDM$ cosmology with $h=0.7\pm0.04$ ($68\%$ credible interval). Nevertheless our models provide only moderately good fits as indicated by $\chi^2/{\rm dof}=2.9$, including a $20\%$ r.m.s. uncertainty in the predicted signal amplitude. The models cannot explain a signal drop on scales around 15 arcmin, which may be caused by systematics. It is unclear whether the discrepancy can be fully explained by residual PSF systematics of which we find evidence at least on scales of a few arcmin. Therefore we need a better understanding of higher-order correlations of cosmic shear and their systematics to confidently apply them as cosmological probes.

We present new constraints on the relationship between galaxies and their host dark matter halos, measured from the location of the peak of the stellar-to-halo mass ratio (SHMR), up to the most massive galaxy clusters at redshift $z\sim0.8$ and over a volume of nearly 0.1~Gpc$^3$. We use a unique combination of deep observations in the CFHTLenS/VIPERS field from the near-UV to the near-IR, supplemented by $\sim60\,000$ secure spectroscopic redshifts, analysing galaxy clustering, galaxy-galaxy lensing and the stellar mass function. We interpret our measurements within the halo occupation distribution (HOD) framework, separating the contributions from central and satellite galaxies. We find that the SHMR for the central galaxies peaks at $M_{\rm h, peak} = 1.9^{+0.2}_{-0.1}\times10^{12} M_{\odot}$ with an amplitude of $0.025$, which decreases to $\sim0.001$ for massive halos ($M_{\rm h} > 10^{14} M_{\odot}$). Compared to central galaxies only, the total SHMR (including satellites) is boosted by a factor 10 in the high-mass regime (cluster-size halos), a result consistent with cluster analyses from the literature based on fully independent methods. After properly accounting for differences in modelling, we have compared our results with a large number of results from the literature up to $z=1$: we find good general agreement, independently of the method used, within the typical stellar-mass systematic errors at low to intermediate mass (${M}_{\star} < 10^{11} M_{\odot}$) and the statistical errors above. We have also compared our SHMR results to semi-analytic simulations and found that the SHMR is tilted compared to our measurements in such a way that they over- (under-) predict star formation efficiency in central (satellite) galaxies.

This paper presents the first application of 3D cosmic shear to a wide-field weak lensing survey. 3D cosmic shear is a technique that analyses weak lensing in three dimensions using a spherical harmonic approach, and does not bin data in the redshift direction. This is applied to CFHTLenS, a 154 square degree imaging survey with a median redshift of 0.7 and an effective number density of 11 galaxies per square arcminute usable for weak lensing. To account for survey masks we apply a 3D pseudo-Cl approach on weak lensing data, and to avoid uncertainties in the highly non-linear regime, we separately analyse radial wave numbers k<=1.5h/Mpc and k<=5.0h/Mpc, and angular wavenumbers l~400-5000. We show how one can recover 2D and tomographic power spectra from the full 3D cosmic shear power spectra and present a measurement of the 2D cosmic shear power spectrum, and measurements of a set of 2-bin and 6-bin cosmic shear tomographic power spectra; in doing so we find that using the 3D power in the calculation of such 2D and tomographic power spectra from data naturally accounts for a minimum scale in the matter power spectrum. We use 3D cosmic shear to constrain cosmologies with parameters OmegaM, OmegaB, sigma8, h, ns, w0, wa. For a non-evolving dark energy equation of state, and assuming a flat cosmology, lensing combined with WMAP7 results in h=0.78+/-0.12, OmegaM=0.252+/-0.079, sigma8=0.88+/-0.23 and w=-1.16+/-0.38 using only scales k<=1.5h/Mpc. We also present results of lensing combined with first year Planck results, where we find no tension with the results from this analysis, but we also find no significant improvement over the Planck results alone. We find evidence of a suppression of power compared to LCDM on small scales 1.5 < k < 5.0 h/Mpc in the lensing data, which is consistent with predictions of the effect of baryonic feedback on the matter power spectrum.

We measure the cross-correlation of cosmic microwave background lensing convergence maps derived from Atacama Cosmology Telescope data with galaxy lensing convergence maps as measured by the Canada-France-Hawaii Telescope Stripe 82 Survey. The CMB-galaxy lensing cross power spectrum is measured for the first time with a significance of 4.2{\sigma}, which corresponds to a 12% constraint on the amplitude of density fluctuations at redshifts ~ 0.9. With upcoming improved lensing data, this novel type of measurement will become a powerful cosmological probe, providing a precise measurement of the mass distribution at intermediate redshifts and serving as a calibrator for systematic biases in weak lensing measurements.

We present the cluster mass-richness scaling relation calibrated by a weak lensing analysis of >18000 galaxy cluster candidates in the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). Detected using the 3D-Matched-Filter cluster-finder of Milkeraitis et al., these cluster candidates span a wide range of masses, from the small group scale up to $\sim10^{15} M_{\odot}$, and redshifts 0.2 $\lesssim z\lesssim$ 0.9. The total significance of the stacked shear measurement amounts to 54$\sigma$. We compare cluster masses determined using weak lensing shear and magnification, finding the measurements in individual richness bins to yield 1$\sigma$ compatibility, but with magnification estimates biased low. This first direct mass comparison yields important insights for improving the systematics handling of future lensing magnification work. In addition, we confirm analyses that suggest cluster miscentring has an important effect on the observed 3D-MF halo profiles, and we quantify this by fitting for projected cluster centroid offsets, which are typically $\sim$ 0.4 arcmin. We bin the cluster candidates as a function of redshift, finding similar cluster masses and richness across the full range up to $z \sim$ 0.9. We measure the 3D-MF mass-richness scaling relation $M_{200} = M_0 (N_{200} / 20)^\beta$. We find a normalization $M_0 \sim (2.7^{+0.5}_{-0.4}) \times 10^{13} M_{\odot}$, and a logarithmic slope of $\beta \sim 1.4 \pm 0.1$, both of which are in 1$\sigma$ agreement with results from the magnification analysis. We find no evidence for a redshift-dependence of the normalization. The CFHTLenS 3D-MF cluster catalogue is now available at cfhtlens.org.

Galaxy-galaxy weak lensing is a direct probe of the mean matter distribution around galaxies. The depth and sky coverage of the CFHT Legacy Survey yield statistically significant galaxy halo mass measurements over a much wider range of stellar masses ($10^{8.75}$ to $10^{11.3} M_{\odot}$) and redshifts ($0.2 < z < 0.8$) than previous weak lensing studies. At redshift $z \sim 0.5$, the stellar-to-halo mass ratio (SHMR) reaches a maximum of $4.0\pm0.2$ percent as a function of halo mass at $\sim 10^{12.25} M_{\odot}$. We find, for the first time from weak lensing alone, evidence for significant evolution in the SHMR: the peak ratio falls as a function of cosmic time from $4.5 \pm 0.3$ percent at $z \sim 0.7$ to $3.4 \pm 0.2$ percent at $z \sim 0.3$, and shifts to lower stellar mass haloes. These evolutionary trends are dominated by red galaxies, and are consistent with a model in which the stellar mass above which star formation is quenched "downsizes" with cosmic time. In contrast, the SHMR of blue, star-forming galaxies is well-fit by a power law that does not evolve with time. This suggests that blue galaxies form stars at a rate that is balanced with their dark matter accretion in such a way that they evolve along the SHMR locus. The redshift dependence of the SHMR can be used to constrain the evolution of the galaxy population over cosmic time.

The Next Generation Virgo Cluster Survey is an optical imaging survey covering 104 deg^2 centered on the Virgo cluster. Currently, the complete survey area has been observed in the u*giz-bands and one third in the r-band. We present the photometric redshift estimation for the NGVS background sources. After a dedicated data reduction, we perform accurate photometry, with special attention to precise color measurements through point spread function-homogenization. We then estimate the photometric redshifts with the Le Phare and BPZ codes. We add a new prior which extends to iAB = 12.5 mag. When using the u*griz-bands, our photometric redshifts for 15.5 \le i \lesssim 23 mag or zphot \lesssim 1 galaxies have a bias |\Delta z| < 0.02, less than 5% outliers, and a scatter \sigma_{outl.rej.} and an individual error on zphot that increase with magnitude (from 0.02 to 0.05 and from 0.03 to 0.10, respectively). When using the u*giz-bands over the same magnitude and redshift range, the lack of the r-band increases the uncertainties in the 0.3 \lesssim zphot \lesssim 0.8 range (-0.05 < \Delta z < -0.02, \sigma_{outl.rej} ~ 0.06, 10-15% outliers, and zphot.err. ~ 0.15). We also present a joint analysis of the photometric redshift accuracy as a function of redshift and magnitude. We assess the quality of our photometric redshifts by comparison to spectroscopic samples and by verifying that the angular auto- and cross-correlation function w(\theta) of the entire NGVS photometric redshift sample across redshift bins is in agreement with the expectations.

The Canada-France-Hawaii Telescope Legacy Survey (CFHTLS) represents the most powerful weak lensing survey carried out to date. The CFHTLenS (Canada-France-Hawaii Telescope Lensing Survey) team was formed in 2008 to analyse the data from the CFHTLS focussing on a rigorous treatment of systematic effects in shape measurements and photometric redshifts. Here we review the technical challenges that we faced in analysing these data and their solutions which set the current standard for weak lensing analyses. We also present some science highlights that were made possible by this effort including cosmic shear tomography, tests for modified gravity models, and the mapping of dark matter structures over unprecedentedly large scales. An outlook is given on current and future surveys that are analysed with the tools prepared for CFHTLenS. CFHTLenS represents the first and only weak lensing data set that has been made publicly available so far. We encourage other surveys to follow this example.

Higher-order, non-Gaussian aspects of the large-scale structure carry valuable information on structure formation and cosmology, which is complementary to second-order statistics. In this work we measure second- and third-order weak-lensing aperture-mass moments from CFHTLenS and combine those with CMB anisotropy probes. The third moment is measured with a significance of $2\sigma$. The combined constraint on $\Sigma_8 = \sigma_8 (\Omega_{\rm m}/0.27)^\alpha$ is improved by 10%, in comparison to the second-order only, and the allowed ranges for $\Omega_{\rm m}$ and $\sigma_8$ are substantially reduced. Including general triangles of the lensing bispectrum yields tighter constraints compared to probing mainly equilateral triangles. Second- and third-order CFHTLenS lensing measurements improve Planck CMB constraints on $\Omega_{\rm m}$ and $\sigma_8$ by 26% for flat $\Lambda$CDM. For a model with free curvature, the joint CFHTLenS-Planck result is $\Omega_{\rm m} = 0.28 \pm 0.02$ (68% confidence), which is an improvement of 43% compared to Planck alone. We test how our results are potentially subject to three astrophysical sources of contamination: source-lens clustering, the intrinsic alignment of galaxy shapes, and baryonic effects. We explore future limitations of the cosmological use of third-order weak lensing, such as the nonlinear model and the Gaussianity of the likelihood function.

SURFSUP is a joint Spitzer and HST Exploration Science program using 10 galaxy clusters as cosmic telescopes to study z >~ 7 galaxies at intrinsically lower luminosities, enabled by gravitational lensing, than blank field surveys of the same exposure time. Our main goal is to measure stellar masses and ages of these galaxies, which are the most likely sources of the ionizing photons that drive reionization. Accurate knowledge of the star formation density and star formation history at this epoch is necessary to determine whether these galaxies indeed reionized the universe. Determination of the stellar masses and ages requires measuring rest frame optical light, which only Spitzer can probe for sources at z >~ 7, for a large enough sample of typical galaxies. Our program consists of 550 hours of Spitzer/IRAC imaging covering 10 galaxy clusters with very well-known mass distributions, making them extremely precise cosmic telescopes. We combine our data with archival observations to obtain mosaics with ~30 hours exposure time in both 3.6$\mu$m and 4.5$\mu$m in the central 4 arcmin x 4 arcmin field and ~15 hours in the flanking fields. This results in 3-$\sigma$ sensitivity limits of ~26.6 and ~26.2AB magnitudes for the central field in the IRAC 3.6 and 4.5$\mu$m bands, respectively. To illustrate the survey strategy and characteristics we introduce the sample, present the details of the data reduction and demonstrate that these data are sufficient for in-depth studies of z >~ 7 sources (using a z=9.5 galaxy behind MACSJ1149.5+2223 as an example). For the first cluster of the survey (the Bullet Cluster) we have released all high-level data mosaics and IRAC empirical PSF models. In the future we plan to release these data products for the entire survey.

Gravitational lensing magnification is measured with a significance of 9.7 sigma on a large sample of galaxy clusters in the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). This survey covers ~154 deg^2 and contains over 18,000 cluster candidates at redshifts 0.2 <= z <= 0.9, detected using the 3D-Matched Filter cluster-finder of Milkeraitis et al. (2010). We fit composite-NFW models to the ensemble, accounting for cluster miscentering, source-lens redshift overlap, as well as nearby structure (the 2-halo term), and recover mass estimates of the cluster dark matter halos in range of ~10^13 M_sun to 2*10^14 M_sun. Cluster richness is measured for the entire sample, and we bin the clusters according to both richness and redshift. A mass-richness relation M_200 = M_0 (N_200 / 20)^beta is fit to the measurements. For two different cluster miscentering models we find consistent results for the normalization and slope, M_0 = (2.3 +/- 0.2)*10^13 M_sun, beta = 1.4 +/- 0.1 and M_0 = (2.2 +/- 0.2)*10^13 M_sun, beta = 1.5 +/- 0.1. We find that accounting for the full redshift distribution of lenses and sources is important, since any overlap can have an impact on mass estimates inferred from flux magnification.

In this paper, we motivate the use of galaxy clustering measurements using photometric redshift information, including a contribution from flux magnification, as a probe of cosmology. We present cosmological forecasts when clustering data alone is used, and when clustering is combined with a cosmic shear analysis. We consider two types of clustering analysis: firstly, clustering with only redshift auto-correlations in tomographic redshift bins; secondly, using all available redshift bin correlations. Finally, we consider how inferred cosmological parameters may be biased using each analysis when flux magnification is neglected. Results are presented for a Stage III ground-based survey, and a Stage IV space-based survey modelled with photometric redshift errors, and values for the slope of the luminosity function inferred from CFHTLenS catalogues. We find that combining clustering information with shear can improve constraints on cosmological parameters, giving an improvement to a Dark Energy Task Force-like figure of merit by a factor of 1.33 when only auto-correlations in redshift are used for the clustering analysis, rising to 1.52 when cross-correlations in redshift are also included. The addition of galaxy-galaxy lensing gives further improvement, with increases in figure of merit by a factor of 2.82 and 3.7 for each type of clustering analysis respectively. The presence of flux magnification in a clustering analysis does not significantly affect the precision of cosmological constraints when combined with cosmic shear and galaxy-galaxy lensing. However if magnification is neglected, inferred cosmological parameter values are biased, with biases in some cosmological parameters larger than statistical errors. (Abridged)

We present a study of the relation between dark matter halo mass and the baryonic content of host galaxies, quantified via luminosity and stellar mass. Our investigation uses 154 deg2 of Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS) lensing and photometric data, obtained from the CFHT Legacy Survey. We employ a galaxy-galaxy lensing halo model which allows us to constrain the halo mass and the satellite fraction. Our analysis is limited to lenses at redshifts between 0.2 and 0.4. We express the relationship between halo mass and baryonic observable as a power law. For the luminosity-halo mass relation we find a slope of 1.32+/-0.06 and a normalisation of 1.19+0.06-0.07x10^13 h70^-1 Msun for red galaxies, while for blue galaxies the best-fit slope is 1.09+0.20-0.13 and the normalisation is 0.18+0.04-0.05x10^13 h70^-1 Msun. Similarly, we find a best-fit slope of 1.36+0.06-0.07 and a normalisation of 1.43+0.11-0.08x10^13 h70^-1 Msun for the stellar mass-halo mass relation of red galaxies, while for blue galaxies the corresponding values are 0.98+0.08-0.07 and 0.84+0.20-0.16x10^13 h70^-1 Msun. For red lenses, the fraction which are satellites tends to decrease with luminosity and stellar mass, with the sample being nearly all satellites for a stellar mass of 2x10^9 h70^-2 Msun. The satellite fractions are generally close to zero for blue lenses, irrespective of luminosity or stellar mass. This, together with the shallower relation between halo mass and baryonic tracer, is a direct confirmation from galaxy-galaxy lensing that blue galaxies reside in less clustered environments than red galaxies. We also find that the halo model, while matching the lensing signal around red lenses well, is prone to over-predicting the large-scale signal for faint and less massive blue lenses. This could be a further indication that these galaxies tend to be more isolated than assumed. [abridged]

Large sets of objects with spectroscopic redshift measurements will be needed for imaging dark energy experiments to achieve their full potential, serving two goals:_training_, i.e., the use of objects with known redshift to develop and optimize photometric redshift algorithms; and_calibration_, i.e., the characterization of moments of redshift (or photo-z error) distributions. Better training makes cosmological constraints from a given experiment stronger, while highly-accurate calibration is needed for photo-z systematics not to dominate errors. In this white paper, we investigate the required scope of spectroscopic datasets which can serve both these purposes for ongoing and next-generation dark energy experiments, as well as the time required to obtain such data with instruments available in the next decade. Large time allocations on kilo-object spectrographs will be necessary, ideally augmented by infrared spectroscopy from space. Alternatively, precision calibrations could be obtained by measuring cross-correlation statistics using samples of bright objects from a large baryon acoustic oscillation experiment such as DESI. We also summarize the additional work on photometric redshift methods needed to prepare for ongoing and future dark energy experiments.

We present a quantitative analysis of the largest contiguous maps of projected mass density obtained from gravitational lensing shear. We use data from the 154 deg2 covered by the Canada-France-Hawaii Telescope Lensing Survey. Our study is the first attempt to quantitatively characterize the scientific value of lensing maps, which could serve in the future as a complementary approach to the study of the dark universe with gravitational lensing. We show that mass maps contain unique cosmological information beyond that of traditional two-points statistical analysis techniques. Using a series of numerical simulations, we first show how, reproducing the CFHTLenS observing conditions, gravitational lensing inversion provides a reliable estimate of the projected matter distribution of large scale structure. We validate our analysis by quantifying the robustness of the maps with various statistical estimators. We then apply the same process to the CFHTLenS data. We find that the 2-points correlation function of the projected mass is consistent with the cosmological analysis performed on the shear correlation function discussed in the CFHTLenS companion papers. The maps also lead to a significant measurement of the third order moment of the projected mass, which is in agreement with analytic predictions, and to a marginal detection of the fourth order moment. Tests for residual systematics are found to be consistent with zero for the statistical estimators we used. A new approach for the comparison of the reconstructed mass map to that predicted from the galaxy distribution reveals the existence of giant voids in the dark matter maps as large as 3 degrees on the sky. Our analysis shows that lensing mass maps can be used for new techniques such as peak statistics and the morphological analysis of the projected dark matter distribution.

Major astrophysical questions related to the formation and evolution of structures, and more specifically of galaxy groups and clusters, will still be open in the coming decade and beyond: what is the interplay of galaxy, supermassive black hole, and intergalactic gas evolution in the most massive objects in the Universe - galaxy groups and clusters? What are the processes driving the evolution of chemical enrichment of the hot diffuse gas in large-scale structures? How and when did the first galaxy groups in the Universe, massive enough to bind more than 10^7 K gas, form? Focussing on the period when groups and clusters assembled (0.5<z<2.5), we show that, due to the continuum and line emission of this hot intergalactic gas at X-ray wavelengths, Athena+, combining high sensitivity with excellent spectral and spatial resolution, will deliver breakthrough observations in view of the aforementioned issues. Indeed, the physical and chemical properties of the hot intra-cluster gas, and their evolution across time, are a key to understand the co-evolution of galaxy and supermassive black hole within their environments.

We present the stellar mass functions (SMFs) of star-forming and quiescent galaxies from observations of 10 rich clusters in the Gemini Cluster Astrophysics Spectroscopic Survey (GCLASS) in the redshift range 0.86<z<1.34. We compare our results with field measurements at similar redshifts using data from a Ks-band selected catalogue of the COSMOS/UltraVISTA field. We construct a Ks-band selected multi-colour catalogue for the clusters in 11 photometric bands covering u-8um, and estimate photometric redshifts and stellar masses using SED fitting techniques. To correct for interlopers in our cluster sample, we use the deep spectroscopic component of GCLASS, which contains spectra for 1282 identified cluster and field galaxies taken with Gemini/GMOS. Both the photometric and spectroscopic samples are sufficiently deep that we can probe the SMF down to masses of 10^10 Msun. We distinguish between star-forming and quiescent galaxies using the rest-frame U-V versus V-J diagram, and find that the best-fitting Schechter parameters alpha and M* are similar within the uncertainties for these galaxy types within the different environments. However, there is a significant difference in the shape and normalisation of the total SMF between the clusters and the field sample. This difference in the total SMF is primarily a reflection of the increased fraction of quiescent galaxies in high-density environments. We apply a simple quenching model that includes components of mass- and environment-driven quenching, and find that in this picture 45% of the star-forming galaxies, which normally would be forming stars in the field, are quenched by the cluster. If galaxies in clusters and the field quench their star formation via different mechanisms, these processes have to conspire in such a way that the shapes of the quiescent and star-forming SMF remain similar in these different environments.

We present a finely-binned tomographic weak lensing analysis of the Canada-France-Hawaii Telescope Lensing Survey, CFHTLenS, mitigating contamination to the signal from the presence of intrinsic galaxy alignments via the simultaneous fit of a cosmological model and an intrinsic alignment model. CFHTLenS spans 154 square degrees in five optical bands, with accurate shear and photometric redshifts for a galaxy sample with a median redshift of zm =0.70. We estimate the 21 sets of cosmic shear correlation functions associated with six redshift bins, each spanning the angular range of 1.5<theta<35 arcmin. We combine this CFHTLenS data with auxiliary cosmological probes: the cosmic microwave background with data from WMAP7, baryon acoustic oscillations with data from BOSS, and a prior on the Hubble constant from the HST distance ladder. This leads to constraints on the normalisation of the matter power spectrum sigma_8 = 0.799 +/- 0.015 and the matter density parameter Omega_m = 0.271 +/- 0.010 for a flat Lambda CDM cosmology. For a flat wCDM cosmology we constrain the dark energy equation of state parameter w = -1.02 +/- 0.09. We also provide constraints for curved Lambda CDM and wCDM cosmologies. We find the intrinsic alignment contamination to be galaxy-type dependent with a significant intrinsic alignment signal found for early-type galaxies, in contrast to the late-type galaxy sample for which the intrinsic alignment signal is found to be consistent with zero.

We use weak gravitational lensing to analyse the dark matter halos around satellite galaxies in galaxy groups in the CFHTLenS dataset. This dataset is derived from the CFHTLS-Wide survey, and encompasses 154 sq. deg of high-quality shape data. Using the photometric redshifts, we divide the sample of lens galaxies with stellar masses in the range 10^9 Msun to 10^10.5 Msun into those likely to lie in high-density environments (HDE) and those likely to lie in low-density environments (LDE). Through comparison with galaxy catalogues extracted from the Millennium Simulation, we show that the sample of HDE galaxies should primarily (~61%) consist of satellite galaxies in groups, while the sample of LDE galaxies should consist of mostly (~87%) non-satellite (field and central) galaxies. Comparing the lensing signals around samples of HDE and LDE galaxies matched in stellar mass, the lensing signal around HDE galaxies clearly shows a positive contribution from their host groups on their lensing signals at radii of ~500--1000 kpc, the typical separation between satellites and group centres. More importantly, the subhalos of HDE galaxies are less massive than those around LDE galaxies by a factor 0.65 +/- 0.12, significant at the 2.9 sigma level. A natural explanation is that the halos of satellite galaxies are stripped through tidal effects in the group environment. Our results are consistent with a typical tidal truncation radius of ~40 kpc.

We present the first direct measurement of the galaxy-matter bispectrum as a function of galaxy luminosity, stellar mass, and SED type. Our analysis uses a galaxy-galaxy-galaxy lensing technique (G3L), on angular scales between 9 arcsec to 50 arcmin, to quantify (i) the excess surface mass density around galaxy pairs (excess mass hereafter) and (ii) the excess shear-shear correlations around single galaxies, both of which yield a measure of two types of galaxy-matter bispectra. We apply our method to the state-of-the-art Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS), spanning 154 square degrees. This survey allows us to detect a significant change of the bispectra with lens properties. Measurements for lens populations with distinct redshift distributions become comparable by a newly devised normalisation technique. That will also aid future comparisons to other surveys or simulations. A significant dependence of the normalised G3L statistics on luminosity within -23<M_r<-18 and stellar mass within 5x10^9 M_sol<M_star<2x10^11 M_sol is found (h=0.73). Both bispectra exhibit a stronger signal for more luminous lenses or those with higher stellar mass (up to a factor 2-3). This is accompanied by a steeper equilateral bispectrum for more luminous or higher stellar mass lenses for the excess mass. Importantly, we find the excess mass to be very sensitive to galaxy type as recently predicted with semi-analytic galaxy models: luminous (M_r<-21) late-type galaxies show no detectable signal, while all excess mass detected for luminous galaxies seems to be associated with early-type galaxies. We also present the first observational constraints on third-order stochastic galaxy biasing parameters.

We present cosmological constraints from 2D weak gravitational lensing by the large-scale structure in the Canada-France Hawaii Telescope Lensing Survey (CFHTLenS) which spans 154 square degrees in five optical bands. Using accurate photometric redshifts and measured shapes for 4.2 million galaxies between redshifts of 0.2 and 1.3, we compute the 2D cosmic shear correlation function over angular scales ranging between 0.8 and 350 arcmin. Using non-linear models of the dark-matter power spectrum, we constrain cosmological parameters by exploring the parameter space with Population Monte Carlo sampling. The best constraints from lensing alone are obtained for the small-scale density-fluctuations amplitude sigma_8 scaled with the total matter density Omega_m. For a flat LambdaCDM model we obtain sigma_8(Omega_m/0.27)^0.6 = 0.79+-0.03. We combine the CFHTLenS data with WMAP7, BOSS and an HST distance-ladder prior on the Hubble constant to get joint constraints. For a flat LambdaCDM model, we find Omega_m = 0.283+-0.010 and sigma_8 = 0.813+-0.014. In the case of a curved wCDM universe, we obtain Omega_m = 0.27+-0.03, sigma_8 = 0.83+-0.04, w_0 = -1.10+-0.15 and Omega_K = 0.006+0.006-0.004. We calculate the Bayesian evidence to compare flat and curved LambdaCDM and dark-energy CDM models. From the combination of all four probes, we find models with curvature to be at moderately disfavoured with respect to the flat case. A simple dark-energy model is indistinguishable from LambdaCDM. Our results therefore do not necessitate any deviations from the standard cosmological model.

The Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS) comprises deep multi-colour (u*g'r'i'z') photometry spanning 154 square degrees, with accurate photometric redshifts and shape measurements. We demonstrate that the redshift probability distribution function summed over galaxies provides an accurate representation of the galaxy redshift distribution accounting for random and catastrophic errors for galaxies with best fitting photometric redshifts z_p < 1.3. We present cosmological constraints using tomographic weak gravitational lensing by large-scale structure. We use two broad redshift bins 0.5 < z_p <= 0.85 and 0.85 < z_p <= 1.3 free of intrinsic alignment contamination, and measure the shear correlation function on angular scales in the range ~1-40 arcmin. We show that the problematic redshift scaling of the shear signal, found in previous CFHTLS data analyses, does not afflict the CFHTLenS data. For a flat Lambda-CDM model and a fixed matter density Omega_m=0.27, we find the normalisation of the matter power spectrum sigma_8=0.771 \pm 0.041. When combined with cosmic microwave background data (WMAP7), baryon acoustic oscillation data (BOSS), and a prior on the Hubble constant from the HST distance ladder, we find that CFHTLenS improves the precision of the fully marginalised parameter estimates by an average factor of 1.5-2. Combining our results with the above cosmological probes, we find Omega_m=0.2762 \pm 0.0074 and sigma_8=0.802 \pm 0.013.

Dust emission at sub-millimetre wavelengths allows us to trace the early phases of star formation in the Universe. In order to understand the physical processes involved in this mode of star formation, it is essential to gain knowledge about the dark matter structures - most importantly their masses - that sub-millimetre galaxies live in. Here we use the magnification effect of gravitational lensing to determine the average mass and dust content of sub-millimetre galaxies with 250mu flux densities of S_250>15mJy selected using data from the Herschel Multi-tiered Extragalactic Survey. The positions of hundreds of sub-millimetre foreground lenses are cross-correlated with the positions of background Lyman-break galaxies at z~3-5 selected using optical data from the Canada-France Hawaii Telescope Legacy Survey. We detect a cross-correlation signal at the 7-sigma level over a sky area of one square degree, with ~80% of this signal being due to magnification, whereas the remaining ~20% comes from dust extinction. Adopting some simple assumptions for the dark matter and dust profiles and the redshift distribution enables us to estimate the average mass of the halos hosting the sub-millimetre galaxies to be log(M_200/M_sun)=13.17+0.05-0.08(stat.) and their average dust mass fraction (at radii of >10kpc) to be M_dust/M_200~6x10^-5. This supports the picture that sub-millimetre galaxies are dusty, forming stars at a high rate, reside in massive group-sized halos, and are a crucial phase in the assembly and evolution of structure in the Universe.

A likelihood-based method for measuring weak gravitational lensing shear in deep galaxy surveys is described and applied to the Canada-France-Hawaii Telescope (CFHT) Lensing Survey (CFHTLenS). CFHTLenS comprises 154 sq deg of multicolour optical data from the CFHT Legacy Survey, with lensing measurements being made in the i' band to a depth i'(AB)<24.7, for galaxies with signal-to-noise ratio greater than about 10. The method is based on the lensfit algorithm described in earlier papers, but here we describe a full analysis pipeline that takes into account the properties of real surveys. The method creates pixel-based models of the varying point spread function (PSF) in individual image exposures. It fits PSF-convolved two-component (disk plus bulge) models, to measure the ellipticity of each galaxy, with bayesian marginalisation over model nuisance parameters of galaxy position, size, brightness and bulge fraction. The method allows optimal joint measurement of multiple, dithered image exposures, taking into account imaging distortion and the alignment of the multiple measurements. We discuss the effects of noise bias on the likelihood distribution of galaxy ellipticity. Two sets of image simulations that mirror the observed properties of CFHTLenS have been created, to establish the method's accuracy and to derive an empirical correction for the effects of noise bias.

We present data products from the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS). CFHTLenS is based on the Wide component of the Canada-France-Hawaii Telescope Legacy Survey (CFHTLS). It encompasses 154 deg^2 of deep, optical, high-quality, sub-arcsecond imaging data in the five optical filters u^*g'r'i'z'. The article presents our data processing of the complete CFHTLenS data set. We were able to obtain a data set with very good image quality and high-quality astrometric and photometric calibration. Our external astrometric accuracy is between 60-70 mas with respect to SDSS data and the internal alignment in all filters is around 30 mas. Our average photometric calibration shows a dispersion on the order of 0.01 to 0.03 mag for g'r'i'z' and about 0.04 mag for u^* with respect to SDSS sources down to i <= 21. In the spirit of the CFHTLS all our data products are released to the astronomical community via the Canadian Astronomy Data Centre. We give a description and how-to manuals of the public products which include image pixel data, source catalogues with photometric redshift estimates and all relevant quantities to perform weak lensing studies.

We present the Canada-France-Hawaii Telescope Lensing Survey (CFHTLenS) that accurately determines a weak gravitational lensing signal from the full 154 square degrees of deep multi-colour data obtained by the CFHT Legacy Survey. Weak gravitational lensing by large-scale structure is widely recognised as one of the most powerful but technically challenging probes of cosmology. We outline the CFHTLenS analysis pipeline, describing how and why every step of the chain from the raw pixel data to the lensing shear and photometric redshift measurement has been revised and improved compared to previous analyses of a subset of the same data. We present a novel method to identify data which contributes a non-negligible contamination to our sample and quantify the required level of calibration for the survey. Through a series of cosmology-insensitive tests we demonstrate the robustness of the resulting cosmic shear signal, presenting a science-ready shear and photometric redshift catalogue for future exploitation.

We report on the detection of gravitational lensing magnification by a population of galaxy groups, at a significance level of 4.9 sigma. Using X-ray selected groups in the COSMOS 1.64 deg^2 field, and high-redshift Lyman break galaxies as sources, we measure a lensing-induced angular cross-correlation between the samples. After satisfying consistency checks that demonstrate we have indeed detected a magnification signal, and are not suffering from contamination by physical overlap of samples, we proceed to implement an optimally weighted cross-correlation function to further boost the signal to noise of the measurement. Interpreting this optimally weighted measurement allows us to study properties of the lensing groups. We model the full distribution of group masses using a composite-halo approach, considering both the singular isothermal sphere and Navarro-Frenk-White profiles, and find our best fit values to be consistent with those recovered using the weak-lensing shear technique. We argue that future weak-lensing studies will need to incorporate magnification along with shear, both to reduce residual systematics and to make full use of all available source information, in an effort to maximize scientific yield of the observations.

Here we present the results of various approaches to measure accurate colours and photometric redshifts (photo-z's) from wide-field imaging data. We use data from the Canada-France-Hawaii-Telescope Legacy Survey (CFHTLS) which have been re- processed by the CFHT Lensing Survey (CFHTLenS) team in order to carry out a number of weak gravitational lensing studies. An emphasis is put on the correction of systematic effects in the photo-z's arising from the different Point Spread Functions (PSF) in the five optical bands. Different ways of correcting these effects are discussed and the resulting photo-z accuracies are quantified by comparing the photo-z's to large spectroscopic redshift (spec-z) data sets. Careful homogenisation of the PSF between bands leads to increased overall accuracy of photo-z's. The gain is particularly pronounced at fainter magnitudes where galaxies are smaller and flux measurements are affected more by PSF-effects. We also study possible re- calibrations of the photometric zeropoints (ZPs) with the help of galaxies with known spec-z's. We find that if PSF-effects are properly taken into account, a re-calibration of the ZPs becomes much less important suggesting that previous such re-calibrations described in the literature could in fact be mostly corrections for PSF-effects rather than corrections for real inaccuracies in the ZPs. The implications of this finding for future surveys like KiDS, DES, LSST, or Euclid are mixed. On the one hand, ZP re-calibrations with spec-z's might not be as accurate as previously thought. On the other hand, careful PSF homogenisation might provide a way out and yield accurate, homogeneous photometry without the need for full spectroscopic coverage. This is the first paper in a series describing the technical aspects of CFHTLenS. (abridged)

In this paper we study the large scale structures and their galaxy content around the most X-ray luminous cluster known, RX J1347.5-1145 at z=0.45. We make use of ugriz CFHT MEGACAM photometry and VIMOS VLT spectroscopy to identify structures around the RXJ1347 on a scale of 20x20 Mpc2. We construct maps of the galaxy distribution and the fraction of blue galaxies. We study the photometric galaxy properties as a function of environment, traced by the galaxy density. We identify group candidates based on galaxy overdensities and study their galaxy content. We also use available GALEX NUV imaging to identify strong unobscured star forming galaxies. We find that the large scale structure around RXJ1347 extends in the NE-SW direction for at least 20 Mpc, in which most of the group candidates are located. As other studies, we find that the fraction of blue galaxies (Fblue) is a function of galaxy number density, but the bulk of the trend is due to galaxies belonging to massive systems. The fraction of the UV-bright galaxies is also function of environment, but their relative numbers compared to the blue population seems to be constant regardless of the environment. These UV emitters also have similar properties at all galaxy densities, indicating that the transition between galaxy types occurs in short time-scales. Candidate galaxy groups show a large variation in their galaxy content and Fblue in those groups display little dependence with galaxy density. This may indicate possible differences in their evolutionary status or the processes that are acting in groups are different than in clusters. The large scale structure around rich clusters are dynamic places for galaxy evolution. In the case of RXJ1347 the transformation may start within infalling groups to finish with the removal of the cold gas once galaxies are accreted in massive systems. (ABRIDGED)

We introduce a novel method to measure the masses of galaxy clusters at high redshift selected from optical and IR Spitzer data via the red-sequence technique. Lyman-break galaxies are used as a well understood, high-redshift background sample allowing mass measurements of lenses at unprecedented high redshifts using weak lensing magnification. By stacking a significant number of clusters at different redshifts with average masses of ~1-3x10^14M_sun, as estimated from their richness, we can calibrate the normalisation of the mass-richness relation. With the current data set (area: 6 deg^2) we detect a magnification signal at the >3-sigma level. There is good agreement between the masses estimated from the richness of the clusters and the average masses estimated from magnification, albeit with large uncertainties. We perform tests that suggest the absence of strong systematic effects and support the robustness of the measurement. This method - when applied to larger data sets in the future - will yield an accurate calibration of the mass-observable relations at z>~1 which will represent an invaluable input for cosmological studies using the galaxy cluster mass function and astrophysical studies of cluster formation. Furthermore this method will probably be the least expensive way to measure masses of large numbers of z>1 clusters detected in future IR-imaging surveys.

Recent work has shown that the star formation-density relation -- in which galaxies with low star formation rates are preferentially found in dense environments -- is still in place at z~1, but the situation becomes less clear at higher redshifts. We use mass-selected samples drawn from the UKIDSS Ultra-Deep Survey to show that galaxies with quenched star formation tend to reside in dense environments out to at least z~1.8. Over most of this redshift range we are able to demonstrate that this star formation-density relation holds even at fixed stellar mass. The environmental quenching of star formation appears to operate with similar efficiency on all galaxies regardless of stellar mass. Nevertheless, the environment plays a greater role in the build-up of the red sequence at lower masses, whereas other quenching processes dominate at higher masses. In addition to a statistical analysis of environmental densities, we investigate a cluster at z=1.6, and show that the central region has an elevated fraction of quiescent objects relative to the field. Although the uncertainties are large, the environmental quenching efficiency in this cluster is consistent with that of galaxy groups and clusters at z~0. In this work we rely on photometric redshifts, and describe some of the pitfalls that large redshift errors can present.

Context: In 2.2m MPG-ESO/WFI data we discovered several mass peaks through weak lensing, forming a possible supercluster at redshift 0.45. Through multi-colour wide-field imaging with CFHT/Megaprime and INT/WFC we identify early-type galaxies and trace the supercluster network with them. Through EMMI/NTT multi-object spectroscopy we verify the initial shear-selected cluster candidates. Using weak lensing we obtain mass estimates for the supercluster centre and the filaments. Results: We identified the centre of the SCL2243-0935 supercluster, MACS J2243-0935, which was found independently by Ebeling et al. (2010). 13 more clusters or overdensities are embedded in a filamentary network, half of them are already spectroscopically confirmed. Three (5-15) Mpc filaments are detected, and we estimate the global size of SCL2243 to 45x15x50 Mpc, making it one of the largest superclusters known at intermediate redshifts. Weak lensing yields r_200=(2.06+/-0.13) Mpc and M_200=(1.54+/-0.29)x10^15 M_sun for MACS J2243 with M/L=428+/-82, very similar to results from size-richness cluster scaling relations. Integrating the weak lensing surface mass density over the supercluster network (defined by increased i-band luminosity or g-i colours), we find (1.53+/-1.01)x10^15 M_sun and M/L=305+/-201 for the three main filaments, consistant with theoretical predictions. The filaments' projected surface mass density is 0.007-0.012, corresponding to 10-100 times the critical density. The greatly varying density of the cosmic web is also reflected in the mean colour of galaxies. Conclusions: SCL2243 is significantly larger and much more richly structured than other known superclusters such as A901/902 or MS0302 studied with weak lensing before. It is a text-book supercluster with little contamination along the line of sight, making it a perfect sandbox for testing new techniques probing the cosmic web.

We identify and study a previously unknown systematic effect on cosmic shear measurements, caused by the selection of galaxies used for shape measurement, in particular the rejection of close (blended) galaxy pairs. We use ray-tracing simulations based on the Millennium Simulation and a semi-analytical model of galaxy formation to create realistic galaxy catalogues. From these, we quantify the bias in the shear correlation functions by comparing measurements made from galaxy catalogues with and without removal of close pairs. A likelihood analysis is used to quantify the resulting shift in estimates of cosmological parameters. The filtering of objects with close neighbours (a) changes the redshift distribution of the galaxies used for correlation function measurements, and (b) correlates the number density of sources in the background with the density field in the foreground. This leads to a scale-dependent bias of the correlation function of several percent, translating into biases of cosmological parameters of similar amplitude. This makes this new systematic effect potentially harmful for upcoming and planned cosmic shear surveys. As a remedy, we propose and test a weighting scheme that can significantly reduce the bias.

We present an optimised galaxy cluster finder, 3D-Matched-Filter (3D-MF), which utilises galaxy cluster radial profiles, luminosity functions and redshift information to detect galaxy clusters in optical surveys. This method is an improvement over other matched-filter methods, most notably through implementing redshift slicing of the data to significantly reduce line-of-sight projections and related false positives. We apply our method to the Canada-France-Hawaii Telescope Legacy Survey (CFHTLS) Deep fields, finding ~170 galaxy clusters per square degree in the 0.2 <= z <= 1.0 redshift range. Future surveys such as LSST and JDEM can exploit 3D-MF's automated methodology to produce complete and reliable galaxy cluster catalogues. We determine the reliability and accuracy of the statistical approach of our method through a thorough analysis of mock data from the Millennium Simulation. We detect clusters with 100% completeness for M_200 >= 3.0x10^(14)M_sun, 88% completeness for M_200 >= 1.0x10^(14)M_sun, and 72% completeness well into the 10^(13)M_sun cluster mass range. We show a 36% multiple detection rate for cluster masses >= 1.5x10^(13)M_sun and a 16% false detection rate for galaxy clusters >~ 5x10^(13)M_sun, reporting that for clusters with masses <~ 5x10^(13)M_sun false detections may increase up to ~24%. Utilising these selection functions we conclude that our galaxy cluster catalogue is the most complete CFHTLS Deep cluster catalogue to date.

We measure and study the evolution of the UV galaxy Luminosity Function (LF) at z=3-5 from the largest high-redshift survey to date, the Deep part of the CFHT Legacy Survey. We also give accurate estimates of the SFR density at these redshifts. We consider ~100,000 Lyman-break galaxies at z~3.1, 3.8 & 4.8 selected from very deep ugriz images of this data set and estimate their rest-frame 1600A luminosity function. Due to the large survey volume, cosmic variance plays a negligible role. Furthermore, we measure the bright end of the LF with unprecedented statistical accuracy. Contamination fractions from stars and low-z galaxy interlopers are estimated from simulations. To correct for incompleteness, we study the detection rate of simulated galaxies injected to the images as a function of magnitude and redshift. We estimate the contribution of several systematic effects in the analysis to test the robustness of our results. We find the bright end of the LF of our u-dropout sample to deviate significantly from a Schechter function. If we modify the function by a recently proposed magnification model, the fit improves. For the first time in an LBG sample, we can measure down to the density regime where magnification affects the shape of the observed LF because of the very bright and rare galaxies we are able to probe with this data set. We find an increase in the normalisation, $\phi^{*}$, of the LF by a factor of 2.5 between z~5 and z~3. The faint-end slope of the LF does not evolve significantly between z~5 and z~3. We do not find a significant evolution of the characteristic magnitude in the studied redshift interval. The SFR density is found to increase by a factor of ~2 from z~5 to z~4. The evolution from z~4 to z~3 is less eminent.

Here we introduce PHAT, the PHoto-z Accuracy Testing programme, an international initiative to test and compare different methods of photo-z estimation. Two different test environments are set up, one (PHAT0) based on simulations to test the basic functionality of the different photo-z codes, and another one (PHAT1) based on data from the GOODS survey. The accuracy of the different methods is expressed and ranked by the global photo-z bias, scatter, and outlier rates. Most methods agree well on PHAT0 but produce photo-z scatters that can differ by up to a factor of two even in this idealised case. A larger spread in accuracy is found for PHAT1. Few methods benefit from the addition of mid-IR photometry. Remaining biases and systematic effects can be explained by shortcomings in the different template sets and the use of priors on the one hand and an insufficient training set on the other hand. Scatters of 4-8% in Delta_z/(1+z) were obtained, consistent with other studies. However, somewhat larger outlier rates (>7.5% with Delta_z/(1+z)>0.15; >4.5% after cleaning) are found for all codes. There is a general trend that empirical codes produce smaller biases than template-based codes. The systematic, quantitative comparison of different photo-z codes presented here is a snapshot of the current state-of-the-art of photo-z estimation and sets a standard for the assessment of photo-z accuracy in the future. The rather large outlier rates reported here for PHAT1 on real data should be investigated further since they are most probably also present (and possibly hidden) in many other studies. The test data sets are publicly available and can be used to compare new methods to established ones and help in guiding future photo-z method development. (abridged)

We propose a new approach for measuring the mass profile and shape of groups and clusters of galaxies, which uses lensing magnification of distant background galaxies. The main advantage of lensing magnification is that, unlike lensing shear, it relies on accurate photometric redshifts only and not galaxy shapes, thus enabling the study of the dark matter distribution with unresolved source galaxies. We present a feasibility study, using a real population of z > 2.5 Lyman Break Galaxies as source galaxies, and where, similar to galaxy-galaxy lensing, foreground lenses are stacked in order to increase the signal-to-noise. We find that there is an interesting new observational window for gravitational lensing as a probe of dark matter halos at high redshift, which does not require measurement of galaxy shapes.

We present a tomographic cosmological weak lensing analysis of the HST COSMOS Survey. Applying our lensing-optimized data reduction, principal component interpolation for the ACS PSF, and improved modelling of charge-transfer inefficiency, we measure a lensing signal which is consistent with pure gravitational modes and no significant shape systematics. We carefully estimate the statistical uncertainty from simulated COSMOS-like fields obtained from ray-tracing through the Millennium Simulation. We test our pipeline on simulated space-based data, recalibrate non-linear power spectrum corrections using the ray-tracing, employ photometric redshifts to reduce potential contamination by intrinsic galaxy alignments, and marginalize over systematic uncertainties. We find that the lensing signal scales with redshift as expected from General Relativity for a concordance LCDM cosmology, including the full cross-correlations between different redshift bins. For a flat LCDM cosmology, we measure sigma_8(Omega_m/0.3)^0.51=0.75+-0.08 from lensing, in perfect agreement with WMAP-5, yielding joint constraints Omega_m=0.266+0.025-0.023, sigma_8=0.802+0.028-0.029 (all 68% conf.). Dropping the assumption of flatness and using HST Key Project and BBN priors only, we find a negative deceleration parameter q_0 at 94.3% conf. from the tomographic lensing analysis, providing independent evidence for the accelerated expansion of the Universe. For a flat wCDM cosmology and prior w in [-2,0], we obtain w<-0.41 (90% conf.). Our dark energy constraints are still relatively weak solely due to the limited area of COSMOS. However, they provide an important demonstration for the usefulness of tomographic weak lensing measurements from space. (abridged)

We have discovered a strong lensing fossil group (J0454) at z=0.26, projected near the well-studied cluster MS0451-0305. Using multicolour Subaru/Suprime-Cam and CFHT/Megaprime imaging together with Keck spectroscopy we identify member galaxies. A VLT/FORS2 spectrum was taken to determine the redshifts of the brightest elliptical and the lensed arc. Using HST/ACS images we determine the group's weak lensing signal and model the strong lens system. This is the first time that a fossil group is analysed with lensing methods. The X-ray luminosity and temperature are derived from XMM-Newton data. We find two filaments extending over 4 Mpc, and within the virial radius we identify 31 members spectroscopically and 33 via the red sequence with i<22 mag. They segregate into spirals and a dynamically cooler central concentration of ellipticals with a velocity dispersion of 480 km/s. Weak lensing and cluster richness relations yield r200 = 830 kpc and M200 = 0.85 x 10^14 Msun. The brightest group galaxy (BGG) is inconsistent with the dynamic centre of J0454. It strongly lenses a galaxy at z~2.1, and we model the lens with a pseudo-isothermal elliptical mass distribution. A large external shear requires that the BGG must be offset from J0454's dark halo centre by at least 90-130 kpc. The X-ray halo is offset by 24 kpc from the BGG, shows no signs of a cooling flow and is well described by a beta-model. With L = 1.4 x 10^43 erg/s J0454 falls well onto standard X-ray cluster scaling relations, but appears cooler (1.1 keV) than expected (2 keV). We conclude that J0454 consists of two systems, a sparse cluster and an infalling fossil group, the latter seeding the BCG. An alternative explanation is that galaxies in a filament, projected along the line of sight, stream towards the denser fossil group.

The mass function of galaxy clusters at high redshifts is a particularly useful probe to learn about the history of structure formation and constrain cosmological parameters. We aim at deriving reliable masses for a high-redshift, high-luminosity sample of clusters of galaxies selected from the 400d survey of X-ray selected clusters. Here, we will focus on a particular object, CL0030+2618 at z=0.50 Using deep imaging in three passbands with the MEGACAM instrument at MMT, we show that MEGACAM is well-suited for measuring gravitational shear. We detect the weak lensing signal of CL0030+2618 at 5.8 sigma significance, using the aperture mass technique. Furthermore, we find significant tangential alignment of galaxies out to ~10 arcmin or >2r_200 distance from the cluster centre. The weak lensing centre of CL0030+2618 agrees with several X-ray measurements and the position of the brightest cluster galaxy. Finally, we infer a weak lensing virial mass of M_200=7.5 10^{14} M_sun for CL0030+2618. Despite complications by a tentative foreground galaxy group in the line of sight, the X-ray and weak lensing estimates for CL0030+2618 are in remarkable agreement. This study paves the way for the largest weak lensing survey of high-redshift galaxy clusters to date.

Weak gravitational lensing (WL) has been established as one of the most promising probes of cosmology. So far, most studies have exploited the shear effect of WL leading to coherent distortions of galaxy shapes. But WL also introduces coherent magnifications. We want to detect this cosmic magnification effect in large samples of high-redshift galaxies selected from the Deep part of the CFHTLS. Lyman-break galaxies (LBGs) at z=2.5-5, are used as a background sample and are cross-correlated to foreground lens galaxies selected by photo-z's. The signals of LBGs in different magnitude bins are compared to predictions from WL theory. An optimally weighted correlation function is estimated by taking into account the slope of external LBG luminosity functions. For the first time, we detect cosmic magnification in a sample of normal galaxies. These background sources are also the ones with the highest redshifts so far used for WL measurements. The amplitude and angular dependence of the cross-correlation functions agree well with theoretical expectations and the lensing signal is detected with high significance. In particular, we detect an anti-correlation between faint LBGs and foreground galaxies which cannot be caused by redshift overlap. (abridged)

This paper studies the relative spatial distribution of red-sequence and blue-cloud galaxies, and their relation to the dark matter distribution in the COMBO-17 survey as function of scale down to z~1. We measure the 2nd-order auto- and cross-correlation functions of galaxy clustering and express the relative biasing by using aperture statistics. Also estimated is the relation between the galaxies and the dark matter distribution exploiting galaxy-galaxy lensing (GGL). All observables are further interpreted in terms of a halo model. To fully explain the galaxy clustering cross-correlation function with a halo model, we need to introduce a new parameter,R, that describes the statistical relation between numbers of red and blue galaxies within the same halo. We find that red and blue galaxies are clearly differently clustered, a significant evolution of the relative clustering with redshift was not found. There is evidence for a scale-dependence of relative biasing. The relative clustering, the GGL and, with some tension, the galaxy numbers can be explained consistently within a halo model. For the cross-correlation function one requires a HOD variance that becomes Poisson even for relatively small occupancy numbers. For our sample, this rules out with high confidence a "Poisson satellite" scenario as found in semi-analytical models. Red galaxies have to be concentrated towards the halo centre, either by a central red galaxy or by a concentration parameter above that for dark matter.The value of R depends on the presence or absence of central galaxies: If no central galaxies or only red central galaxies are allowed, R is consistent with zero, whereas a positive correlation $R=+0.5\pm0.2$ is needed if both blue and red galaxies can have central galaxies.[ABRIDGED]

Aims: We measure the clustering properties for a large samples of u- (z~3), g- (z~4), and r- (z~5) dropouts from the Canada-France-Hawaii Telescope Legacy Survey (CFHTLS) Deep fields. Methods: Photometric redshift distributions along with simulations allow us to de-project the angular correlation measurements and estimate physical quantities such as the correlation length, halo mass, galaxy bias, and halo occupation as a function of UV luminosity. Results: For the first time we detect a significant one-halo term in the correlation function at z~5. The comoving correlation lengths and halo masses of LBGs are found to decrease with decreasing rest-frame UV-luminosity. No significant redshift evolution is found in either quantity. The typical halo mass hosting an LBG is M>~10^{12}h^{-1}M_\odot and the halos are typically occupied by less than one galaxy. Clustering segregation with UV luminosity is clearly observed in the dropout samples, however redshift evolution cannot clearly be disentangled from systematic uncertainties introduced by the redshift distributions. We study a range of possible redshift distributions to illustrate the effect of this choice. Spectroscopy of representative subsamples is required to make high-accuracy absolute measurements of high-z halo masses.

We present a new K-selected, optical-to-near-infrared photometric catalog of the Extended Chandra Deep Field South (ECDFS), making it publicly available to the astronomical community. The dataset is founded on publicly available imaging, supplemented by original zJK imaging data obtained as part of the MUltiwavelength Survey by Yale-Chile (MUSYC). The final photometric catalog consists of photometry derived from nine band U-K imaging covering the full 0.5x0.5 sq. deg. of the ECDFS, plus H band data for approximately 80% of the field. The 5sigma flux limit for point-sources is K = 22.0 (AB). This is also the nominal completeness and reliability limit of the catalog: the empirical completeness for 21.75 < K < 22.00 is 85+%. We have verified the quality of the catalog through both internal consistency checks, and comparisons to other existing and publicly available catalogs. As well as the photometric catalog, we also present catalogs of photometric redshifts and restframe photometry derived from the ten band photometry. We have collected robust spectroscopic redshift determinations from published sources for 1966 galaxies in the catalog. Based on these sources, we have achieved a (1sigma) photometric redshift accuracy of Dz/(1+z) = 0.036, with an outlier fraction of 7.8%. Most of these outliers are X-ray sources. Finally, we describe and release a utility for interpolating restframe photometry from observed SEDs, dubbed InterRest. Particularly in concert with the wealth of already publicly available data in the ECDFS, this new MUSYC catalog provides an excellent resource for studying the changing properties of the massive galaxy population at z < 2. (Abridged)

We present the CFHTLS-Archive-Research Survey (CARS). It is a virtual multi-colour survey based on public archive images from the CFHT-Legacy-Survey. Our main scientific interests in CARS are optical searches for galaxy clusters from low to high redshift and their subsequent study with photometric and weak-gravitational lensing techniques. As a first step of the project we present multi-colour catalogues from 37 sq. degrees of the CFHTLS-Wide component. Our aims are to create astrometrically and photometrically well calibrated co-added images. Second goal are five-band (u*, g', r', i', z') multi-band catalogues with an emphasis on reliable estimates for object colours. These are subsequently used for photometric redshift estimates. The article explains in detail data processing, multi-colour catalogue creation and photometric redshift estimation. Furthermore we apply a novel technique, based on studies of the angular galaxy cross-correlation function, to quantify the reliability of photo-z's. The accuracy of our high-confidence photo-z sample (10-15 galaxies per sq. arcmin) is estimated to $\sigma_{\Delta_z/(1+z)}\approx 0.04-0.05$ up to i'<24 with typically only 1-3% outliers. Interested users can obtain access to our data by request to the authors.

We present an update to the photometric calibration of the COMBO-17 catalogue on the Extended Chandra Deep Field South, which is now consistent with the GaBoDS and MUSYC catalogues. As a result, photometric redshifts become slightly more accurate, with <0.01 rms and little bias in the delta_z/(1+z) of galaxies with R<21 and of QSOs with R<24. With increasing photon noise the rms of galaxies reaches 0.02 for R<23 and 0.035 at R~23.5. Consequences for the rest-frame colours of galaxies at z<1 are discussed.

The galaxy cluster RX J1347-1145 is one of the most X-ray luminous and most massive clusters known. Its extreme mass makes it a prime target for studying issues addressing cluster formation and cosmology. In this paper we present new high-resolution HST/ACS and Chandra X-ray data. The high resolution and sensitivity of ACS enabled us to detect and quantify several new multiply imaged sources, we now use a total of eight for the strong lensing analysis. Combining this information with shape measurements of weak lensing sources in the central regions of the cluster, we derive a high-resolution, absolutely-calibrated mass map. This map provides the best available quantification of the total mass of the central part of the cluster to date. We compare the reconstructed mass with that inferred from the new Chandra X-ray data, and conclude that both mass estimates agree extremely well in the observed region, namely within 400 / h_70 kpc of the cluster center. In addition we study the major baryonic components (gas and stars) and hence derive the dark matter distribution in the center of the cluster. We find that the dark matter and baryons are both centered on the BCG within the uncertainties (alignment is better than <10 kpc). We measure the corresponding 1-D profiles and find that dark matter distribution is consistent with both NFW and cored profiles, indicating that a more extended radial analysis is needed to pinpoint the concentration parameter, and hence the inner slope of the dark matter profile.

Aims. We analyse the relative performance of different photo-z codes in blind applications to ground-based data. Methods. We tested the codes on imaging datasets with different depths and filter coverages and compared the results to large spectroscopic catalogues. The photo-z error behaviour was analysed to select cleaner subsamples with more secure photo-z estimates. We consider Hyperz, BPZ, and the code used in the CADIS, COMBO-17, and HIROCS surveys. Results. The photo-z error estimates of the three codes do not correlate tightly with the accuracy of the photo-z's. While very large errors sometimes indicate a true catastrophic photo-z failure, smaller errors are usually not meaningful. For any given dataset, we find significant differences in redshift accuracy and outlier rates between the different codes when compared to spectroscopic redshifts. However, different codes excel in different regimes. The agreement between different sets of photo-z's is better for the subsample with secure spectroscopic redshifts than for the whole catalogue. Conclusions. Running today's photo-z codes on well-calibrated ground-based data can lead to reasonable accuracy. The actual performance on a given dataset is largely dependent on the template choice and on realistic instrumental response curves. It would be desirable to improve the photo-z error estimation for future applications so as to get a better handle on rejecting objects with grossly inaccurate photo-z's. The secure spectroscopic subsamples commonly used for assessments of photo-z accuracy may be biased toward objects for which the photo-z's are easier to estimate than for a complete flux-limited sample, resulting in very optimistic estimates. (abridged)

High resolution HST/ACS images of the galaxy cluster RX J1347-1145 have enabled us to identify several new multiple image candidates in the cluster, including a 5 image system with a central image. The multiple images allow us to construct an accurate 2-dimensional mass map of the central part of the cluster. The modelling of the cluster mass includes the most prominent cluster galaxies modelled as truncated isothermal spheres and a smooth halo component that is described with 2 parametric profiles. The mass reconstruction is done using a Markov chain Monte Carlo method that provides us with a total projected mass density as well as estimates for the parameters of interest and their respective errors. The mass profile is in reasonable agreement with previous mass estimates based on the X-ray emission from the hot intra-cluster gas, however the X-ray mass estimates are systematically lower than what we obtain with gravitational lensing.

This is the first paper of a series describing our measurement of weak lensing by large-scale structure using archival observations from the Advanced Camera for Surveys (ACS) on board the Hubble Space Telescope (HST). In this work we present results from a pilot study testing the capabilities of the ACS for cosmic shear measurements with early parallel observations and presenting a re-analysis of HST/ACS data from the GEMS survey and the GOODS observations of the Chandra Deep Field South (CDFS). We describe our new correction scheme for the time-dependent ACS PSF based on observations of stellar fields. This is currently the only technique which takes the full time variation of the PSF between individual ACS exposures into account. We estimate that our PSF correction scheme reduces the systematic contribution to the shear correlation functions due to PSF distortions to < 2*10^{-6} for galaxy fields containing at least 10 stars. We perform a number of diagnostic tests indicating that the remaining level of systematics is consistent with zero for the GEMS and GOODS data confirming the success of our PSF correction scheme. For the parallel data we detect a low level of remaining systematics which we interpret to be caused by a lack of sufficient dithering of the data. Combining the shear estimate of the GEMS and GOODS observations using 96 galaxies arcmin^{-2} with the photometric redshift catalogue of the GOODS-MUSIC sample, we determine a local single field estimate for the mass power spectrum normalisation sigma_{8,CDFS}=0.52^{+0.11}_{-0.15} (stat) +/- 0.07 (sys) (68% confidence assuming Gaussian cosmic variance) at fixed Omega_m=0.3 for a LambdaCDM cosmology. We interpret this exceptionally low estimate to be due to a local under-density of the foreground structures in the CDFS.

We introduce our publicly available Wide-Field-Imaging reduction pipeline THELI. The procedures applied for the efficient pre-reduction and astrometric calibration are presented. A special emphasis is put on the methods applied to the photometric calibration. As a test case the reduction of optical data from the ESO Deep Public Survey including the WFI-GOODS data is described. The end-products of this project are now available via the ESO archive Advanced Data Products section.

Aims. The clustering properties of a large sample of U-dropouts are investigated and compared to very precise results for B-dropouts from other studies to identify a possible evolution from z=4 to z=3. Methods. A population of ~8800 candidates for star-forming galaxies at z=3 is selected via the well-known Lyman-break technique from a large optical multicolour survey (the ESO Deep Public Survey). The selection efficiency, contamination rate, and redshift distribution of this population are investigated by means of extensive simulations. Photometric redshifts are estimated for every Lyman-break galaxy (LBG) candidate from its UBVRI photometry yielding an empirical redshift distribution. The measured angular correlation function is deprojected and the resulting spatial correlation lengths and slopes of the correlation function of different subsamples are compared to previous studies. Results. By fitting a simple power law to the correlation function we do not see an evolution in the correlation length and the slope from other studies at z=4 to our study at z=3. In particular, the dependence of the slope on UV-luminosity similar to that recently detected for a sample of B-dropouts is confirmed also for our U-dropouts. For the first time number statistics for U-dropouts are sufficient to clearly detect a departure from a pure power law on small scales down to ~2" reported by other groups for B-dropouts.

Aims. We present a cosmic shear analysis and data validation of 15 square degree high-quality R-band data of the Garching-Bonn Deep Survey obtained with the Wide Field Imager of the MPG/ESO 2.2m telescope. Methods. We measure the two-point shear correlation functions to calculate the aperture mass dispersion. Both statistics are used to perform the data quality control. Combining the cosmic shear signal with a photometric redshift distribution of a galaxy sub-sample obtained from two square degree of UBVRI-band observations of the Deep Public Survey we determine constraints for the matter density Omega_m, the mass power spectrum normalisation sigma_8 and the dark energy density Omega_Lambda in the magnitude interval R in [21.5,24.5]. In this magnitude interval the effective number density of source galaxies is n=12.5/sq. arcmin, and their mean redshift is z_m=0.78. To estimate the posterior likelihood we employ the Monte Carlo Markov Chain method. Results. Using the aperture mass dispersion we obtain for the mass power spectrum normalisation sigma_8=0.80 +- 0.10 (1 sigma statistical error) at a fixed matter density Omega_m=0.30 assuming a flat universe with negligible baryon content and marginalising over the Hubble parameter and the uncertainties in the fitted redshift distribution.

Aims. In this paper the optical data of the ESO Deep-Public-Survey observed with the Wide Field Imager and reduced with the THELI pipeline are described. Methods. Here we present 63 fully reduced and stacked images. The astrometric and photometric calibrations are discussed and the properties of the images are compared to images released by the ESO Imaging Survey team covering a subset of our data. Results. These images are publicly released to the community. Our main scientific goals with this survey are to study the high-redshift universe by optically pre-selecting high-redshift objects from imaging data and to use VLT instruments for follow-up spectroscopy as well as weak lensing applications.

We present first results of our search for high-redshift galaxies in deep CCD mosaic images. As a pilot study for a larger survey, very deep images of the Chandra Deep Field South (CDFS), taken withWFI@MPG/ESO2.2m, are used to select large samples of 1070 U-band and 565 B-band dropouts with the Lyman-break method. The data of these Lyman-break galaxies are made public as an electronic table. These objects are good candidates for galaxies at z~3 and z~4 which is supported by their photometric redshifts. The distributions of apparent magnitudes and the clustering properties of the two populations are analysed, and they show good agreement to earlier studies. We see no evolution in the comoving clustering scale length from z~3 to z~4. The techniques presented here will be applied to a much larger sample of U-dropouts from the whole survey in near future.

We present our image processing system for the reduction of optical imaging data from multi-chip cameras. In the framework of the Garching Bonn Deep Survey (GaBoDS; Schirmer et al. 2003) consisting of about 20 square degrees of high-quality data from WFI@MPG/ESO 2.2m, our group developed an imaging pipeline for the homogeneous and efficient processing of this large data set. Having weak gravitational lensing as the main science driver, our algorithms are optimised to produce deep co-added mosaics from individual exposures obtained from empty field observations. However, the modular design of our pipeline allows an easy adaption to different scientific applications. Our system has already been ported to a large variety of optical instruments and its products have been used in various scientific contexts. In this paper we give a thorough description of the algorithms used and a careful evaluation of the accuracies reached. This concerns the removal of the instrumental signature, the astrometric alignment, photometric calibration and the characterisation of final co-added mosaics. In addition we give a more general overview on the image reduction process and comment on observing strategies where they have significant influence on the data quality.