We present weak lensing shear catalogues from the fourth data release of the Kilo-Degree Survey, KiDS-1000, spanning 1006 square degrees of deep and high-resolution imaging. Our `gold-sample' of galaxies, with well calibrated photometric redshift distributions, consists of 21 million galaxies with an effective number density of $\sim 6.2$ galaxies per square arcminute. We quantify the accuracy of the spatial, temporal and flux-dependent point-spread function (PSF) model, verifying that the model meets our requirements to induce less than a $0.1\sigma$ change in the inferred cosmic shear constraints on the clustering cosmological parameter $S_8 = \sigma_8\sqrt{\Omega_{\rm m}/0.3}$. Through a series of two-point null-tests we validate the shear estimates, finding no evidence for significant non-lensing B-mode distortions in the data. PSF residuals are detected in the highest-redshift bins, originating from object selection and/or weight bias. The amplitude is however shown to be sufficiently low and within our stringent requirements. With a shear-ratio null-test we verify the expected redshift scaling of the galaxy-galaxy lensing signal around luminous red galaxies. We conclude that the joint KiDS-1000 shear and photometric redshift calibration is sufficiently robust for combined-probe gravitational lensing and spectroscopic clustering analyses.

We present the methodology for a joint cosmological analysis of weak gravitational lensing from the fourth data release of the ESO Kilo-Degree Survey (KiDS-1000) and galaxy clustering from the partially overlapping BOSS and 2dFLenS surveys. Cross-correlations between galaxy positions and ellipticities are incorporated into the analysis, developing a hybrid model of non-linear scales that blends perturbative and non-perturbative approaches, and assessing signal contributions by astrophysical effects. All weak lensing signals are measured consistently via Fourier-space statistics that are insensitive to the survey mask and display low levels of mode mixing. The calibration of photometric redshift distributions and multiplicative gravitational shear bias is updated, and a more complete tally of residual calibration uncertainties is propagated into the likelihood. A dedicated suite of more than 20000 mocks is used to assess the performance of covariance models and to quantify the impact of survey geometry and spatial variations of survey depth on signals and their errors. The sampling distributions for the likelihood and the $\chi^2$ goodness-of-fit statistic are validated, with proposed changes to calculating the effective number of degrees of freedom. Standard weak lensing point estimates on $S_8=\sigma_8\,(\Omega_{\rm m}/0.3)^{1/2}$ derived from its marginal posterior are easily misinterpreted to be biased low, and an alternative estimator and associated credible interval are proposed. Known systematic effects pertaining to weak lensing modelling and inference are shown to bias $S_8$ by no more than 0.1 standard deviations, with the caveat that no conclusive validation data exist for models of intrinsic galaxy alignments. Compared to the previous KiDS analyses, $S_8$ constraints are expected to improve by 20% for weak lensing alone and by 29% for the joint analysis. [abridged]

We simultaneously present constraints on the stellar-to-halo mass relation for central and satellite galaxies through a weak lensing analysis of spectroscopically classified galaxies. Using overlapping data from the 4th data release of the Kilo-Degree Survey (KiDS), and the Galaxy And Mass Assembly survey (GAMA), we find that satellite galaxies are hosted by halo masses that are $0.53 \pm 0.39$ dex (68% confidence, $3\sigma$ detection) smaller than those of central galaxies of the same stellar mass (for a stellar mass of $\log(M_{\star}/M_{\odot}) = 10.6$). This is consistent with galaxy formation models, whereby infalling satellite galaxies are preferentially stripped of their dark matter. We find consistent results with similar uncertainties when comparing constraints from a standard azimuthally averaged galaxy-galaxy lensing analysis and a two-dimensional likelihood analysis of the full shear field. As the latter approach is somewhat biased due to the lens incompleteness, and does not provide any improvement in precision when applied to actual data, we conclude that stacked tangential shear measurements are best-suited for studies of the galaxy-halo connection.

Several semi-analytic models (SAMs) try to explain how galaxies form, evolve and interact inside the dark matter large-scale structure. These SAMs can be tested by comparing their predictions for galaxy-galaxy-galaxy-lensing (G3L), which is weak gravitational lensing around galaxy pairs, with observations. We evaluate the SAMs by Henriques et al. (2015; H15) and by Lagos et al. (2012; L12), implemented in the Millennium Run, by comparing their predictions for G3L to observations at smaller scales than previous studies and also for pairs of lens galaxies from different populations. We compare the G3L signal predicted by the SAMs to measurements in the overlap of the Galaxy And Mass Assembly survey (GAMA), the Kilo-Degree Survey (KiDS), and the VISTA Kilo-degree Infrared Galaxy survey (VIKING), splitting lens galaxies into two colour and five stellar-mass samples. Using an improved G3L estimator, we measure the three-point correlation of the matter distribution for mixed lens pairs with galaxies from different samples, and unmixed lens pairs with galaxies from the same sample. Predictions by the H15 SAM agree with the observations for all colour-selected and all but one stellar-mass-selected sample with 95% confidence. Deviations occur for lenses with stellar masses below $9.5h^{-2}\mathrm{M}_\odot$ at scales below $0.2h^{-1}\mathrm{Mpc}$. Predictions by the L12 SAM for stellar-mass selected samples and red galaxies are significantly higher than observed, while the predicted signal for blue galaxy pairs is too low. The L12 SAM predicts more pairs of small stellar-mass and red galaxies than the H15 SAM and the observations, as well as fewer pairs of blue galaxies. Likely explanations are different treatments of environmental effects by the SAMs and different models of the initial mass function. We conclude that G3L provides a stringent test for models of galaxy formation and evolution.

The physics of gravity on cosmological scales affects both the rate of assembly of large-scale structure, and the gravitational lensing of background light through this cosmic web. By comparing the amplitude of these different observational signatures, we can construct tests that can distinguish General Relativity from its potential modifications. We use the latest weak gravitational lensing dataset from the Kilo-Degree Survey, KiDS-1000, in conjunction with overlapping galaxy spectroscopic redshift surveys BOSS and 2dFLenS, to perform the most precise existing amplitude-ratio test. We measure the associated E_G statistic with 15-20% errors, in five dz = 0.1 tomographic redshift bins in the range 0.2 < z < 0.7, on projected scales up to 100 Mpc/h. The scale-independence and redshift-dependence of these measurements are consistent with the theoretical expectation of General Relativity in a Universe with matter density Omega_m = 0.27 +/- 0.04. We demonstrate that our results are robust against different analysis choices, including schemes for correcting the effects of source photometric redshift errors, and compare the performance of angular and projected galaxy-galaxy lensing statistics.

We present updated cosmological constraints for the KiDS+VIKING-450 cosmic shear dataset (KV450), estimated using redshift distributions and photometric samples defined using self organising maps (SOMs). Our fiducial analysis finds marginal posterior constraints of $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.716^{+0.043}_{-0.038}$; smaller than, but fully consistent with, previous work using this dataset ($|\Delta S_8| = 0.023$). We analyse additional samples and redshift distributions constructed in three ways: excluding certain spectroscopic surveys during redshift calibration, excluding lower-confidence spectroscopic redshifts in redshift calibration, and considering only photometric sources which are jointly calibrated by at least three spectroscopic surveys. In all cases, the method utilised here proves robust: we find a maximal deviation from our fiducial analysis of $|\Delta S_8| \leq 0.009$ for all samples defined and analysed using our SOM. Our largest shift in $S_8$ is found when calibrating redshift distributions without the DEEP2 spectroscopic subset, where we find $S_8=0.707_{-0.042}^{+0.046}$. This difference with respect to the fiducial is both significantly smaller than, and in the opposite direction to, the equivalent shift from previous work. No sample analysed in this work results in a meaningful positive shift in $S_8$ with respect to our fiducial constraints. These results suggest that our improved cosmological parameter estimates are insensitive to pathological misrepresentation of photometric sources by the spectroscopy used for direct redshift calibration, and therefore that this systematic effect cannot be responsible for the observed difference between $S_8$ estimates made with KV450 and Planck CMB probes.

We present a combined tomographic weak gravitational lensing analysis of the Kilo Degree Survey (KV450) and the Dark Energy Survey (DES-Y1). We homogenize the analysis of these two public cosmic shear datasets by adopting consistent priors and modeling of nonlinear scales, and determine new redshift distributions for DES-Y1 based on deep public spectroscopic surveys. Adopting these revised redshifts results in a $0.8\sigma$ reduction in the DES-inferred value for $S_8$, which decreases to a $0.5\sigma$ reduction when including a systematic redshift calibration error model from mock DES data based on the MICE2 simulation. The combined KV450 + DES-Y1 constraint on $S_8 = 0.762^{+0.025}_{-0.024}$ is in tension with the Planck 2018 constraint from the cosmic microwave background at the level of $2.5\sigma$. This result highlights the importance of developing methods to provide accurate redshift calibration for current and future weak lensing surveys.

We present cosmological constraints from a joint cosmic shear analysis of the Kilo-Degree Survey (KV450) and the Dark Energy Survey (DES-Y1), conducted using Complete Orthogonal Sets of E/B-Integrals (COSEBIs). With COSEBIs we isolate any B-modes which have a non-cosmic shear origin and demonstrate the robustness of our cosmological E-mode analysis as no significant B-modes are detected. We highlight how COSEBIs are fairly insensitive to the amplitude of the non-linear matter power spectrum at high $k$-scales, mitigating the uncertain impact of baryon feedback in our analysis. COSEBIs, therefore, allow us to utilise additional small-scale information, improving the DES-Y1 joint constraints on $S_8=\sigma_8(\Omega_{\rm m}/0.3)^{0.5}$ and $\Omega_{\rm m}$ by $20\%$. Adopting a flat $\Lambda$CDM model we find $S_8=0.755^{+0.019}_{-0.021}$, which is in $3.2\sigma$ tension with the Planck Legacy analysis of the cosmic microwave background.

We reanalyse the anisotropic galaxy clustering measurement from the Baryon Oscillation Spectroscopic Survey (BOSS), demonstrating that using the full shape information provides cosmological constraints that are comparable to other low-redshift probes. We find $\Omega_\mathrm{m} = 0.317^{+0.015}_{-0.019}$, $\sigma_8 = 0.710\pm 0.049$, and $h = 0.704\pm 0.024$ for flat $\Lambda$CDM cosmologies using uninformative priors on $\Omega_\mathrm{c}h^2$, $100\theta_\mathrm{MC}$, $\ln 10^{10} A_{s}$, and $n_{s}$, and a prior on $\Omega_\mathrm{b}h^2$ that is much wider than current constraints. We quantify the agreement between the Planck 2018 constraints from the cosmic microwave background and BOSS, finding the two data sets to be consistent within a flat $\Lambda$CDM cosmology using the Bayes factor as well as the prior-insensitive suspiciousness statistic. Combining two low-redshift probes, we jointly analyse the clustering of BOSS galaxies with weak lensing measurements from the Kilo-Degree Survey (KV450). The combination of BOSS and KV450 improves the measurement by up to 45%, constraining $\sigma_8 = 0.702\pm 0.029$ and $S_8 = \sigma_8\sqrt{\Omega_\mathrm{m}/0.3} = 0.728\pm 0.026$. Over the full 5D parameter space, the odds in favour of single cosmology describing galaxy clustering, lensing, and the cosmic microwave background are $7\pm2$. The suspiciousness statistic signals a $2.1\pm0.3\sigma$ tension between the combined low-redshift probes and measurements from the cosmic microwave background.

We present a weak lensing detection of filamentary structures in the cosmic web, combining data from the Kilo-Degree Survey, the Red Cluster Sequence Lensing Survey and the Canada-France-Hawaii Telescope Lensing Survey. The line connecting luminous red galaxies with a separation of $3 - 5\, h^{-1}\text{Mpc}$ is chosen as a proxy for the location of filaments. We measure the average weak lensing shear around $\sim$11,000 candidate filaments selected in this way from the Sloan Digital Sky Survey. After nulling the shear induced by the dark matter haloes around each galaxy, we report a $3.4\,\sigma$ detection of an anisotropic shear signal from the matter that connects them. Adopting a filament density profile, motivated from $N$-body simulations, the average density at the centre of these filamentary structures is found to be $15 \pm 4$ times the critical density.

We present a tomographic cosmic shear analysis of the Kilo-Degree Survey (KiDS) combined with the VISTA Kilo-Degree Infrared Galaxy Survey (VIKING). This is the first time that a full optical to near-infrared data set has been used for a wide-field cosmological weak lensing experiment. This unprecedented data, spanning $450~$deg$^2$, allows us to improve significantly the estimation of photometric redshifts, such that we are able to include robustly higher-redshift sources for the lensing measurement, and - most importantly - solidify our knowledge of the redshift distributions of the sources. Based on a flat $\Lambda$CDM model we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.737_{-0.036}^{+0.040}$ in a blind analysis from cosmic shear alone. The tension between KiDS cosmic shear and the Planck-Legacy CMB measurements remains in this systematically more robust analysis, with $S_8$ differing by $2.3\sigma$. This result is insensitive to changes in the priors on nuisance parameters for intrinsic alignment, baryon feedback, and neutrino mass. KiDS shear measurements are calibrated with a new, more realistic set of image simulations and no significant B-modes are detected in the survey, indicating that systematic errors are under control. When calibrating our redshift distributions by assuming the 30-band COSMOS-2015 photometric redshifts are correct (following the Dark Energy Survey and the Hyper Suprime-Cam Survey), we find the tension with Planck is alleviated. The robust determination of source redshift distributions remains one of the most challenging aspects for future cosmic shear surveys.

Our aim is to quantify the impact of systematic effects on the inference of cosmological parameters from cosmic shear. We present an end-to-end approach that introduces sources of bias in a modelled weak lensing survey on a galaxy-by-galaxy level. Residual biases are propagated through a pipeline from galaxy properties (one end) through to cosmic shear power spectra and cosmological parameter estimates (the other end), to quantify how imperfect knowledge of the pipeline changes the maximum likelihood values of dark energy parameters. We quantify the impact of an imperfect correction for charge transfer inefficiency (CTI) and modelling uncertainties of the point spread function (PSF) for Euclid, and find that the biases introduced can be corrected to acceptable levels.

Exploiting the full statistical power of future cosmic shear surveys will necessitate improvements to the accuracy with which the gravitational lensing signal is measured. We present a framework for calibrating shear with image simulations that demonstrates the importance of including realistic correlations between galaxy morphology, size and more importantly, photometric redshifts. This realism is essential so that selection and shape measurement biases can be calibrated accurately for a tomographic cosmic shear analysis. We emulate Kilo-Degree Survey (KiDS) observations of the COSMOS field using morphological information from {\it Hubble} Space Telescope imaging, faithfully reproducing the measured galaxy properties from KiDS observations of the same field. We calibrate our shear measurements from lensfit, and find through a range of sensitivity tests that lensfit is robust and unbiased within the allowed 2 per cent tolerance of our study. Our results show that the calibration has to be performed by selecting the tomographic samples in the simulations, consistent with the actual cosmic shear analysis, because the joint distributions of galaxy properties are found to vary with redshift. Ignoring this redshift variation could result in misestimating the shear bias by an amount that exceeds the allowed tolerance. To improve the calibration for future cosmic shear analyses, it will be essential to also correctly account for the measurement of photometric redshifts, which requires simulating multi-band observations.

The outer regions of galaxies are more susceptible to the tidal interactions that lead to intrinsic alignments of galaxies. The resulting alignment signal may therefore depend on the passband if the colours of galaxies vary spatially. To quantify this, we measured the shapes of galaxies with spectroscopic redshifts from the GAMA survey using deep gri imaging data from the KiloDegree Survey. The performance of the moment-based shape measurement algorithm DEIMOS was assessed using dedicated image simulations, which showed that the ellipticities could be determined with an accuracy better than 1% in all bands. Additional tests for potential systematic errors did not reveal any issues. We measure a significant difference of the alignment signal between the g,r and i-band observations. This difference exceeds the amplitude of the linear alignment model on scales below 2 Mpc/h. Separating the sample into central/satellite and red/blue galaxies, we find that that the difference is dominated by red satellite galaxies.

We use the overlap between multiband photometry of the Kilo-Degree Survey (KiDS) and spectroscopic data based on the Sloan Digital Sky Survey (SDSS) and Galaxy And Mass Assembly (GAMA) to infer the colour-magnitude relation of red-sequence galaxies. We then use this inferred relation to select luminous red galaxies (LRGs) in the redshift range of $0.1<z<0.7$ over the entire KiDS Data Release 3 footprint. We construct two samples of galaxies with different constant comoving densities and different luminosity thresholds. The selected red galaxies have photometric redshifts with typical photo-z errors of $\sigma_z \sim 0.014 (1+z)$ that are nearly uniform with respect to observational systematics. This makes them an ideal set of galaxies for lensing and clustering studies. As an example, we use the KiDS-450 cosmic shear catalogue to measure the mean tangential shear signal around the selected LRGs. We detect a significant weak lensing signal for lenses out to $z \sim 0.7$.

We present a weak gravitational lensing analysis of 815 square degree of $i$-band imaging from the Kilo-Degree Survey (KiDS-$i$-800). In contrast to the deep $r$-band observations, which take priority during excellent seeing conditions and form the primary KiDS dataset (KiDS-$r$-450), the complementary yet shallower KiDS-$i$-800 spans a wide range of observing conditions. The overlapping KiDS-$i$-800 and KiDS-$r$-450 imaging therefore provides a unique opportunity to assess the robustness of weak lensing measurements. In our analysis, we introduce two new `null' tests. The `nulled' two-point shear correlation function uses a matched catalogue to show that the calibrated KiDS-$i$-800 and KiDS-$r$-450 shear measurements agree at the level of $1 \pm 4$\%. We use five galaxy lens samples to determine a `nulled' galaxy-galaxy lensing signal from the full KiDS-$i$-800 and KiDS-$r$-450 surveys and find that the measurements agree to $7 \pm 5$\% when the KiDS-$i$-800 source redshift distribution is calibrated using either spectroscopic redshifts, or the 30-band photometric redshifts from the COSMOS survey.

We present a public suite of weak lensing mock data, extending the Scinet Light Cone Simulations (SLICS) to simulate cross-correlation analyses with different cosmological probes. These mocks include KiDS-450- and LSST-like lensing data, cosmic microwave background lensing maps and simulated spectroscopic surveys that emulate the GAMA, BOSS and 2dFLenS galaxy surveys. With 844 independent realisations, our mocks are optimised for combined-probe covariance estimation, which we illustrate for the case of a joint measurement involving cosmic shear, galaxy-galaxy lensing and galaxy clustering from KiDS-450 and BOSS data. With their high spatial resolution, the SLICS are also optimal for predicting the signal for novel lensing estimators, for the validation of analysis pipelines, and for testing a range of systematic effects such as the impact of neighbour-exclusion bias on the measured tomographic cosmic shear signal. For surveys like KiDS and DES, where the rejection of neighbouring galaxies occurs within ~2 arcseconds, we show that the measured cosmic shear signal will be biased low, but by less than a percent on the angular scales that are typically used in cosmic shear analyses. The amplitude of the neighbour-exclusion bias doubles in deeper, LSST-like data. The simulation products described in this paper are made available at http://slics.roe.ac.uk/.

Unlike optical CCDs, near-infrared detectors, which are based on CMOS hybrid readout technology, typically suffer from electrical crosstalk between the pixels. The interpixel capacitance (IPC) responsible for the crosstalk affects the point-spread function (PSF) of the telescope, increasing the size and modifying the shape of all objects in the images while correlating the Poisson noise. Upcoming weak lensing surveys that use these detectors, such as WFIRST, place stringent requirements on the PSF size and shape (and the level at which these are known), which in turn must be translated into requirements on IPC. To facilitate this process, we present a first study of the effect of IPC on WFIRST PSF sizes and shapes. Realistic PSFs are forward-simulated from physical principles for each WFIRST bandpass. We explore how the PSF size and shape depends on the range of IPC coupling with pixels that are connected along an edge or corner; for the expected level of IPC in WFIRST, IPC increases the PSF sizes by $\sim$5\%. We present a linear fitting formula that describes the uncertainty in the PSF size or shape due to uncertainty in the IPC, which could arise for example due to unknown time evolution of IPC as the detectors age or due to spatial variation of IPC across the detector. We also study of the effect of a small anisotropy in the IPC, which further modifies the PSF shapes. Our results are a first, critical step in determining the hardware and characterization requirements for the detectors used in the WFIRST survey.

Weak gravitational lensing (WL) is one of the most powerful techniques to learn about the dark sector of the universe. To extract the WL signal from astronomical observations, galaxy shapes must be measured and corrected for the point spread function (PSF) of the imaging system with extreme accuracy. Future WL missions (such as the Wide-Field Infrared Survey Telescope, WFIRST) will use a family of hybrid nearinfrared CMOS detectors (HAWAII-4RG) that are untested for accurate WL measurements. Like all image sensors, these devices are subject to conversion gain nonlinearities (voltage response to collected photo-charge) that bias the shape and size of bright objects such as reference stars that are used in PSF determination. We study this type of detector nonlinearity (NL) and show how to derive requirements on it from WFIRST PSF size and ellipticity requirements. We simulate the PSF optical profiles expected for WFIRST and measure the fractional error in the PSF size and the absolute error in the PSF ellipticity as a function of star magnitude and the NL model. For our nominal NL model (a quadratic correction), we find that, uncalibrated, NL can induce an error of 0.01 (fractional size) and 0.00175 (absolute ellipticity error) in the H158 bandpass for the brightest unsaturated stars in WFIRST. In addition, our simulations show that to limit the bias of the size and ellipticity errors in the H158 band to approximately 10% of the estimated WFIRST error budget, the parameter of our quadratic NL model must be calibrated to about 1% and 2.4%, respectively. We present a fitting formula that can be used to estimate WFIRST detector NL requirements once a true PSF error budget is established.

Upcoming weak lensing surveys will survey large cosmological volumes to measure the growth of cosmological structure with time and thereby constrain dark energy. One major systematic uncertainty in this process is the calibration of the weak lensing shape distortions, or shears. Most upcoming surveys plan to test several aspects of their shear estimation algorithms using sophisticated image simulations that include realistic galaxy populations based on high-resolution data from the Hubble Space Telescope (HST). However, existing datasets from the (HST) cover very small cosmological volumes, so cosmic variance could cause the galaxy populations in them to be atypical. A narrow redshift slice from such surveys could be dominated by a single large overdensity or underdensity. In that case, the morphology-density relation could alter the local galaxy populations and yield an incorrect calibration of shear estimates as a function of redshift. We directly test this scenario using the COSMOS survey, the largest-area (HST) survey to date, and show how the statistical distributions of galaxy shapes and morphological parameters (e.g., S\'{e}rsic $n$) are influenced by redshift-dependent cosmic variance. The typical variation in RMS ellipticity due to environmental effects is 5 per cent (absolute, not relative) for redshift bins of width $\Delta z=0.05$, which could result in uncertain shear calibration at the 1 per cent level. We conclude that the cosmic variance effects are large enough to exceed the systematic error budget of future surveys, but can be mitigated with careful choice of training dataset and sufficiently large redshift binning.

The GRavitational lEnsing Accuracy Testing 3 (GREAT3) challenge is the third in a series of image analysis challenges, with a goal of testing and facilitating the development of methods for analyzing astronomical images that will be used to measure weak gravitational lensing. This measurement requires extremely precise estimation of very small galaxy shape distortions, in the presence of far larger intrinsic galaxy shapes and distortions due to the blurring kernel caused by the atmosphere, telescope optics, and instrumental effects. The GREAT3 challenge is posed to the astronomy, machine learning, and statistics communities, and includes tests of three specific effects that are of immediate relevance to upcoming weak lensing surveys, two of which have never been tested in a community challenge before. These effects include realistically complex galaxy models based on high-resolution imaging from space; spatially varying, physically-motivated blurring kernel; and combination of multiple different exposures. To facilitate entry by people new to the field, and for use as a diagnostic tool, the simulation software for the challenge is publicly available, though the exact parameters used for the challenge are blinded. Sample scripts to analyze the challenge data using existing methods will also be provided. See http://great3challenge.info and http://great3.projects.phys.ucl.ac.uk/leaderboard/ for more information.