We present the methodology for a joint cosmological analysis of weak gravitational lensing from the fourth data release of the ESO Kilo-Degree Survey (KiDS-1000) and galaxy clustering from the partially overlapping BOSS and 2dFLenS surveys. Cross-correlations between galaxy positions and ellipticities are incorporated into the analysis, developing a hybrid model of non-linear scales that blends perturbative and non-perturbative approaches, and assessing signal contributions by astrophysical effects. All weak lensing signals are measured consistently via Fourier-space statistics that are insensitive to the survey mask and display low levels of mode mixing. The calibration of photometric redshift distributions and multiplicative gravitational shear bias is updated, and a more complete tally of residual calibration uncertainties is propagated into the likelihood. A dedicated suite of more than 20000 mocks is used to assess the performance of covariance models and to quantify the impact of survey geometry and spatial variations of survey depth on signals and their errors. The sampling distributions for the likelihood and the $\chi^2$ goodness-of-fit statistic are validated, with proposed changes to calculating the effective number of degrees of freedom. Standard weak lensing point estimates on $S_8=\sigma_8\,(\Omega_{\rm m}/0.3)^{1/2}$ derived from its marginal posterior are easily misinterpreted to be biased low, and an alternative estimator and associated credible interval are proposed. Known systematic effects pertaining to weak lensing modelling and inference are shown to bias $S_8$ by no more than 0.1 standard deviations, with the caveat that no conclusive validation data exist for models of intrinsic galaxy alignments. Compared to the previous KiDS analyses, $S_8$ constraints are expected to improve by 20% for weak lensing alone and by 29% for the joint analysis. [abridged]

Measuring cosmic shear in wide-field imaging surveys requires accurate knowledge of the redshift distribution of all sources. The clustering-redshift technique exploits the angular cross-correlation of a target galaxy sample with unknown redshifts and a reference sample with known redshifts, and is an attractive alternative to colour-based methods of redshift calibration. We test the performance of such clustering redshift measurements using mock catalogues that resemble the Kilo-Degree Survey (KiDS). These mocks are created from the MICE simulation and closely mimic the properties of the KiDS source sample and the overlapping spectroscopic reference samples. We quantify the performance of the clustering redshifts by comparing the cross-correlation results with the true redshift distributions in each of the five KiDS photometric redshift bins. Such a comparison to an informative model is necessary due to the incompleteness of the reference samples at high redshifts. Clustering mean redshifts are unbiased at $|\Delta z|<0.006$ under these conditions. The redshift evolution of the galaxy bias can be reliably mitigated at this level of precision using auto-correlation measurements and self-consistency relations, and will not become a dominant source of systematic error until the arrival of Stage-IV cosmic shear surveys. Using redshift distributions from a direct colour-based estimate instead of the true redshift distributions as a model for comparison with the clustering redshifts increases the biases in the mean to up to $|\Delta z|\sim0.04$. This indicates that the interpretation of clustering redshifts in real-world applications will require more sophisticated (parameterised) models of the redshift distribution in the future. If such better models are available, the clustering-redshift technique promises to be a highly complementary alternative to other methods of redshift calibration.

We present weak lensing shear catalogues from the fourth data release of the Kilo-Degree Survey, KiDS-1000, spanning 1006 square degrees of deep and high-resolution imaging. Our `gold-sample' of galaxies, with well calibrated photometric redshift distributions, consists of 21 million galaxies with an effective number density of $\sim 6.2$ galaxies per square arcminute. We quantify the accuracy of the spatial, temporal and flux-dependent point-spread function (PSF) model, verifying that the model meets our requirements to induce less than a $0.1\sigma$ change in the inferred cosmic shear constraints on the clustering cosmological parameter $S_8 = \sigma_8\sqrt{\Omega_{\rm m}/0.3}$. Through a series of two-point null-tests we validate the shear estimates, finding no evidence for significant non-lensing B-mode distortions in the data. PSF residuals are detected in the highest-redshift bins, originating from object selection and/or weight bias. The amplitude is however shown to be sufficiently low and within our stringent requirements. With a shear-ratio null-test we verify the expected redshift scaling of the galaxy-galaxy lensing signal around luminous red galaxies. We conclude that the joint KiDS-1000 shear and photometric redshift calibration is sufficiently robust for combined-probe gravitational lensing and spectroscopic clustering analyses.

The physics of gravity on cosmological scales affects both the rate of assembly of large-scale structure, and the gravitational lensing of background light through this cosmic web. By comparing the amplitude of these different observational signatures, we can construct tests that can distinguish General Relativity from its potential modifications. We use the latest weak gravitational lensing dataset from the Kilo-Degree Survey, KiDS-1000, in conjunction with overlapping galaxy spectroscopic redshift surveys BOSS and 2dFLenS, to perform the most precise existing amplitude-ratio test. We measure the associated E_G statistic with 15-20% errors, in five dz = 0.1 tomographic redshift bins in the range 0.2 < z < 0.7, on projected scales up to 100 Mpc/h. The scale-independence and redshift-dependence of these measurements are consistent with the theoretical expectation of General Relativity in a Universe with matter density Omega_m = 0.27 +/- 0.04. We demonstrate that our results are robust against different analysis choices, including schemes for correcting the effects of source photometric redshift errors, and compare the performance of angular and projected galaxy-galaxy lensing statistics.

We present updated cosmological constraints for the KiDS+VIKING-450 cosmic shear dataset (KV450), estimated using redshift distributions and photometric samples defined using self organising maps (SOMs). Our fiducial analysis finds marginal posterior constraints of $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.716^{+0.043}_{-0.038}$; smaller than, but fully consistent with, previous work using this dataset ($|\Delta S_8| = 0.023$). We analyse additional samples and redshift distributions constructed in three ways: excluding certain spectroscopic surveys during redshift calibration, excluding lower-confidence spectroscopic redshifts in redshift calibration, and considering only photometric sources which are jointly calibrated by at least three spectroscopic surveys. In all cases, the method utilised here proves robust: we find a maximal deviation from our fiducial analysis of $|\Delta S_8| \leq 0.009$ for all samples defined and analysed using our SOM. Our largest shift in $S_8$ is found when calibrating redshift distributions without the DEEP2 spectroscopic subset, where we find $S_8=0.707_{-0.042}^{+0.046}$. This difference with respect to the fiducial is both significantly smaller than, and in the opposite direction to, the equivalent shift from previous work. No sample analysed in this work results in a meaningful positive shift in $S_8$ with respect to our fiducial constraints. These results suggest that our improved cosmological parameter estimates are insensitive to pathological misrepresentation of photometric sources by the spectroscopy used for direct redshift calibration, and therefore that this systematic effect cannot be responsible for the observed difference between $S_8$ estimates made with KV450 and Planck CMB probes.

We present a combined tomographic weak gravitational lensing analysis of the Kilo Degree Survey (KV450) and the Dark Energy Survey (DES-Y1). We homogenize the analysis of these two public cosmic shear datasets by adopting consistent priors and modeling of nonlinear scales, and determine new redshift distributions for DES-Y1 based on deep public spectroscopic surveys. Adopting these revised redshifts results in a $0.8\sigma$ reduction in the DES-inferred value for $S_8$, which decreases to a $0.5\sigma$ reduction when including a systematic redshift calibration error model from mock DES data based on the MICE2 simulation. The combined KV450 + DES-Y1 constraint on $S_8 = 0.762^{+0.025}_{-0.024}$ is in tension with the Planck 2018 constraint from the cosmic microwave background at the level of $2.5\sigma$. This result highlights the importance of developing methods to provide accurate redshift calibration for current and future weak lensing surveys.

Accurate photometric redshift calibration is central to the robustness of all cosmology constraints from cosmic shear surveys. Analyses of the KiDS re-weighted training samples from all overlapping spectroscopic surveys to provide a direct redshift calibration. Using self-organising maps (SOMs) we demonstrate that this spectroscopic compilation is sufficiently complete for KiDS, representing $99\%$ of the effective 2D cosmic shear sample. We use the SOM to define a $100\%$ represented `gold' cosmic shear sample, per tomographic bin. Using mock simulations of KiDS and the spectroscopic training set, we estimate the uncertainty on the SOM redshift calibration, and find that photometric noise, sample variance, and spectroscopic selection effects (including redshift and magnitude incompleteness) induce a combined maximal scatter on the bias of the redshift distribution reconstruction ($\Delta \langle z \rangle=\langle z \rangle_{\rm est}-\langle z \rangle_{\rm true}$) of $\sigma_{\Delta \langle z \rangle} \leq 0.006$ in all tomographic bins. We show that the SOM calibration is unbiased in the cases of noiseless photometry and perfectly representative spectroscopic datasets, as expected from theory. The inclusion of both photometric noise and spectroscopic selection effects in our mock data introduces a maximal bias of $\Delta \langle z \rangle =0.013\pm0.006$, or $\Delta \langle z \rangle \leq 0.025$ at $97.5\%$ confidence, once quality flags have been applied to the SOM. The method presented here represents a significant improvement over the previously adopted direct redshift calibration implementation for KiDS, owing to its diagnostic and quality assurance capabilities. The implementation of this method in future cosmic shear studies will allow better diagnosis, examination, and mitigation of systematic biases in photometric redshift calibration.

We present cosmological constraints from a joint cosmic shear analysis of the Kilo-Degree Survey (KV450) and the Dark Energy Survey (DES-Y1), conducted using Complete Orthogonal Sets of E/B-Integrals (COSEBIs). With COSEBIs we isolate any B-modes which have a non-cosmic shear origin and demonstrate the robustness of our cosmological E-mode analysis as no significant B-modes are detected. We highlight how COSEBIs are fairly insensitive to the amplitude of the non-linear matter power spectrum at high $k$-scales, mitigating the uncertain impact of baryon feedback in our analysis. COSEBIs, therefore, allow us to utilise additional small-scale information, improving the DES-Y1 joint constraints on $S_8=\sigma_8(\Omega_{\rm m}/0.3)^{0.5}$ and $\Omega_{\rm m}$ by $20\%$. Adopting a flat $\Lambda$CDM model we find $S_8=0.755^{+0.019}_{-0.021}$, which is in $3.2\sigma$ tension with the Planck Legacy analysis of the cosmic microwave background.

We present a semi-analytic model for the shear two-point correlation function of a cosmic shear survey with non-uniform depth. Ground-based surveys are subject to depth variations that primarily arise through varying atmospheric conditions. For a survey like the Kilo-Degree Survey (KiDS), we find that the measured depth variation increases the amplitude of the observed shear correlation function at the level of a few percent out to degree-scales, relative to the assumed uniform-depth case. The impact on the inferred cosmological parameters is shown to be insignificant for a KiDS-like survey. For next-generation cosmic shear experiments, however, we conclude that variable depth should be accounted for.

We present a tomographic cosmic shear analysis of the Kilo-Degree Survey (KiDS) combined with the VISTA Kilo-Degree Infrared Galaxy Survey (VIKING). This is the first time that a full optical to near-infrared data set has been used for a wide-field cosmological weak lensing experiment. This unprecedented data, spanning $450~$deg$^2$, allows us to improve significantly the estimation of photometric redshifts, such that we are able to include robustly higher-redshift sources for the lensing measurement, and - most importantly - solidify our knowledge of the redshift distributions of the sources. Based on a flat $\Lambda$CDM model we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.737_{-0.036}^{+0.040}$ in a blind analysis from cosmic shear alone. The tension between KiDS cosmic shear and the Planck-Legacy CMB measurements remains in this systematically more robust analysis, with $S_8$ differing by $2.3\sigma$. This result is insensitive to changes in the priors on nuisance parameters for intrinsic alignment, baryon feedback, and neutrino mass. KiDS shear measurements are calibrated with a new, more realistic set of image simulations and no significant B-modes are detected in the survey, indicating that systematic errors are under control. When calibrating our redshift distributions by assuming the 30-band COSMOS-2015 photometric redshifts are correct (following the Dark Energy Survey and the Hyper Suprime-Cam Survey), we find the tension with Planck is alleviated. The robust determination of source redshift distributions remains one of the most challenging aspects for future cosmic shear surveys.

We present an analysis of the mass distribution inferred from strong lensing by SPT-CL J0356-5337, a cluster of galaxies at redshift z = 1.0359 revealed in the follow-up of the SPT-SZ clusters. The cluster has an Einstein radius of Erad=14 for a source at z = 3 and a mass within 500 kpc of M_500kpc = 4.0+-0.8x10^14Msol. Our spectroscopic identification of three multiply-imaged systems (z = 2.363, z = 2.364, and z = 3.048), combined with HST F606W-band imaging allows us to build a strong lensing model for this cluster with an rms of <0.3'' between the predicted and measured positions of the multiple images. Our modeling reveals a two-component mass distribution in the cluster. One mass component is dominated by the brightest cluster galaxy and the other component, separated by ~170 kpc, contains a group of eight red elliptical galaxies confined in a ~9'' (~70 kpc) diameter circle. We estimate the mass ratio between the two components to be between 1:1.25 and 1:1.58. In addition, spectroscopic data reveal that these two near-equal mass cores have only a small velocity difference of 300 km/s between the two components. This small radial velocity difference suggests that most of the relative velocity takes place in the plane of the sky, and implies that SPT-CL J0356-5337 is a major merger with a small impact parameter seen face-on. We also assess the relative contributions of galaxy-scale halos to the overall mass of the core of the cluster and find that within 800 kpc from the brightest cluster galaxy about 27% of the total mass can be attributed to visible and dark matter associated with galaxies, whereas only 73% of the total mass in the core comes from cluster-scale dark matter halos.

The Planck satellite has detected cluster candidates via the Sunyaev Zel'dovich (SZ) effect, but the optical follow-up required to confirm these candidates is still incomplete, especially at high redshifts and for SZ detections at low significance. In this work we present our analysis of optical observations obtained for 32 Planck cluster candidates using ACAM on the 4.2-m William Herschel Telescope. These cluster candidates were preselected using SDSS, WISE, and Pan-STARRS images to likely represent distant clusters at redshifts $z \gtrsim 0.7$. We obtain photometric redshift and richness estimates for all of the cluster candidates from a red-sequence analysis of $r$-, $i$-, and $z$-band imaging data. In addition, long-slit observations allow us to measure the redshifts of a subset of the clusters spectroscopically. The optical richness is often lower than expected from the inferred SZ mass when compared to scaling relations previously calibrated at low redshifts. This likely indicates the impact of Eddington bias and projection effects or noise-induced detections, especially at low SZ-significance. Thus, optical follow-up not only provides redshift measurements, but also an important independent verification method. We find that 18 (7) of the candidates at redshifts $z > 0.5$ ($z > 0.8$) are at least half as rich as expected from scaling relations, thereby clearly confirming these candidates as massive clusters. While the complex selection function of our sample due to our preselection hampers its use for cosmological studies, we do provide a validation of massive high-redshift clusters particularly suitable for further astrophysical investigations.

We present cosmological parameter constraints from a tomographic weak gravitational lensing analysis of ~450deg$^2$ of imaging data from the Kilo Degree Survey (KiDS). For a flat $\Lambda$CDM cosmology with a prior on $H_0$ that encompasses the most recent direct measurements, we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.745\pm0.039$. This result is in good agreement with other low redshift probes of large scale structure, including recent cosmic shear results, along with pre-Planck cosmic microwave background constraints. A $2.3$-$\sigma$ tension in $S_8$ and `substantial discordance' in the full parameter space is found with respect to the Planck 2015 results. We use shear measurements for nearly 15 million galaxies, determined with a new improved `self-calibrating' version of $lens$fit validated using an extensive suite of image simulations. Four-band $ugri$ photometric redshifts are calibrated directly with deep spectroscopic surveys. The redshift calibration is confirmed using two independent techniques based on angular cross-correlations and the properties of the photometric redshift probability distributions. Our covariance matrix is determined using an analytical approach, verified numerically with large mock galaxy catalogues. We account for uncertainties in the modelling of intrinsic galaxy alignments and the impact of baryon feedback on the shape of the non-linear matter power spectrum, in addition to the small residual uncertainties in the shear and redshift calibration. The cosmology analysis was performed blind. Our high-level data products, including shear correlation functions, covariance matrices, redshift distributions, and Monte Carlo Markov Chains are available at http://kids.strw.leidenuniv.nl.