The physics of gravity on cosmological scales affects both the rate of assembly of large-scale structure, and the gravitational lensing of background light through this cosmic web. By comparing the amplitude of these different observational signatures, we can construct tests that can distinguish General Relativity from its potential modifications. We use the latest weak gravitational lensing dataset from the Kilo-Degree Survey, KiDS-1000, in conjunction with overlapping galaxy spectroscopic redshift surveys BOSS and 2dFLenS, to perform the most precise existing amplitude-ratio test. We measure the associated E_G statistic with 15-20% errors, in five dz = 0.1 tomographic redshift bins in the range 0.2 < z < 0.7, on projected scales up to 100 Mpc/h. The scale-independence and redshift-dependence of these measurements are consistent with the theoretical expectation of General Relativity in a Universe with matter density Omega_m = 0.27 +/- 0.04. We demonstrate that our results are robust against different analysis choices, including schemes for correcting the effects of source photometric redshift errors, and compare the performance of angular and projected galaxy-galaxy lensing statistics.

For ground-based optical imaging with current CCD technology, the Poisson fluctuations in source and sky background photon arrivals dominate the noise budget and are readily estimated. Another component of noise, however, is the signal from the undetected population of stars and galaxies. Using injection of artificial galaxies into images, we demonstrate that the measured variance of galaxy moments (used for weak gravitational lensing measurements) in Dark Energy Survey (DES) images is significantly in excess of the Poisson predictions, by up to 30\%, and that the background sky levels are overestimated by current software. By cross-correlating distinct images of "empty" sky regions, we establish that there is a significant image noise contribution from undetected static sources (US), which on average are mildly resolved at DES resolution. Treating these US as a stationary noise source, we compute a correction to the moment covariance matrix expected from Poisson noise. The corrected covariance matrix matches the moment variances measured on the injected DES images to within 5\%. Thus we have an empirical method to statistically account for US in weak lensing measurements, rather than requiring extremely deep sky simulations. We also find that local sky determinations can remove the bias in flux measurements, at a small penalty in additional, but quantifiable, noise.

We present a weak lensing detection of filamentary structures in the cosmic web, combining data from the Kilo-Degree Survey, the Red Cluster Sequence Lensing Survey and the Canada-France-Hawaii Telescope Lensing Survey. The line connecting luminous red galaxies with a separation of $3 - 5\, h^{-1}\text{Mpc}$ is chosen as a proxy for the location of filaments. We measure the average weak lensing shear around $\sim$11,000 candidate filaments selected in this way from the Sloan Digital Sky Survey. After nulling the shear induced by the dark matter haloes around each galaxy, we report a $3.4\,\sigma$ detection of an anisotropic shear signal from the matter that connects them. Adopting a filament density profile, motivated from $N$-body simulations, the average density at the centre of these filamentary structures is found to be $15 \pm 4$ times the critical density.

We present a tomographic cosmic shear analysis of the Kilo-Degree Survey (KiDS) combined with the VISTA Kilo-Degree Infrared Galaxy Survey (VIKING). This is the first time that a full optical to near-infrared data set has been used for a wide-field cosmological weak lensing experiment. This unprecedented data, spanning $450~$deg$^2$, allows us to improve significantly the estimation of photometric redshifts, such that we are able to include robustly higher-redshift sources for the lensing measurement, and - most importantly - solidify our knowledge of the redshift distributions of the sources. Based on a flat $\Lambda$CDM model we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.737_{-0.036}^{+0.040}$ in a blind analysis from cosmic shear alone. The tension between KiDS cosmic shear and the Planck-Legacy CMB measurements remains in this systematically more robust analysis, with $S_8$ differing by $2.3\sigma$. This result is insensitive to changes in the priors on nuisance parameters for intrinsic alignment, baryon feedback, and neutrino mass. KiDS shear measurements are calibrated with a new, more realistic set of image simulations and no significant B-modes are detected in the survey, indicating that systematic errors are under control. When calibrating our redshift distributions by assuming the 30-band COSMOS-2015 photometric redshifts are correct (following the Dark Energy Survey and the Hyper Suprime-Cam Survey), we find the tension with Planck is alleviated. The robust determination of source redshift distributions remains one of the most challenging aspects for future cosmic shear surveys.

Wide-field imaging surveys such as the Dark Energy Survey (DES) rely on coarse measurements of spectral energy distributions in a few filters to estimate the redshift distribution of source galaxies. In this regime, sample variance, shot noise, and selection effects limit the attainable accuracy of redshift calibration and thus of cosmological constraints. We present a new method to combine wide-field, few-filter measurements with catalogs from deep fields with additional filters and sufficiently low photometric noise to break degeneracies in photometric redshifts. The multi-band deep field is used as an intermediary between wide-field observations and accurate redshifts, greatly reducing sample variance, shot noise, and selection effects. Our implementation of the method uses self-organizing maps to group galaxies into phenotypes based on their observed fluxes, and is tested using a mock DES catalog created from N-body simulations. It yields a typical uncertainty on the mean redshift in each of five tomographic bins for an idealized simulation of the DES Year 3 weak-lensing tomographic analysis of $\sigma_{\Delta z} = 0.007$, which is a 60% improvement compared to the Year 1 analysis. Although the implementation of the method is tailored to DES, its formalism can be applied to other large photometric surveys with a similar observing strategy.

In response to the need for the Astro2020 Decadal Survey to explicitly engage early career astronomers, the National Academies of Sciences, Engineering, and Medicine hosted the Early Career Astronomer and Astrophysicist Focus Session (ECFS) on October 8-9, 2018 under the auspices of Committee of Astronomy and Astrophysics. The meeting was attended by fifty six pre-tenure faculty, research scientists, postdoctoral scholars, and senior graduate students, as well as eight former decadal survey committee members, who acted as facilitators. The event was designed to educate early career astronomers about the decadal survey process, to solicit their feedback on the role that early career astronomers should play in Astro2020, and to provide a forum for the discussion of a wide range of topics regarding the astrophysics career path. This white paper presents highlights and themes that emerged during two days of discussion. In Section 1, we discuss concerns that emerged regarding the coming decade and the astrophysics career path, as well as specific recommendations from participants regarding how to address them. We have organized these concerns and suggestions into five broad themes. These include (sequentially): (1) adequately training astronomers in the statistical and computational techniques necessary in an era of "big data", (2) responses to the growth of collaborations and telescopes, (3) concerns about the adequacy of graduate and postdoctoral training, (4) the need for improvements in equity and inclusion in astronomy, and (5) smoothing and facilitating transitions between early career stages. Section 2 is focused on ideas regarding the decadal survey itself, including: incorporating early career voices, ensuring diverse input from a variety of stakeholders, and successfully and broadly disseminating the results of the survey.

We analyse three public cosmic shear surveys; the Kilo-Degree Survey (KiDS-450), the Dark Energy Survey (DES-SV) and the Canada France Hawaii Telescope Lensing Survey (CFHTLenS). Adopting the COSEBIs statistic to cleanly and completely separate the lensing E-modes from the non-lensing B-modes, we detect B-modes in KiDS-450 and CFHTLenS at the level of about 2.7 $\sigma$. For DES- SV we detect B-modes at the level of 2.8 $\sigma$ in a non-tomographic analysis, increasing to a 5.5 $\sigma$ B-mode detection in a tomographic analysis. In order to understand the origin of these detected B-modes we measure the B-mode signature of a range of different simulated systematics including PSF leakage, random but correlated PSF modelling errors, camera-based additive shear bias and photometric redshift selection bias. We show that any correlation between photometric-noise and the relative orientation of the galaxy to the point-spread-function leads to an ellipticity selection bias in tomographic analyses. This work therefore introduces a new systematic for future lensing surveys to consider. We find that the B-modes in DES-SV appear similar to a superposition of the B-mode signatures from all of the systematics simulated. The KiDS-450 and CFHTLenS B-mode measurements show features that are consistent with a repeating additive shear bias.

We present a new measurement of $E_{\rm G}$, which combines measurements of weak gravitational lensing, real-space galaxy clustering and redshift space distortions. This statistic was proposed as a consistency test of General Relativity (GR) that is insensitive to linear, deterministic galaxy bias and the matter clustering amplitude. We combine deep imaging data from KiDS with overlapping spectroscopy from 2dFLenS, BOSS DR12 and GAMA and find $E_{\rm G}(\overline{z}=0.267)=0.43 \pm 0.13$ (GAMA), $E_{\rm G}(\overline{z}=0.305)=0.27 \pm 0.08$ (LOWZ+2dFLOZ) and $E_{\rm G}(\overline{z}=0.554)=0.26 \pm 0.07$ (CMASS+2dFHIZ). We demonstrate that the existing tension in the value of the matter density parameter hinders the robustness of this statistic as solely a test of GR. We find that our $E_{\rm G}$ measurements, as well as existing ones in the literature, favour a lower matter density cosmology than the Cosmic Microwave Background. For a flat $\Lambda$CDM Universe and assuming GR, we find $\Omega_{\rm m}(z=0)=0.25\pm0.03$. With this paper we publicly release the 2dFLenS dataset at: \url{http://2dflens.swin.edu.au}.

We present a weak gravitational lensing analysis of 815 square degree of $i$-band imaging from the Kilo-Degree Survey (KiDS-$i$-800). In contrast to the deep $r$-band observations, which take priority during excellent seeing conditions and form the primary KiDS dataset (KiDS-$r$-450), the complementary yet shallower KiDS-$i$-800 spans a wide range of observing conditions. The overlapping KiDS-$i$-800 and KiDS-$r$-450 imaging therefore provides a unique opportunity to assess the robustness of weak lensing measurements. In our analysis, we introduce two new `null' tests. The `nulled' two-point shear correlation function uses a matched catalogue to show that the calibrated KiDS-$i$-800 and KiDS-$r$-450 shear measurements agree at the level of $1 \pm 4$\%. We use five galaxy lens samples to determine a `nulled' galaxy-galaxy lensing signal from the full KiDS-$i$-800 and KiDS-$r$-450 surveys and find that the measurements agree to $7 \pm 5$\% when the KiDS-$i$-800 source redshift distribution is calibrated using either spectroscopic redshifts, or the 30-band photometric redshifts from the COSMOS survey.

We present a public suite of weak lensing mock data, extending the Scinet Light Cone Simulations (SLICS) to simulate cross-correlation analyses with different cosmological probes. These mocks include KiDS-450- and LSST-like lensing data, cosmic microwave background lensing maps and simulated spectroscopic surveys that emulate the GAMA, BOSS and 2dFLenS galaxy surveys. With 844 independent realisations, our mocks are optimised for combined-probe covariance estimation, which we illustrate for the case of a joint measurement involving cosmic shear, galaxy-galaxy lensing and galaxy clustering from KiDS-450 and BOSS data. With their high spatial resolution, the SLICS are also optimal for predicting the signal for novel lensing estimators, for the validation of analysis pipelines, and for testing a range of systematic effects such as the impact of neighbour-exclusion bias on the measured tomographic cosmic shear signal. For surveys like KiDS and DES, where the rejection of neighbouring galaxies occurs within ~2 arcseconds, we show that the measured cosmic shear signal will be biased low, but by less than a percent on the angular scales that are typically used in cosmic shear analyses. The amplitude of the neighbour-exclusion bias doubles in deeper, LSST-like data. The simulation products described in this paper are made available at http://slics.roe.ac.uk/.

We present the first "clipped" cosmic shear measurement using data from the Kilo-Degree Survey (KiDS-450). "Clipping" transformations suppress the signal from the highest density, non-linear regions of cosmological fields. We demonstrate that these transformations improve constraints on $S_8=\sigma_8(\Omega_{\rm{m}}/0.3)^{0.5}$ when used in combination with conventional two-point statistics. For the KiDS-450 data, we find that the combined measurements improve the constraints on $S_8$ by 17%, compared to shear correlation functions alone. We determine the expectation value of the clipped shear correlation function using a suite of numerical simulations, and develop methodology to mitigate the impact of masking and shot noise. Future improvements in numerical simulations and mass reconstruction methodology will permit the precise calibration of clipped cosmic shear statistics such that clipping can become a standard tool in weak lensing analyses.

We measure the projected galaxy clustering and galaxy-galaxy lensing signals using the Galaxy And Mass Assembly (GAMA) survey and Kilo-Degree Survey (KiDS) to study galaxy bias. We use the concept of non-linear and stochastic galaxy biasing in the framework of halo occupation statistics to constrain the parameters of the halo occupation statistics and to unveil the origin of galaxy biasing. The bias function $\Gamma_{\text{gm}}(r_{\text{p}})$, where $r_{\text{p}}$ is the projected comoving separation, is evaluated using the analytical halo model from which the scale dependence of $\Gamma_{\text{gm}}(r_{\text{p}})$, and the origin of the non-linearity and stochasticity in halo occupation models can be inferred. Our observations unveil the physical reason for the non-linearity and stochasticity, further explored using hydrodynamical simulations, with the stochasticity mostly originating from the non-Poissonian behaviour of satellite galaxies in the dark matter haloes and their spatial distribution, which does not follow the spatial distribution of dark matter in the halo. The observed non-linearity is mostly due to the presence of the central galaxies, as was noted from previous theoretical work on the same topic. We also see that overall, more massive galaxies reveal a stronger scale dependence, and out to a larger radius. Our results show that a wealth of information about galaxy bias is hidden in halo occupation models. These models should therefore be used to determine the influence of galaxy bias in cosmological studies.

We present a machine-learning photometric redshift analysis of the Kilo-Degree Survey Data Release 3, using two neural-network based techniques: ANNz2 and MLPQNA. Despite limited coverage of spectroscopic training sets, these ML codes provide photo-zs of quality comparable to, if not better than, those from the BPZ code, at least up to zphot<0.9 and r<23.5. At the bright end of r<20, where very complete spectroscopic data overlapping with KiDS are available, the performance of the ML photo-zs clearly surpasses that of BPZ, currently the primary photo-z method for KiDS. Using the Galaxy And Mass Assembly (GAMA) spectroscopic survey as calibration, we furthermore study how photo-zs improve for bright sources when photometric parameters additional to magnitudes are included in the photo-z derivation, as well as when VIKING and WISE infrared bands are added. While the fiducial four-band ugri setup gives a photo-z bias $\delta z=-2e-4$ and scatter $\sigma_z<0.022$ at mean z = 0.23, combining magnitudes, colours, and galaxy sizes reduces the scatter by ~7% and the bias by an order of magnitude. Once the ugri and IR magnitudes are joined into 12-band photometry spanning up to 12 $\mu$, the scatter decreases by more than 10% over the fiducial case. Finally, using the 12 bands together with optical colours and linear sizes gives $\delta z<4e-5$ and $\sigma_z<0.019$. This paper also serves as a reference for two public photo-z catalogues accompanying KiDS DR3, both obtained using the ANNz2 code. The first one, of general purpose, includes all the 39 million KiDS sources with four-band ugri measurements in DR3. The second dataset, optimized for low-redshift studies such as galaxy-galaxy lensing, is limited to r<20, and provides photo-zs of much better quality than in the full-depth case thanks to incorporating optical magnitudes, colours, and sizes in the GAMA-calibrated photo-z derivation.

We perform a combined analysis of cosmic shear tomography, galaxy-galaxy lensing tomography, and redshift-space multipole power spectra (monopole and quadrupole) using 450 deg$^2$ of imaging data by the Kilo Degree Survey (KiDS) overlapping with two spectroscopic surveys: the 2-degree Field Lensing Survey (2dFLenS) and the Baryon Oscillation Spectroscopic Survey (BOSS). We restrict the galaxy-galaxy lensing and multipole power spectrum measurements to the overlapping regions with KiDS, and self-consistently compute the full covariance between the different observables using a large suite of $N$-body simulations. We methodically analyze different combinations of the observables, finding that galaxy-galaxy lensing measurements are particularly useful in improving the constraint on the intrinsic alignment amplitude (by 30%, positive at $3.5\sigma$ in the fiducial data analysis), while the multipole power spectra are useful in tightening the constraints along the lensing degeneracy direction (e.g. factor of two stronger matter density constraint in the fiducial analysis). The fully combined constraint on $S_8 \equiv \sigma_8 \sqrt{\Omega_{\rm m}/0.3} = 0.742 \pm 0.035$, which is an improvement by 20% compared to KiDS alone, corresponds to a $2.6\sigma$ discordance with Planck, and is not significantly affected by fitting to a more conservative set of scales. Given the tightening of the parameter space, we are unable to resolve the discordance with an extended cosmology that is simultaneously favored in a model selection sense, including the sum of neutrino masses, curvature, evolving dark energy, and modified gravity. The complementarity of our observables allows for constraints on modified gravity degrees of freedom that are not simultaneously bounded with either probe alone, and up to a factor of three improvement in the $S_8$ constraint in the extended cosmology compared to KiDS alone.

The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products, adding further area, homogenized photometric calibration, photometric redshifts and weak lensing shear measurements to the first two releases. A dedicated pipeline embedded in the Astro-WISE information system is used for the production of the main release. Modifications with respect to earlier releases are described in detail. Photometric redshifts have been derived using both Bayesian template fitting, and machine-learning techniques. For the weak lensing measurements, optimized procedures based on the THELI data reduction and lensfit shear measurement packages are used. In DR3 stacked ugri images, weight maps, masks, and source lists for 292 new survey tiles (~300 sq.deg) are made available. The multi-band catalogue, including homogenized photometry and photometric redshifts, covers the combined DR1, DR2 and DR3 footprint of 440 survey tiles (447 sq.deg). Limiting magnitudes are typically 24.3, 25.1, 24.9, 23.8 (5 sigma in a 2 arcsec aperture) in ugri, respectively, and the typical r-band PSF size is less than 0.7 arcsec. The photometric homogenization scheme ensures accurate colors and an absolute calibration stable to ~2% for gri and ~3% in u. Separately released are a weak lensing shear catalogue and photometric redshifts based on two different machine-learning techniques.

We present a new training set for estimating empirical photometric redshifts of galaxies, which was created as part of the 2dFLenS project. This training set is located in a 700 sq deg area of the KiDS South field and is randomly selected and nearly complete at r<19.5. We investigate the photometric redshift performance obtained with ugriz photometry from VST-ATLAS and W1/W2 from WISE, based on several empirical and template methods. The best redshift errors are obtained with kernel-density estimation, as are the lowest biases, which are consistent with zero within statistical noise. The 68th percentiles of the redshift scatter for magnitude-limited samples at r<(15.5, 17.5, 19.5) are (0.014, 0.017, 0.028). In this magnitude range, there are no known ambiguities in the colour-redshift map, consistent with a small rate of redshift outliers. In the fainter regime, the KDE method produces p(z) estimates per galaxy that represent unbiased and accurate redshift frequency expectations. The p(z) sum over any subsample is consistent with the true redshift frequency plus Poisson noise. Further improvements in redshift precision at r<20 would mostly be expected from filter sets with narrower passbands to increase the sensitivity of colours to small changes in redshift.

We develop a statistical estimator to infer the redshift probability distribution of a photometric sample of galaxies from its angular cross-correlation in redshift bins with an overlapping spectroscopic sample. This estimator is a minimum variance weighted quadratic function of the data: a quadratic estimator. This extends and modifies the methodology presented by McQuinn & White (2013). The derived source redshift distribution is degenerate with the source galaxy bias, which must be constrained via additional assumptions. We apply this estimator to constrain source galaxy redshift distributions in the Kilo-Degree imaging survey through cross-correlation with the spectroscopic 2-degree Field Lensing Survey, presenting results first as a binned step-wise distribution in the range z < 0.8, and then building a continuous distribution using a Gaussian process model. We demonstrate the robustness of our methodology using mock catalogues constructed from N-body simulations, and comparisons with other techniques for inferring the redshift distribution.

We present cosmological parameter constraints from a tomographic weak gravitational lensing analysis of ~450deg$^2$ of imaging data from the Kilo Degree Survey (KiDS). For a flat $\Lambda$CDM cosmology with a prior on $H_0$ that encompasses the most recent direct measurements, we find $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.745\pm0.039$. This result is in good agreement with other low redshift probes of large scale structure, including recent cosmic shear results, along with pre-Planck cosmic microwave background constraints. A $2.3$-$\sigma$ tension in $S_8$ and `substantial discordance' in the full parameter space is found with respect to the Planck 2015 results. We use shear measurements for nearly 15 million galaxies, determined with a new improved `self-calibrating' version of $lens$fit validated using an extensive suite of image simulations. Four-band $ugri$ photometric redshifts are calibrated directly with deep spectroscopic surveys. The redshift calibration is confirmed using two independent techniques based on angular cross-correlations and the properties of the photometric redshift probability distributions. Our covariance matrix is determined using an analytical approach, verified numerically with large mock galaxy catalogues. We account for uncertainties in the modelling of intrinsic galaxy alignments and the impact of baryon feedback on the shape of the non-linear matter power spectrum, in addition to the small residual uncertainties in the shear and redshift calibration. The cosmology analysis was performed blind. Our high-level data products, including shear correlation functions, covariance matrices, redshift distributions, and Monte Carlo Markov Chains are available at http://kids.strw.leidenuniv.nl.

We present the 2-degree Field Lensing Survey (2dFLenS), a new galaxy redshift survey performed at the Anglo-Australian Telescope. 2dFLenS is the first wide-area spectroscopic survey specifically targeting the area mapped by deep-imaging gravitational lensing fields, in this case the Kilo-Degree Survey. 2dFLenS obtained 70,079 redshifts in the range z < 0.9 over an area of 731 sq deg, and is designed to extend the datasets available for testing gravitational physics and promote the development of relevant algorithms for joint imaging and spectroscopic analysis. The redshift sample consists first of 40,531 Luminous Red Galaxies (LRGs), which enable analyses of galaxy-galaxy lensing, redshift-space distortion, and the overlapping source redshift distribution by cross-correlation. An additional 28,269 redshifts form a magnitude-limited (r < 19.5) nearly-complete sub-sample, allowing direct source classification and photometric-redshift calibration. In this paper, we describe the motivation, target selection, spectroscopic observations, and clustering analysis of 2dFLenS. We use power spectrum multipole measurements to fit the redshift-space distortion parameter of the LRG sample in two redshift ranges 0.15 < z < 0.43 and 0.43 < z < 0.7 as beta = 0.49 +/- 0.15 and beta = 0.26 +/- 0.09, respectively. These values are consistent with those obtained from LRGs in the Baryon Oscillation Spectroscopic Survey. 2dFLenS data products will be released via our website http://2dflens.swin.edu.au.

The Kilo-Degree Survey (KiDS) is a multi-band imaging survey designed for cosmological studies from weak lensing and photometric redshifts. It uses the ESO VLT Survey Telescope with its wide-field camera OmegaCAM. KiDS images are taken in four filters similar to the SDSS ugri bands. The best-seeing time is reserved for deep r-band observations that reach a median 5-sigma limiting AB magnitude of 24.9 with a median seeing that is better than 0.7arcsec. Initial KiDS observations have concentrated on the GAMA regions near the celestial equator, where extensive, highly complete redshift catalogues are available. A total of 109 survey tiles, one square degree each, form the basis of the first set of lensing analyses, which focus on measurements of halo properties of GAMA galaxies. 9 galaxies per square arcminute enter the lensing analysis, for an effective inverse shear variance of 69 per square arcminute. Accounting for the shape measurement weight, the median redshift of the sources is 0.53. KiDS data processing follows two parallel tracks, one optimized for galaxy shape measurement (for weak lensing), and one for accurate matched-aperture photometry in four bands (for photometric redshifts). This technical paper describes how the lensing and photometric redshift catalogues have been produced (including an extensive description of the Gaussian Aperture and Photometry pipeline), summarizes the data quality, and presents extensive tests for systematic errors that might affect the lensing analyses. We also provide first demonstrations of the suitability of the data for cosmological measurements, and explain how the shear catalogues were blinded to prevent confirmation bias in the scientific analyses. The KiDS shear and photometric redshift catalogues, presented in this paper, are released to the community through http://kids.strw.leidenuniv.nl .