We present the methodology for a joint cosmological analysis of weak gravitational lensing from the fourth data release of the ESO Kilo-Degree Survey (KiDS-1000) and galaxy clustering from the partially overlapping BOSS and 2dFLenS surveys. Cross-correlations between galaxy positions and ellipticities are incorporated into the analysis, developing a hybrid model of non-linear scales that blends perturbative and non-perturbative approaches, and assessing signal contributions by astrophysical effects. All weak lensing signals are measured consistently via Fourier-space statistics that are insensitive to the survey mask and display low levels of mode mixing. The calibration of photometric redshift distributions and multiplicative gravitational shear bias is updated, and a more complete tally of residual calibration uncertainties is propagated into the likelihood. A dedicated suite of more than 20000 mocks is used to assess the performance of covariance models and to quantify the impact of survey geometry and spatial variations of survey depth on signals and their errors. The sampling distributions for the likelihood and the $\chi^2$ goodness-of-fit statistic are validated, with proposed changes to calculating the effective number of degrees of freedom. Standard weak lensing point estimates on $S_8=\sigma_8\,(\Omega_{\rm m}/0.3)^{1/2}$ derived from its marginal posterior are easily misinterpreted to be biased low, and an alternative estimator and associated credible interval are proposed. Known systematic effects pertaining to weak lensing modelling and inference are shown to bias $S_8$ by no more than 0.1 standard deviations, with the caveat that no conclusive validation data exist for models of intrinsic galaxy alignments. Compared to the previous KiDS analyses, $S_8$ constraints are expected to improve by 20% for weak lensing alone and by 29% for the joint analysis. [abridged]

We test the regime of validity of one-loop galaxy bias for a wide variety of biased tracers. Our most stringent test asks the bias model to simultaneously match the galaxy-galaxy and galaxy-mass spectrum, using the measured nonlinear matter spectrum from the simulations to test one-loop effects from the bias expansion alone. In addition, we investigate the relevance of short-range nonlocality and halo exclusion through higher-derivative and scale-dependent noise terms, as well as the impact of using co-evolution relations to reduce the number of free fitting parameters. From comparing validity and merit of these assumptions we find that a four-parameter model (linear, quadratic, cubic nonlocal bias, and constant shot noise) with fixed quadratic tidal bias provides a robust modeling choice for the auto power spectrum of the less massive halos in our set of samples and their galaxy populations (up to $k_{\mathrm{max}} = 0.35\,h/\mathrm{Mpc}$ for a sample volume of $6\,(\mathrm{Gpc}/h)^3$). For the more biased tracers it is most beneficial to include scale-dependent noise. This is also the preferred option when considering combinations of the auto and cross power spectrum, which might be relevant in joint studies of galaxy clustering and weak lensing. We also test the use of perturbation theory to account for matter loops through gRPT, EFT and the hybrid approach RESPRESSO. While all these have similar performance, we find the latter to be the best in terms of validity and recovered mean posterior values, in accordance with it being based partially on simulations.

We compute the covariance of the galaxy power spectrum multipoles in perturbation theory, including the effects of nonlinear evolution, nonlinear and nonlocal bias, radial redshift-space distortions, arbitrary survey window and shot noise. We rewrite the power spectrum FKP estimator in terms of the usual windowed galaxy fluctuations and the fluctuations in the number of galaxies inside the survey volume. We show that this leads to a stronger super-sample covariance than assumed in the literature and causes a substantial leakage of Gaussian information. We decompose the covariance matrix into several contributions that provide an insight into its behavior for different biased tracers. We show that for realistic surveys, the covariance of power spectrum multipoles is already dominated by shot noise and super survey mode coupling in the weakly non-linear regime. Both these effects can be accurately modeled analytically, making a perturbative treatment of the covariance very compelling. Our method allows for the covariance to be varied as a function of cosmology and bias parameters very efficiently, with survey geometry entering as fixed kernels that can be computed separately using FFTs. We find excellent agreement between our analytic covariance and that estimated from BOSS DR12 Patchy mock catalogs in the whole range we tested, up to $k=0.6$ h/Mpc. This bodes well for application to future surveys such as DESI and Euclid.

Massive neutrinos suppress the growth of structure below their free-streaming scale and leave an imprint on large-scale structure. Measuring this imprint allows us to constrain the sum of neutrino masses, $M_\nu$, a key parameter in particle physics beyond the Standard Model. However, degeneracies among cosmological parameters, especially between $M_\nu$ and $\sigma_8$, limit the constraining power of standard two-point clustering statistics. In this work, we investigate whether we can break these degeneracies and constrain $\smnu$ with the next higher-order correlation function --- the bispectrum. We first examine the redshift-space halo bispectrum of $800$ $N$-body simulations from the HADES suite and demonstrate that the bispectrum helps break the $M_\nu$--$\sigma_8$ degeneracy. Then using 22,000 $N$-body simulations of the Quijote suite, we quantify for the first time the full information content of the redshift-space halo bispectrum down to nonlinear scales using a Fisher matrix forecast of $\{\Omega_m$, $\Omega_b$, $h$, $n_s$, $\sigma_8$, $M_\nu\}$. For $k_{\rm max}{=}0.5~h/{\rm Mpc}$, the bispectrum provides $\Omega_m$, $\Omega_b$, $h$, $n_s$, and $\sigma_8$ constraints 1.9, 2.6, 3.1, 3.6, and 2.6 times tighter than the power spectrum. For $M_\nu$, the bispectrum improves the 1$\sigma$ constraint from 0.2968 to 0.0572 eV --- over 5 times tighter than the power spectrum. Even with priors from {\em Planck}, the bispectrum improves $M_\nu$ constraints by a factor of 1.8. Although we reserve marginalizing over a more complete set of bias parameters to the next paper of the series, these constraints are derived for a $(1~h^{-1}{\rm Gpc})^3$ box, a substantially smaller volume than upcoming surveys. Thus, our results demonstrate that the bispectrum offers significant improvements over the power spectrum, especially for constraining $M_\nu$.

The Quijote simulations are a set of 43100 full N-body simulations spanning more than 7000 cosmological models in the $\{\Omega_{\rm m}, \Omega_{\rm b}, h, n_s, \sigma_8, M_\nu, w \}$ hyperplane. At a single redshift the simulations contain more than 8.5 trillions of particles over a combined volume of 43100 $(h^{-1}{\rm Gpc})^3$. Billions of dark matter halos and cosmic voids have been identified in the simulations, whose runs required more than 35 million core hours. The Quijote simulations have been designed for two main purposes: 1) to quantify the information content on cosmological observables, and 2) to provide enough data to train machine learning algorithms. In this paper we describe the simulations and show a few of their applications. We also release the Petabyte of data generated, comprising hundreds of thousands of simulation snapshots at multiple redshifts, halo and void catalogs, together with millions of summary statistics such as power spectra, bispectra, correlation functions, marked power spectra, and estimated probability density functions.

In this paper, we verify the large scale structure consistency relations using N-body simulations, including modes in the highly non-linear regime. These relations (pointed out by Kehagias & Riotto and Peloso & Pietroni) follow from the symmetry of the dynamics under a shift of the Newtonian potential by a constant and a linear gradient, and predict the absence of certain poles in the ratio between the (equal time) squeezed bispectrum and power spectrum. The consistency relations, as symmetry statements, are exact, but have not been previously checked beyond the perturbative regime. Our test using N-body simulations not only offers a non-perturbative check, but also serves as a warm-up exercise for applications to observational data. A number of subtleties arise when taking the squeezed limit of the bispectrum--we show how to circumvent or address them. An interesting by-product of our investigation is an explicit demonstration that the linear-gradient symmetry is unaffected by the periodic boundary condition of the simulations. Lastly, we verify using simulations that the consistency relations are violated when the initial conditions are non-gaussian (of the local fNL type). The methodology developed here paves the way for constraining primordial non-gaussianity using large scale structure data, including (numerous) highly non-linear modes that are otherwise hard to interpret and utilize.

Combination of the power spectrum and bispectrum is a powerful way of breaking degeneracies between galaxy bias and cosmological parameters, enabling us to maximize the constraining power from galaxy surveys. Recent cosmological constraints treat the power spectrum and bispectrum on an uneven footing: they include one-loop bias corrections for the power spectrum but not the bispectrum. To bridge this gap, we develop the galaxy bias description up to fourth order in perturbation theory, conveniently expressed through a basis of Galilean invariants that clearly split contributions that are local and nonlocal in the second derivatives of the linear gravitational potential. In addition, we consider relevant contributions from short-range nonlocality (higher-derivative terms), stress-tensor corrections and stochasticity. To sidestep the usual renormalization of bias parameters that complicates predictions beyond leading order, we recast the bias expansion in terms of multipoint propagators, which take a simple form in our split-basis with loop corrections depending only on bias parameters corresponding to nonlocal operators. We show how to take advantage of Galilean invariance to compute the time evolution of bias and present results for the fourth-order parameters for the first time. We also discuss the possibilities of reducing the bias parameter space by using the evolution of bias and exploiting degeneracies between various bias contributions in the large-scale limit. Our baseline model allows to verify these simplifications for any application to large-scale structure data sets.

Interpreting the small-scale clustering of galaxies with halo models can elucidate the connection between galaxies and dark matter halos. Unfortunately, the modelling is typically not sufficiently accurate for ruling out models statistically. It is thus difficult to use the information encoded in small scales to test cosmological models or probe subtle features of the galaxy-halo connection. In this paper, we attempt to push halo modelling into the "accurate" regime with a fully numerical mock-based methodology and careful treatment of statistical and systematic errors. With our forward-modelling approach, we can incorporate clustering statistics beyond the traditional two-point statistics. We use this modelling methodology to test the standard $\Lambda\mathrm{CDM}$ + halo model against the clustering of SDSS DR7 galaxies. Specifically, we use the projected correlation function, group multiplicity function and galaxy number density as constraints. We find that while the model fits each statistic separately, it struggles to fit them simultaneously. Adding group statistics leads to a more stringent test of the model and significantly tighter constraints on model parameters. We explore the impact of varying the adopted halo definition and cosmological model and find that changing the cosmology makes a significant difference. The most successful model we tried (Planck cosmology with Mvir halos) matches the clustering of low luminosity galaxies, but exhibits a 2.3$\sigma$ tension with the clustering of luminous galaxies, thus providing evidence that the "standard" halo model needs to be extended. This work opens the door to adding interesting freedom to the halo model and including additional clustering statistics as constraints.

We explore the cosmological implications of anisotropic clustering measurements of the quasar sample from Data Release 14 of the Sloan Digital Sky Survey IV Extended Baryon Oscillation Spectroscopic Survey (eBOSS) in configuration space. The $\sim 147,000$ quasar sample observed by eBOSS offers a direct tracer of the density field and bridges the gap of previous BAO measurements between redshift $0.8<z<2.2$. By analysing the two-point correlation function characterized by clustering wedges $\xi_{\rm w_i}(s)$ and multipoles $\xi_{\ell}(s)$, we measure the angular diameter distance, Hubble parameter and cosmic structure growth rate. We define a systematic error budget for our measurements based on the analysis of $N$-body simulations and mock catalogues. Based on the DR14 large scale structure quasar sample at the effective redshift $z_{\rm eff}=1.52$, we find the growth rate of cosmic structure $f\sigma_8(z_{\rm eff})=0.396\pm 0.079$, and the geometric parameters $D_{\rm V}(z)/r_{\rm d}=26.47\pm 1.23$, and $F_{\rm AP}(z)=2.53\pm 0.22$, where the uncertainties include both statistical and systematic errors. These values are in excellent agreement with the best-fitting standard ${\rm \Lambda CDM}$ model to the latest cosmic microwave background data from Planck.

We compare reduced three-point correlations $Q$ of matter, haloes (as proxies for galaxies) and their cross correlations, measured in a total simulated volume of $\sim100 \ (h^{-1} \text{Gpc})^{3}$, to predictions from leading order perturbation theory on a large range of scales in configuration space. Predictions for haloes are based on the non-local bias model, employing linear ($b_1$) and non-linear ($c_2$, $g_2$) bias parameters, which have been constrained previously from the bispectrum in Fourier space. We also study predictions from two other bias models, one local ($g_2=0$) and one in which $c_2$ and $g_2$ are determined by $b_1$ via an approximately universal relation. Overall, measurements and predictions agree when $Q$ is derived for triangles with $(r_1r_2r_3)^{1/3} \gtrsim 60 h^{-1}\text{Mpc}$, where $r_{1-3}$ are the sizes of the triangle legs. Predictions for $Q_{matter}$, based on the linear power spectrum, show significant deviations from the measurements at the BAO scale (given our small measurement errors), which strongly decrease when adding a damping term or using the non-linear power spectrum, as expected. Predictions for $Q_{halo}$ agree best with measurements at large scales when considering non-local contributions. The universal bias model works well for haloes and might therefore be also useful for tightening constraints on $b_1$ from $Q$ in galaxy surveys. Such constraints are independent of the amplitude of matter density fluctuation ($\sigma_8$) and hence break the degeneracy between $b_1$ and $\sigma_8$, present in galaxy two-point correlations.

The window function for protohalos in Lagrangian space is often assumed to be a tophat in real space. We measure this profile directly and find that it is more extended than a tophat but less extended than a Gaussian; its shape is well-described by rounding the edges of the tophat by convolution with a Gaussian that has a scale length about 5 times smaller. This effective window $W_{\rm eff}$ is particularly simple in Fourier space, and has an analytic form in real space. Together with the excursion set bias parameters, $W_{\rm eff}$ describes the scale-dependence of the Lagrangian halo-matter cross correlation up to $kR_{\rm Lag} \sim 10 $, where $R_{\rm Lag}$ is the Lagrangian size of the protohalo. Moreover, with this $W_{\rm eff}$, all the spectral moments of the power spectrum are finite, allowing a straightforward estimate of the excursion set peak mass function. This estimate requires a prescription of the critical overdensity enclosed within a protohalo if it is to collapse, which we calibrate from simulations. We find that the resulting estimate of halo abundances is only accurate to about 20%, and we discuss why: A tophat in `infall time' towards the protohalo center need not correspond to a tophat in the initial spatial distribution, so models in which infall rather than smoothed overdensity is the relevant variable may be more accurate.

Efficient estimators of Fourier-space statistics for large number of objects rely on Fast Fourier Transforms (FFTs), which are affected by aliasing from unresolved small scale modes due to the finite FFT grid. Aliasing takes the form of a sum over images, each of them corresponding to the Fourier content displaced by increasing multiples of the sampling frequency of the grid. These spurious contributions limit the accuracy in the estimation of Fourier-space statistics, and are typically ameliorated by simultaneously increasing grid size and discarding high-frequency modes. This results in inefficient estimates for e.g. the power spectrum when desired systematic biases are well under per-cent level. We show that using interlaced grids removes odd images, which include the dominant contribution to aliasing. In addition, we discuss the choice of interpolation kernel used to define density perturbations on the FFT grid and demonstrate that using higher-order interpolation kernels than the standard Cloud in Cell algorithm results in significant reduction of the remaining images. We show that combining fourth-order interpolation with interlacing gives very accurate Fourier amplitudes and phases of density perturbations. This results in power spectrum and bispectrum estimates that have systematic biases below 0.01% all the way to the Nyquist frequency of the grid, thus maximizing the use of unbiased Fourier coefficients for a given grid size and greatly reducing systematics for applications to large cosmological datasets.

(abridged) We investigate the signatures left by the cosmic neutrino background on the clustering of matter, CDM+baryons and halos in redshift-space using a set of more than 1000 N-body and hydrodynamical simulations with massless and massive neutrinos. We find that the effect neutrinos induce on the clustering of CDM+baryons in redshift-space on small scales is almost entirely due to the change in $\sigma_8$. Neutrinos imprint a characteristic signature in the quadrupole of the matter (CDM+baryons+neutrinos) field on small scales, that can be used to disentangle the effect of $\sigma_8$ and $M_\nu$. We show that the effect of neutrinos on the clustering of halos is very different, on all scales, to the one induced by $\sigma_8$. We find that the effects of neutrinos of the growth rate of CDM+baryons ranges from $\sim0.3\%$ to $2\%$ on scales $k\in[0.01, 0.5]~h{\rm Mpc}^{-1}$ for neutrinos with masses $M_\nu \leqslant 0.15$ eV. We compute the bias between the momentum of halos and the momentum of CDM+baryon and find it to be 1 on large scales for all models with massless and massive neutrinos considered. This point towards a velocity bias between halos and total matter on large scales that it is important to account for in order to extract unbiased neutrino information from velocity/momentum surveys such as kSZ observations. We show that baryonic effects can affect the clustering of matter and CDM+baryons in redshift-space by up to a few percent down to $k=0.5~h{\rm Mpc}^{-1}$. We find that hydrodynamics and astrophysical processes, as implemented in our simulations, only distort the relative effect that neutrinos induce on the anisotropic clustering of matter, CDM+baryons and halos in redshift-space by less than $1\%$. Thus, the effect of neutrinos in the fully non-linear regime can be written as a transfer function with very weak dependence on astrophysics.

We investigate the cosmological implications of studying galaxy clustering using a tomographic approach applied to the final BOSS DR12 galaxy sample, including both auto- and cross-correlation functions between redshift shells. We model the signal of the full shape of the angular correlation function, $\omega(\theta)$, in redshift bins using state-of-the-art modelling of non-linearities, bias and redshift-space distortions. We present results on the redshift evolution of the linear bias of BOSS galaxies, which cannot be obtained with traditional methods for galaxy-clustering analysis. We also obtain constraints on cosmological parameters, combining this tomographic analysis with measurements of the cosmic microwave background (CMB) and type Ia supernova (SNIa). We explore a number of cosmological models, including the standard $\Lambda$CDM model and its most interesting extensions, such as deviations from $w_\rm{DE} = -1$, non-minimal neutrino masses, spatial curvature and deviations from general relativity using the growth-index $\gamma$ parametrisation. These results are, in general, comparable to the most precise present-day constraints on cosmological parameters, and show very good agreement with the standard model. In particular, combining CMB, $\omega(\theta)$ and SNIa, we find a value of $w_\rm{DE}$ consistent with $-1$ to a precision better than 5\% when it is assumed to be constant in time, and better than 6\% when we also allow for a spatially-curved Universe.

The protohalo patches from which halos form are defined by a number of constraints imposed on the Lagrangian dark matter density field. Each of these constraints contributes to biasing the spatial distribution of the protohalos relative to the matter. We show how measurements of this spatial distribution -- linear combinations of protohalo bias factors -- can be used to make inferences about the physics of halo formation. Our analysis exploits the fact that halo bias factors satisfy consistency relations which encode this physics, and that these relations are the same even for sub-populations in which assembly bias has played a role. We illustrate our methods using a model in which three parameters matter: a density threshold, the local slope and the curvature of the smoothed density field. The latter two are nearly degenerate; our approach naturally allows one to build an accurate effective two-parameter model for which the consistency relations still apply. This, with an accurate description of the smoothing window, allows one to describe the protohalo-matter cross-correlation very well, both in Fourier and configuration space. We then use our determination of the large scale bias parameters together with the consistency relations, to estimate the enclosed density and mean slope on the Lagrangian radius scale of the protohalos. Direct measurements of these quantities, made on smaller scales than those on which the bias parameters are typically measured, are in good agreement.

We extract cosmological information from the anisotropic power spectrum measurements from the recently completed Baryon Oscillation Spectroscopic Survey (BOSS), extending the concept of clustering wedges to Fourier space. Making use of new FFT-based estimators, we measure the power spectrum clustering wedges of the BOSS sample by filtering out the information of Legendre multipoles l > 4. Our modelling of these measurements is based on novel approaches to describe non-linear evolution, bias, and redshift-space distortions, which we test using synthetic catalogues based on large-volume N-body simulations. We are able to include smaller scales than in previous analyses, resulting in tighter cosmological constraints. Using three overlapping redshift bins, we measure the angular diameter distance, the Hubble parameter, and the cosmic growth rate, and explore the cosmological implications of our full shape clustering measurements in combination with CMB and SN Ia data. Assuming a {\Lambda}CDM cosmology, we constrain the matter density to {\Omega}_m = 0.311 -0.010 +0.009 and the Hubble parameter to H_0 = 67.6 -0.6 +0.7 km s^-1 Mpc^-1, at a confidence level (CL) of 68 per cent. We also allow for non-standard dark energy models and modifications of the growth rate, finding good agreement with the {\Lambda}CDM paradigm. For example, we constrain the equation-of-state parameter to w = -1.019 -0.039 +0.048. This paper is part of a set that analyses the final galaxy clustering dataset from BOSS. The measurements and likelihoods presented here are combined with others in Alam et al. 2016 to produce the final cosmological constraints from BOSS.

Fiber-fed multi-object spectroscopic surveys, with their ability to collect an unprecedented number of redshifts, currently dominate large-scale structure studies. However, physical constraints limit these surveys from successfully collecting redshifts from galaxies too close to each other on the focal plane. This ultimately leads to significant systematic effects on galaxy clustering measurements. Using simulated mock catalogs, we demonstrate that fiber collisions have a significant impact on the power spectrum, $P(k)$, monopole and quadrupole that exceeds sample variance at scales smaller than $k\sim0.1~h/Mpc$. We present two methods to account for fiber collisions in the power spectrum. The first, statistically reconstructs the clustering of fiber collided galaxy pairs by modeling the distribution of the line-of-sight displacements between them. It also properly accounts for fiber collisions in the shot-noise correction term of the $P(k)$ estimator. Using this method, we recover the true $P(k)$ monopole of the mock catalogs with residuals of $<0.5\%$ at $k=0.3~h/Mpc$ and $<4\%$ at $k=0.83~h/Mpc$ -- a significant improvement over existing correction methods. The quadrupole, however, does not improve significantly. The second method models the effect of fiber collisions on the power spectrum as a convolution with a configuration space top-hat function that depends on the physical scale of fiber collisions. It directly computes theoretical predictions of the fiber-collided $P(k)$ multipoles and reduces the influence of smaller scales to a set of nuisance parameters. Using this method, we reliably model the effect of fiber collisions on the monopole and quadrupole down to the scale limits of theoretical predictions. The methods we present in this paper will allow us to robustly analyze galaxy power spectrum multipole measurements to much smaller scales than previously possible.

We explore the cosmological implications of anisotropic clustering measurements in configuration space of the final galaxy samples from Data Release 12 of the SDSS-III Baryon Oscillation Spectroscopic Survey. We implement a new detailed modelling of the effects of non-linearities, galaxy bias and redshift-space distortions that can be used to extract unbiased cosmological information from our measurements for scales $s \gtrsim 20\,h^{-1}{\rm Mpc}$. We combined the galaxy clustering information from BOSS with the latest cosmic microwave background (CMB) observations and Type Ia supernovae samples and found no significant evidence for a deviation from the $\Lambda$CDM cosmological model. In particular, these data sets can constrain the dark energy equation of state parameter to $w_{\rm DE}=-0.996\pm0.042$ when assumed time-independent, the curvature of the Universe to $\Omega_{k}=-0.0007\pm 0.0030$ and the sum of the neutrino masses to $\sum m_{\nu} < 0.25\,{\rm eV}$ at 95 per cent CL. We explore the constraints on the growth rate of cosmic structures assuming $f(z)=\Omega_{\rm m}(z)^\gamma$ and obtain $\gamma = 0.609\pm 0.079$, in good agreement with the predictions of general relativity of $\gamma=0.55$. We compress the information of our clustering measurements into constraints on the parameter combinations $D_{\rm V}(z)/r_{\rm d}$, $F_{\rm AP}(z)$ and $f\sigma_8(z)$ at the effective redshifts of $z=0.38$, $0.51$ and $0.61$ with their respective covariance matrices and find good agreement with the predictions for these parameters obtained from the best-fitting $\Lambda$CDM model to the CMB data from the Planck satellite. This paper is part of a set that analyses the final galaxy clustering dataset from BOSS. The measurements and likelihoods presented here are combined with others in Alam et al. (2016) to produce the final cosmological constraints from BOSS.

We present a study of the clustering and halo occupation distribution of BOSS CMASS galaxies in the redshift range 0.43 < z < 0.7 drawn from the Final SDSS-III Data Release. We compare the BOSS results with the predictions of a Halo Abundance Matching (HAM) clustering model that assigns galaxies to dark matter halos selected from the large BigMultiDark $N$-body simulation of a flat $\Lambda$CDM Planck cosmology. We compare the observational data with the simulated ones on a light-cone constructed from 20 subsequent outputs of the simulation. Observational effects such as incompleteness, geometry, veto masks and fiber collisions are included in the model, which reproduces within 1-$\sigma$ errors the observed monopole of the 2-point correlation function at all relevant scales: from the smallest scales, 0.5 $h^{-1}$ Mpc, up to scales beyond the Baryonic Acoustic Oscillation feature. This model also agrees remarkably well with the BOSS galaxy power spectrum (up to $k\sim1$ $h$ Mpc$^{-1}$), and the Three-point correlation function. The quadrupole of the correlation function presents some tensions with observations. We discuss possible causes that can explain this disagreement, including target selection effects. Overall, the standard HAM model describes remarkably well the clustering statistics of the CMASS sample. We compare the stellar to halo mass relation for the CMASS sample measured using weak lensing in the CFHT Stripe 82 Survey with the prediction of our clustering model, and find a good agreement within 1-$\sigma$. The BigMD-BOSS light-cone including properties of BOSS galaxies and halo properties is made publicly available.

We apply the Alcock-Paczynski (AP) test to the stacked voids identified using the large-scale structure galaxy catalog from the Baryon Oscillation Spectroscopic Survey (BOSS). This galaxy catalog is part of the Sloan Digital Sky Survey (SDSS) Data Release 12 and is the final catalog of SDSS-III. We also use 1000 mock galaxy catalogs that match the geometry, density, and clustering properties of the BOSS sample in order to characterize the statistical uncertainties of our measurements and take into account systematic errors such as redshift space distortions. For both BOSS data and mock catalogs, we use the ZOBOV algorithm to identify voids, we stack together all voids with effective radii of 30-100Mpc/h in the redshift range 0.43-0.7, and we accurately measure the shape of the stacked voids. Our tests with the mock catalogs show that we measure the stacked void ellipticity with a statistical precision of 2.6%. We find that the stacked voids in redshift space are slightly squashed along the line of sight, which is consistent with previous studies. We repeat this measurement of stacked void shape in the BOSS data assuming several values of Omega_m within the flat LCDM model, and we compare to the mock catalogs in redshift space in order to perform the AP test. We obtain a constraint of $\Omega_m = 0.38^{+0.18}_{-0.15}$ at the 68% confidence level from the AP test. We discuss the various sources of statistical and systematic noise that affect the constraining power of this method. In particular, we find that the measured ellipticity of stacked voids scales more weakly with cosmology than the standard AP prediction, leading to significantly weaker constraints. We discuss how AP constraints will improve in future surveys with larger volumes and densities.

Future galaxy surveys require one percent precision in the theoretical knowledge of the power spectrum over a large range including very nonlinear scales. While this level of accuracy is easily obtained in the linear regime with perturbation theory, it represents a serious challenge for small scales where numerical simulations are required. In this paper we quantify the precision of present-day $N$-body methods, identifying main potential error sources from the set-up of initial conditions to the measurement of the final power spectrum. We directly compare three widely used $N$-body codes, Ramses, Pkdgrav3, and Gadget3 which represent three main discretisation techniques: the particle-mesh method, the tree method, and a hybrid combination of the two. For standard run parameters, the codes agree to within one percent at $k\leq1$ $h\,\rm Mpc^{-1}$ and to within three percent at $k\leq10$ $h\,\rm Mpc^{-1}$. We also consider the bispectrum and show that the reduced bispectra agree at the sub-percent level for $k\leq 2$ $h\,\rm Mpc^{-1}$. In a second step, we quantify potential errors due to initial conditions, box size, and resolution using an extended suite of simulations performed with our fastest code Pkdgrav3. We demonstrate that the simulation box size should not be smaller than $L=0.5$ $h^{-1}\rm Gpc$ to avoid systematic finite-volume effects (while much larger boxes are required to beat down the statistical sample variance). Furthermore, a maximum particle mass of $M_{\rm p}=10^{9}$ $h^{-1}\rm M_{\odot}$ is required to conservatively obtain one percent precision of the matter power spectrum. As a consequence, numerical simulations covering large survey volumes of upcoming missions such as DES, LSST, and Euclid will need more than a trillion particles to reproduce clustering properties at the targeted accuracy.

We present a cosmic void catalog using the large-scale structure galaxy catalog from the Baryon Oscillation Spectroscopic Survey (BOSS). This galaxy catalog is part of the Sloan Digital Sky Survey (SDSS) Data Release 12 and is the final catalog of SDSS-III. We take into account the survey boundaries, masks, and angular and radial selection functions, and apply the ZOBOV void finding algorithm to the galaxy catalog. We identify a total of 10,643 voids. After making quality cuts to ensure that the voids represent real underdense regions, we obtain 1,228 voids with effective radii spanning the range 20-100Mpc/h and with central densities that are, on average, 30% of the mean sample density. We release versions of the catalogs both with and without quality cuts. We discuss the basic statistics of voids, such as their size and redshift distributions, and measure the radial density profile of the voids via a stacking technique. In addition, we construct mock void catalogs from 1000 mock galaxy catalogs, and find that the properties of BOSS voids are in good agreement with those in the mock catalogs. We compare the stellar mass distribution of galaxies living inside and outside of the voids, and find no significant difference. These BOSS and mock void catalogs are useful for a number of cosmological and galaxy environment studies.

We report a measurement of the large-scale 3-point correlation function of galaxies using the largest dataset for this purpose to date, 777, 202 Luminous Red Galaxies in the Sloan Digital Sky Survey Baryon Acoustic Oscillation Spectroscopic Survey (SDSS BOSS) DR12 CMASS sample. This work exploits the novel algorithm of Slepian & Eisenstein (2015b) to compute the multipole moments of the 3PCF in $\mathcal{O}(N^2)$ time, with $N$ the number of galaxies. Leading-order perturbation theory models the data well in a compressed basis where one triangle side is integrated out. We also present an accurate and computationally efficient means of estimating the covariance matrix. With these techniques the redshift-space linear and non-linear bias are measured, with 2.6% precision on the former if $\sigma_8$ is fixed. The data also indicates a $2.8\sigma$ preference for the BAO, confirming the presence of BAO in the 3-point function.

Redshift-space distortions in galaxy surveys happen along the radial direction, breaking statistical translation invariance. We construct estimators for radial distortions that, using only Fast Fourier Transforms (FFTs) of the overdensity field multipoles for a given survey geometry, compute the power spectrum monopole, quadrupole and hexadecapole, and generalize such estimators to the bispectrum. Using realistic mock catalogs we compare the signal to noise of two estimators for the power spectrum hexadecapole that require different number of FFTs and measure the bispectrum monopole, quadrupole and hexadecapole. The resulting algorithm is very efficient, e.g. for the BOSS survey requires about three minutes for $\ell=0,2,4$ power spectra for scales up to $k=0.3~h/$Mpc and about fifteen additional minutes for $\ell=0,2,4$ bispectra for all scales and triangle shapes up to $k=0.2~h/$Mpc on a single core. The speed of these estimators is essential as it makes possible to compute covariance matrices from large number of realizations of mock catalogs with realistic survey characteristics, and paves the way for improved constrains of gravity on cosmological scales, inflation and galaxy bias.

We measure the angular clustering of galaxies from the Sloan Digital Sky Survey Data Release 7 in order to probe the spatial distribution of satellite galaxies within their dark matter halos. Specifically, we measure the angular correlation function on very small scales (7-320") in a range of luminosity threshold samples (absolute r-band magnitudes of -18 up to -21) that are constructed from the subset of SDSS that has been spectroscopically observed more than once (the so-called plate overlap region). We choose to measure angular clustering in this reduced survey footprint in order to minimize the effects of fiber collision incompleteness, which are otherwise substantial on these small scales. We model our clustering measurements using a fully numerical halo model that populates dark matter halos in N-body simulations to create realistic mock galaxy catalogs. The model has free parameters that specify both the number and spatial distribution of galaxies within their host halos. We adopt a flexible density profile for the spatial distribution of satellite galaxies that is similar to the dark matter Navarro-Frenk-White (NFW) profile, except that the inner slope is allowed to vary. We find that the angular clustering of our most luminous samples (Mr< -20 and -21) suggests that luminous satellite galaxies have substantially steeper inner density profiles than NFW. Lower luminosity samples are less constraining, however, and are consistent with satellite galaxies having shallow density profiles. Our results confirm the findings of Watson et al. 2012 while using different clustering measurements and modeling methodology.

We use cosmological N-body simulations to investigate whether measurements of the moments of large-scale structure can yield constraints on primordial non-Gaussianity. We measure the variance, skewness, and kurtosis of the evolved density field from simulations with Gaussian and three different non-Gaussian initial conditions: a local model with f_NL=100, an equilateral model with f_NL=-400, and an orthogonal model with f_NL=-400. We show that the moments of the dark matter density field differ significantly between Gaussian and non-Gaussian models. We also make the measurements on mock galaxy catalogs that contain galaxies with clustering properties similar to those of luminous red galaxies (LRGs). We find that, in the case of skewness and kurtosis, galaxy bias reduces the detectability of non-Gaussianity, though we can still clearly discriminate between different models in our simulation volume. However, in the case of the variance, galaxy bias greatly amplifies the detectability of non-Gaussianity. In all cases we find that redshift distortions do not significantly affect the detectability. When we restrict our measurements to volumes equivalent to the Sloan Digital Sky Survey II (SDSS-II) or Baryon Oscillation Spectroscopic Survey (BOSS) samples, the probability of detecting a departure from the Gaussian model is high by using measurements of the variance, but very low by using only skewness and kurtosis measurements. We find that skewness and kurtosis measurements are never likely to yield useful constraints on primordial non-Gaussianity, but future surveys should be large enough to place meaningful constraints using measurements of the galaxy variance.(Abridged)

The discovery of cosmic acceleration has stimulated theorists to consider dark energy or modifications to Einstein's General Relativity as possible explanations. The last decade has seen advances in theories that go beyond smooth dark energy -- modified gravity and interactions of dark energy. While the theoretical terrain is being actively explored, the generic presence of fifth forces and dark sector couplings suggests a set of distinct observational signatures. This report focuses on observations that differ from the conventional probes that map the expansion history or large-scale structure. Examples of such novel probes are: detection of scalar fields via lab experiments, tests of modified gravity using stars and galaxies in the nearby universe, comparison of lensing and dynamical masses of galaxies and clusters, and the measurements of fundamental constants at high redshift. The observational expertise involved is very broad as it spans laboratory experiments, high resolution astronomical imaging and spectroscopy and radio observations. In the coming decade, searches for these effects have the potential for discovering fundamental new physics. We discuss how the searches can be carried out using experiments that are already under way or with modest adaptations of existing telescopes or planned experiments. The accompanying paper on the Growth of Cosmic Structure describes complementary tests of gravity with observations of large-scale structure.

Halos are biased tracers of the dark matter distribution. It is often assumed that the patches from which halos formed are locally biased with respect to the initial fluctuation field, meaning that the halo-patch fluctuation field can be written as a Taylor series in that of the dark matter. If quantities other than the local density influence halo formation, then this Lagrangian bias will generically be nonlocal; the Taylor series must be performed with respect to these other variables as well. We illustrate the effect with Monte-Carlo simulations of a model in which halo formation depends on the local shear (the quadrupole of perturbation theory), and provide an analytic model which provides a good description of our results. Our model, which extends the excursion set approach to walks in more than one dimension, works both when steps in the walk are uncorrelated, as well as when there are correlations between steps. For walks with correlated steps, our model includes two distinct types of nonlocality: one is due to the fact that the initial density profile around a patch which is destined to form a halo must fall sufficiently steeply around it -- this introduces k-dependence to even the linear bias factor, but otherwise only affects the monopole of the clustering signal. The other is due to the surrounding shear field; this affects the quadratic and higher order bias factors, and introduces an angular dependence to the clustering signal. In both cases, our analysis shows that these nonlocal Lagrangian bias terms can be significant, particularly for massive halos; they must be accounted for in analyses of higher order clustering such as the halo bispectrum in Lagrangian or Eulerian space. Although we illustrate these effects using halos, our analysis and conclusions also apply to the other constituents of the cosmic web -- filaments, sheets and voids.

We develop a new test of local bias, by constructing a locally biased halo density field from sampling the dark matter-halo distribution. Our test differs from conventional tests in that it preserves the full scatter in the bias relation and it does not rely on perturbation theory. We put forward that bias parameters obtained using a smoothing scale R can only be applied to computing the halo power spectrum at scales k ~ 1/R. Our calculations can automatically include the running of bias parameters and give vanishingly small loop corrections at low-k. Our proposal results in much better agreement of the sampling and perturbation theory results with simulations. In particular, unlike the standard interpretation of local bias in the literature, our treatment of local bias does not generate a constant power in the low-k limit. We search for extra noise in the Poisson corrected halo power spectrum at wavenumbers below its turn-over and find no evidence of significant positive noise (as predicted by the standard interpretation) while we find evidence of negative noise coming from halo exclusion for very massive halos. Using perturbation theory and our non-perturbative sampling technique we also demonstrate that nonlocal bias effects discovered recently in simulations impact the power spectrum only at the few percent level in the weakly nonlinear regime.

We put forward and test a simple description of multi-point propagators (MP), which serve as building-blocks to calculate the nonlinear matter power spectrum. On large scales these propagators reduce to the well-known kernels in standard perturbation theory, while at smaller scales they are suppresed due to nonlinear couplings. Through extensive testing with numerical simulations we find that this decay is characterized by the same damping scale for both two and three-point propagators. In turn this transition can be well modeled with resummation results that exponentiate one-loop computations. For the first time, we measure the four components of the non-linear (two-point) propagator using dedicated simulations started from two independent random Gaussian fields for positions and velocities, verifying in detail the fundamentals of propagator resummation. We use these results to develop an implementation of the MP-expansion for the nonlinear power spectrum that only requires seconds to evaluate at BAO scales. To test it we construct six suites of large numerical simulations with different cosmologies. From these and LasDamas runs we show that the nonlinear power spectrum can be described at the ~ 2% level at BAO scales for redshifts in the range [0-2.5]. We make a public release of the MPTbreeze code with the hope that it can be useful to the community.

We present a new scheme for the general computation of cosmic propagators that allow to interpolate between standard perturbative results at low-k and their expected large-k resummed behavior. This scheme is applicable to any multi-point propagator and allows the matching of perturbative low-k calculations to any number of loops to their large-k behavior, and can potentially be applied in case of non-standard cosmological scenarios such as those with non-Gaussian initial conditions. The validity of our proposal is checked against previous prescriptions and measurements in numerical simulations showing a remarkably good agreement. Such a generic prescription for multi-point propagators provides the necessary building blocks for the computation of polyspectra in the context of the so-called Gamma-expansion introduced by Bernardeau et al. (2008). As a concrete application we present a consistent calculation of the matter bispectrum at one-loop order.

We present a fast method of producing mock galaxy catalogues that can be used to compute covariance matrices of large-scale clustering measurements and test the methods of analysis. Our method populates a 2nd-order Lagrangian Perturbation Theory (2LPT) matter field, where we calibrate masses of dark matter halos by detailed comparisons with N-body simulations. We demonstrate the clustering of halos is recovered at ~10 per cent accuracy. We populate halos with mock galaxies using a Halo Occupation Distribution (HOD) prescription, which has been calibrated to reproduce the clustering measurements on scales between 30 and 80 Mpc/h. We compare the sample covariance matrix from our mocks with analytic estimates, and discuss differences. We have used this method to make catalogues corresponding to Data Release 9 of the Baryon Oscillation Spectroscopic Survey (BOSS),producing 600 mock catalogues of the "CMASS" galaxy sample. These mocks enabled detailed tests of methods and errors that formed an integral part of companion analyses of these galaxy data.

We study the scale-dependence of halo bias in generic (non-local) primordial non-Gaussian (PNG) initial conditions of the type motivated by inflation, parametrized by an arbitrary quadratic kernel. We first show how to generate non-local PNG initial conditions with minimal overhead compared to local PNG models for a general class of primordial bispectra that can be written as linear combinations of separable templates. We run cosmological simulations for the local, and non-local equilateral and orthogonal models and present results on the scale-dependence of halo bias. We also derive a general formula for the Fourier-space bias using the peak-background split (PBS) in the context of the excursion set approach to halos and discuss the difference and similarities with the known corresponding result from local bias models. Our PBS bias formula generalizes previous results in the literature to include non-Markovian effects and non-universality of the mass function and are in better agreement with measurements in numerical simulations than previous results for a variety of halo masses, redshifts and halo definitions. We also derive for the first time quadratic bias results for arbitrary non-local PNG, and show that non-linear bias loops give small corrections at large-scales. The resulting well-behaved perturbation theory paves the way to constrain non-local PNG from measurements of the power spectrum and bispectrum in galaxy redshift surveys.

The relationship between galaxy and matter overdensities, bias, is most often assumed to be local. This is however unstable under time evolution, we provide proofs under several sets of assumptions. In the simplest model galaxies are created locally and linearly biased at a single time, and subsequently move with the matter (no velocity bias) conserving their comoving number density (no merging). We show that, after this formation time, the bias becomes unavoidably non-local and non-linear at large scales. We identify the non-local gravitationally induced fields in which the galaxy overdensity can be expanded, showing that they can be constructed out of the invariants of the deformation tensor (Galileons). In addition, we show that this result persists if we include an arbitrary evolution of the comoving number density of tracers. We then include velocity bias, and show that new contributions appear, a dipole field being the signature at second order. We test these predictions by studying the dependence of halo overdensities in cells of fixed matter density: measurements in simulations show that departures from the mean bias relation are strongly correlated with the non-local gravitationally induced fields identified by our formalism. The effects on non-local bias seen in the simulations are most important for the most biased halos, as expected from our predictions. The non-locality seen in the simulations is not fully captured by assuming local bias in Lagrangian space. Accounting for these effects when modeling galaxy bias is essential for correctly describing the dependence on triangle shape of the galaxy bispectrum, and hence constraining cosmological parameters and primordial non-Gaussianity. We show that using our formalism we remove an important systematic in the determination of bias parameters from the galaxy bispectrum, particularly for luminous galaxies. (abridged)

We constrain the linear and quadratic bias parameters from the configuration dependence of the three-point correlation function (3PCF) in both redshift and projected space, utilizing measurements of spectroscopic galaxies in the Sloan Digital Sky Survey (SDSS) Main Galaxy Sample. We show that bright galaxies (M_r < -21.5) are biased tracers of mass, measured at a significance of 4.5 sigma in redshift space and 2.5 sigma in projected space by using a thorough error analysis in the quasi-linear regime (9-27 Mpc/h). Measurements on a fainter galaxy sample are consistent with an unbiased model. We demonstrate that a linear bias model appears sufficient to explain the galaxy-mass bias of our samples, although a model using both linear and quadratic terms results in a better fit. In contrast, the bias values obtained from the linear model appear in better agreement with the data by inspection of the relative bias, and yield implied values of sigma_8 that are more consistent with current constraints. We investigate the covariance of the 3PCF, which itself is a measurement of galaxy clustering. We assess the accuracy of our error estimates by comparing results from mock galaxy catalogs to jackknife re-sampling methods. We identify significant differences in the structure of the covariance. However, the impact of these discrepancies appears to be mitigated by an eigenmode analysis that can account for the noisy, unresolved modes. Our results demonstrate that using this technique is sufficient to remove potential systematics even when using less-than-ideal methods to estimate errors.

The three-point correlation function (3PCF) provides an important view into the clustering of galaxies that is not available to its lower order cousin, the two-point correlation function (2PCF). Higher order statistics, such as the 3PCF, are necessary to probe the non-Gaussian structure and shape information expected in these distributions. We measure the clustering of spectroscopic galaxies in the Main Galaxy Sample of the Sloan Digital Sky Survey (SDSS), focusing on the shape or configuration dependence of the reduced 3PCF in both redshift and projected space. This work constitutes the largest number of galaxies ever used to investigate the reduced 3PCF, using over 220,000 galaxies in three volume-limited samples. We find significant configuration dependence of the reduced 3PCF at 3-27 Mpc/h, in agreement with LCDM predictions and in disagreement with the hierarchical ansatz. Below 6 Mpc/h, the redshift space reduced 3PCF shows a smaller amplitude and weak configuration dependence in comparison with projected measurements suggesting that redshift distortions, and not galaxy bias, can make the reduced 3PCF appear consistent with the hierarchical ansatz. The reduced 3PCF shows a weaker dependence on luminosity than the 2PCF, with no significant dependence on scales above 9 Mpc/h. On scales less than 9 Mpc/h, the reduced 3PCF appears more affected by galaxy color than luminosty. We demonstrate the extreme sensitivity of the 3PCF to systematic effects such as sky completeness and binning scheme, along with the difficulty of resolving the errors. Some comparable analyses make assumptions that do not consistently account for these effects.

Models of galaxy and halo clustering commonly assume that the tracers can be treated as a continuous field locally biased with respect to the underlying mass distribution. In the peak model pioneered by BBKS, one considers instead density maxima of the initial, Gaussian mass density field as an approximation to the formation site of virialized objects. In this paper, the peak model is extended in two ways to improve its predictive accuracy. Firstly, we derive the two-point correlation function of initial density peaks up to second order and demonstrate that a peak-background split approach can be applied to obtain the k-independent and k-dependent peak bias factors at all orders. Secondly, we explore the gravitational evolution of the peak correlation function within the Zel'dovich approximation. We show that the local (Lagrangian) bias approach emerges as a special case of the peak model, in which all bias parameters are scale-independent and there is no statistical velocity bias. We apply our formulae to study how the Lagrangian peak biasing, the diffusion due to large scale flows and the mode-coupling due to nonlocal interactions affect the scale dependence of bias from small separations up to the baryon acoustic oscillation (BAO) scale. For 2-sigma density peaks collapsing at z=0.3, our model predicts a ~ 5% residual scale-dependent bias around the acoustic scale that arises mostly from first-order Lagrangian peak biasing (as opposed to second-order gravity mode-coupling). We also search for a scale dependence of bias in the large scale auto-correlation of massive halos extracted from a very large N-body simulation provided by the MICE collaboration. For halos with mass M>10^{14}Msun/h, our measurements demonstrate a scale-dependent bias across the BAO feature which is very well reproduced by a prediction based on the peak model.

We analyze the line-of-sight baryonic acoustic feature in the two-point correlation function {\xi} of the Sloan Digital Sky Survey (SDSS) luminous red galaxy (LRG) sample (0.16 < z < 0.47). By defining a narrow line-of-sight region, rp < 5.5 Mpc/h, where rp is the transverse separation component, we measure a strong excess of clustering at ~ 110 Mpc/h, as previously reported in the literature. We also test these results in an alternative coordinate system, by defining the line-of-sight as {\theta} < 3{\deg}, where {\theta} is the opening angle. This clustering excess appears much stronger than the feature in the better-measured monopole. A fiducial {\Lambda}CDM non-linear model in redshift-space predicts a much weaker signature. We use realistic mock catalogs to model the expected signal and noise. We find that the line-of-sight measurements can be explained well by our mocks as well as by a featureless {\xi} = 0. We conclude that there is no convincing evidence that the strong clustering measurement is the line-of-sight baryonic acoustic feature. We also evaluate how detectable such a signal would be in the upcoming Baryon Oscillation Spectroscopic Survey LRG volume (BOSS). Mock LRG catalogs (z < 0.6) suggest that: (i) the narrow line- of-sight cylinder and cone defined above probably will not reveal a detectable acoustic feature in BOSS; (ii) a clustering measurement as high as that in the current sample can be ruled out (or confirmed) at a high confidence level using a BOSS-sized data set; and (iii) an analysis with wider angular cuts, which provide better signal-to-noise ratios, can nevertheless be used to compare line-of-sight and transverse distances, and thereby constrain the expansion rate H(z) and diameter distance DA(z).

The peak-background split argument is commonly used to relate the abundance of dark matter halos to their spatial clustering. Testing this argument requires an accurate determination of the halo mass function. We present a Maximum Likelihood method for fitting parametric functional forms to halo abundances which differs from previous work because it does not require binned counts. Our conclusions do not depend on whether we use our method or more conventional ones. In addition, halo abundances depend on how halos are defined. Our conclusions do not depend on the choice of link length associated with the friends-of-friends halo-finder, nor do they change if we identify halos using a spherical overdensity algorithm instead. The large scale halo bias measured from the matter-halo cross spectrum b_x and the halo autocorrelation function b_xi (on scales k~0.03h/Mpc and r ~50 Mpc/h) can differ by as much as 5% for halos that are significantly more massive than the characteristic mass M*. At these large masses, the peak background split estimate of the linear bias factor b1 is 3-5% smaller than b_xi, which is 5% smaller than b_x. We discuss the origin of these discrepancies: deterministic nonlinear local bias, with parameters determined by the peak-background split argument, is unable to account for the discrepancies we see. A simple linear but nonlocal bias model, motivated by peaks theory, may also be difficult to reconcile with our measurements. More work on such nonlocal bias models may be needed to understand the nature of halo bias at this level of precision.

We use N-body simulations to study the nonlinear structure formation in brane-induced gravity, developing a new method that requires alternate use of Fast Fourier Transforms and relaxation. This enables us to compute the nonlinear matter power spectrum and bispectrum, the halo mass function, and the halo bias. From the simulation results, we confirm the expectations based on analytic arguments that the Vainshtein mechanism does operate as anticipated, with the density power spectrum approaching that of standard gravity within a modified background evolution in the nonlinear regime. The transition is very broad and there is no well defined Vainshtein scale, but roughly this corresponds to k_*~ 2 at redshift z=1 and k_*~ 1 at z=0. We checked that while extrinsic curvature fluctuations go nonlinear, and the dynamics of the brane-bending mode C receives important nonlinear corrections, this mode does get suppressed compared to density perturbations, effectively decoupling from the standard gravity sector. At the same time, there is no violation of the weak field limit for metric perturbations associated with C. We find good agreement between our measurements and the predictions for the nonlinear power spectrum presented in paper I, that rely on a renormalization of the linear spectrum due to nonlinearities in the modified gravity sector. A similar prediction for the mass function shows the right trends. Our simulations also confirm the induced change in the bispectrum configuration dependence predicted in paper I.

We examine the correlation function \xi of the Sloan Digital Sky Survey (SDSS) Luminous Red Galaxy sample (LRG) at large scales (60<s<400 Mpc/h) using the final data release (DR7; 105,831 LRGs between 0.16<z<0.47). Using mock catalogs, we demonstrate that the observed baryonic acoustic peak and larger scale signal are consistent with LCDM at the 1.5\sigma level. The signal at 155<s<200 Mpc/h tends to be high relative to theoretical expectations; this slight deviation can be attributed to a bright subsample of the LRGs. Fitting data to a non-linear, redshift-space, template based-model, we constrain the peak position at s_p=103.6+3.6-2.4 Mpc/h when fitting the range 60<s<150 Mpc/h (1\sigma uncertainties measured from the mocks. This redshift-space distance s_p is related to the comoving sound horizon scale r_s after taking into account matter clustering non-linearities, redshift distortions and galaxy clustering bias. Mock catalogs show that the probability that a DR7-sized sample would not have an identifiable peak is at least 10%. As a consistency check of a fiducial cosmology, we use the observed s_p to obtain the distance D_V=[(1+z)^2D_A^2cz/H(z)]^(1/3) relative to the acoustic scale. We find r_s/D_V(z=0.278)=0.1394+-0.0049. This result is in excellent agreement with Percival et. al (2009), who examine roughly the same data set, but using the power spectrum. Comparison with other determinations in the literature are also in very good agreement. We have tested our results against a battery of possible systematic effects, finding all effects are smaller than our estimated sample variance.

We study the generation of vorticity and velocity dispersion by orbit crossing using cosmological numerical simulations, and calculate the backreaction of these effects on the evolution of large-scale density and velocity divergence power spectra. We use Delaunay tessellations to define the velocity field, showing that the power spectra of velocity divergence and vorticity measured in this way are unbiased and have better noise properties than for standard interpolation methods that deal with mass weighted velocities. We show that high resolution simulations are required to recover the correct large-scale vorticity power spectrum, while poor resolution can spuriously amplify its amplitude by more than one order of magnitude. We measure the scalar and vector modes of the stress tensor induced by orbit crossing using an adaptive technique, showing that its vector modes lead, when input into the vorticity evolution equation, to the same vorticity power spectrum obtained from the Delaunay method. We incorporate orbit crossing corrections to the evolution of large scale density and velocity fields in perturbation theory by using the measured stress tensor modes. We find that at large scales (k~0.1 h/Mpc) vector modes have very little effect in the density power spectrum, while scalar modes (velocity dispersion) can induce percent level corrections at z=0, particularly in the velocity divergence power spectrum. In addition, we show that the velocity power spectrum is smaller than predicted by linear theory until well into the nonlinear regime, with little contribution from virial velocities.

We study the growth of subhorizon perturbations in brane-induced gravity using perturbation theory. We solve for the linear evolution of perturbations taking advantage of the symmetry under gauge transformations along the extra-dimension to decouple the bulk equations in the quasistatic approximation, which we argue may be a better approximation at large scales than thought before. We then study the nonlinearities in the bulk and brane equations, concentrating on the workings of the Vainshtein mechanism by which the theory becomes general relativity (GR) at small scales. We show that at the level of the power spectrum, to a good approximation, the effect of nonlinearities in the modified gravity sector may be absorbed into a renormalization of the gravitational constant. Since the relation between the lensing potential and density perturbations is entirely unaffected by the extra physics in these theories, the modified gravity can be described in this approximation by a single function, an effective gravitational constant for nonrelativistic motion that depends on space and time. We develop a resummation scheme to calculate it, and provide predictions for the nonlinear power spectrum. At the level of the large-scale bispectrum, the leading order corrections are obtained by standard perturbation theory techniques, and show that the suppression of the brane-bending mode leads to characteristic signatures in the non-Gaussianity generated by gravity, generic to models that become GR at small scales through second-derivative interactions. We compare the predictions in this work to numerical simulations in a companion paper.

A new and powerful probe of the origin and evolution of structures in the Universe has emerged and been actively developed over the last decade. In the coming decade, non-Gaussianity, i.e., the study of non-Gaussian contributions to the correlations of cosmological fluctuations, will become an important probe of both the early and the late Universe. Specifically, it will play a leading role in furthering our understanding of two fundamental aspects of cosmology and astrophysics: (i) the physics of the very early universe that created the primordial seeds for large-scale structures, and (ii) the subsequent growth of structures via gravitational instability and gas physics at later times. To date, observations of fluctuations in the Cosmic Microwave Background (CMB) and the Large-Scale Structure of the Universe (LSS) have focused largely on the Gaussian contribution as measured by the two-point correlations (or the power spectrum) of density fluctuations. However, an even greater amount of information is contained in non-Gaussianity and a large discovery space therefore still remains to be explored. Many observational probes can be used to measure non-Gaussianity, including CMB, LSS, gravitational lensing, Lyman-alpha forest, 21-cm fluctuations, and the abundance of rare objects such as clusters of galaxies and high-redshift galaxies. Not only does the study of non-Gaussianity maximize the science return from a plethora of present and future cosmological experiments and observations, but it also carries great potential for important discoveries in the coming decade.

We introduce the concept of multi-point propagators between linear cosmic fields and their nonlinear counterparts in the context of cosmological perturbation theory. Such functions express how a non-linearly evolved Fourier mode depends on the full ensemble of modes in the initial density field. We identify and resum the dominant diagrams in the large-$k$ limit, showing explicitly that multi-point propagators decay into the nonlinear regime at the same rate as the two-point propagator. These analytic results generalize the large-$k$ limit behavior of the two-point propagator to arbitrary order. We measure the three-point propagator as a function of triangle shape in numerical simulations and confirm the results of our high-$k$ resummation. We show that any $n-$point spectrum can be reconstructed from multi-point propagators, which leads to a physical connection between nonlinear corrections to the power spectrum at small scales and higher-order correlations at large scales. As a first application of these results, we calculate the reduced bispectrum at one-loop in renormalized perturbation theory and show that we can predict the decrease in its dependence on triangle shape at redshift zero, when standard perturbation theory is least successful.

We develop an analytic theory for the redshift space bispectrum of dark matter, haloes and galaxies. This is done within the context of the halo model of structure formation, as this allows for the self-consistent inclusion of linear and non-linear redshift space distortions and also for the non-linearity of the halo bias. The model is applicable over a wide range of scales: on the largest scales the predictions reduce to those of the standard perturbation theory (PT); on smaller scales they are determined primarily by the nonlinear virial velocities of galaxies within haloes, and this gives rise to the U-shaped anisotropy in the reduced bispectrum -- a finger print of the Finger-Of-God distortions. We then confront the predictions with bispectrum measurements from an ensemble of numerical simulations. On very large scales, k=0.05 h/Mpc, we find reasonably good agreement between our Halo Model, PT and the data, to within the errors. On smaller scales, k=0.1 h/Mpc, the measured bispectra differ from the PT at the level of 10-20%, especially for colinear triangle configurations. The Halo Model predictions improve over PT, but are accurate to no better than 10%. On smaller scales k=0.5-1.0 h/Mpc, our model provides a significant improvement over PT, which breaks down. This implies that studies which use the lowest order PT to extract galaxy bias information are not robust on scales k>0.1 h/Mpc. The analytic and simulation results also indicate that there is no observable scale for which the configuration dependence of the reduced bispectrum is constant--hierarchical models for the higher order correlation functions in redshift space are unlikely to be useful. It is hoped that our model will facilitate extraction of information from large-scale structure surveys of the Universe.

The baryonic acoustic signature in the large-scale clustering pattern of galaxies has been detected in the two-point correlation function. Its precise spatial scale has been forwarded as a rigid-rod ruler test for the space-time geometry, and hence as a probe for tracking the evolution of Dark Energy. Percent-level shifts in the measured position can bias such a test and erode its power to constrain cosmology. This paper addresses some of the systematic effects that might induce shifts: namely non-linear corrections from matter evolution, redshift space distortions and biasing. We tackle these questions through analytic methods and through a large battery of numerical simulations, with total volume of the order ~100[Gpc\h]^3. A toy-model calculation shows that if the non-linear corrections simply smooth the acoustic peak, then this gives rise to an `apparent' shifting to smaller scales. However if tilts in the broad band power spectrum are induced then this gives rise to more pernicious `physical' shifts. Our numerical simulations show evidence of both: in real space and at z=0, for the dark matter we find percent level shifts; for haloes the shifts depend on halo mass, with larger shifts being found for the most biased samples, up to 3%. From our analysis we find that physical shifts are greater than ~0.4% at z=0. In redshift space these effects are exacerbated, but at higher redshifts are alleviated. We develop an analytical model to understand this, based on solutions to the pair conservation equation using characteristic curves. When combined with modeling of pairwise velocities the model reproduces the main trends found in the data. The model may also help to unbias the acoustic peak.

We study the nonlinear evolution of the baryon acoustic oscillations (BAO) in the dark matter power spectrum and correlation function using renormalized perturbation theory (RPT). In a previous paper we showed that RPT successfully predicts the damping of acoustic oscillations; here we extend our calculation to the enhancement of power due to mode-coupling. We show that mode-coupling generates additional oscillations that are out of phase with those in the linear spectrum, leading to shifts in the scales of oscillation nodes defined with respect to a smooth spectrum. When Fourier transformed, these out of phase oscillations induce percent-level shifts in the acoustic peak of the two-point correlation function. We present predictions for these shifts as a function of redshift; these should be considered as a robust lower limit to the more realistic case that includes in addition redshift distortions and galaxy bias. We show that these nonlinear effects occur at very large scales, leading to a breakdown of linear theory at scales much larger than commonly thought. We discuss why virialized halo profiles are not responsible for these effects, which can be understood from basic physics of gravitational instability. Our results are in excellent agreement with numerical simulations, and can be used as a starting point for modeling BAO in future observations. To meet this end, we suggest a simple physically motivated model to correct for the shifts caused by mode-coupling.

We examine the scale dependence of dark matter, halo and galaxy clustering on very large scales (0.01<k[h/Mpc]<0.15), due to non-linear effects from dynamics and halo bias. We pursue a two line offensive: high resolution numerical simulations are used to establish several new results, and an analytic model is developed to understand their origins. Our simulations show: (i) that the z=0 dark matter power spectrum is suppressed relative to linear theory by ~5% on scales (0.05<k[h/Mpc]<0.075); (ii) that, indeed, halo bias is non-linear over the scales we probe and that the scale dependence is a strong function of halo mass. High mass haloes show no suppression of power on scales (k<0.07[h/Mpc]), and only show amplification on smaller scales, whereas low mass haloes show strong, ~5-10%, suppression over the range (0.05 <k[h/Mpc] <0.15). Our results have relevance for studies of the baryon acoustic oscillation features. Non-linear mode-mode coupling: (i) damps these features on progressively larger scales as halo mass increases; (ii) produces small shifts in the positions of the peaks and troughs which depend on halo mass. Our analytic model is described in the language of the `halo-model'. However, for the first time the halo-halo clustering term is propagated into the non-linear regime using `1-loop' perturbation theory and a non-linear halo bias model. We show that, with bias parameters derived from simulations, the model predictions are in agreement with the numerical results. We then use the model to explore the scale dependence of galaxies of different colour and find significant differences between the power spectra of the two populations. Thus understanding the scale dependent bias for a given galaxy sample will be crucial for deriving accurate cosmological constraints. (Abridged)

We measure the large-scale real-space power spectrum P(k) using luminous red galaxies (LRGs) in the Sloan Digital Sky Survey (SDSS) and use this measurement to sharpen constraints on cosmological parameters from the Wilkinson Microwave Anisotropy Probe (WMAP). We employ a matrix-based power spectrum estimation method using Pseudo-Karhunen-Loeve eigenmodes, producing uncorrelated minimum-variance measurements in 20 k-bands of both the clustering power and its anisotropy due to redshift-space distortions, with narrow and well-behaved window functions in the range 0.01h/Mpc < k < 0.2h/Mpc. Results from the LRG and main galaxy samples are consistent, with the former providing higher signal-to-noise. Our results are robust to omitting angular and radial density fluctuations and are consistent between different parts of the sky. They provide a striking confirmation of the predicted large-scale LCDM power spectrum. Combining only SDSS LRG and WMAP data places robust constraints on many cosmological parameters that complement prior analyses of multiple data sets. The LRGs provide independent cross-checks on Om and the baryon fraction in good agreement with WMAP. Within the context of flat LCDM models, our LRG measurements complement WMAP by sharpening the constraints on the matter density, the neutrino density and the tensor amplitude by about a factor of two, giving Omega_m=0.24+-0.02 (1 sigma), sum m_nu < 0.9 eV (95%) and r<0.3 (95%). Baryon oscillations are clearly detected and provide a robust measurement of the comoving distance to the median survey redshift z=0.35 independent of curvature and dark energy properties. Within the LCDM framework, our power spectrum measurement improves the evidence for spatial flatness, sharpening the curvature constraint Omega_tot=1.05+-0.05 from WMAP alone to Omega_tot=1.003+-0.010. Assuming Omega_tot=1, the equation of state parameter is constrained to w=-0.94+-0.09, indicating the potential for more ambitious future LRG measurements to provide precision tests of the nature of dark energy. All these constraints are essentially independent of scales k>0.1h/Mpc and associated nonlinear complications, yet agree well with more aggressive published analyses where nonlinear modeling is crucial.

We investigate the clustering of galaxy groups and clusters in the SDSS using the Berlind et al. (2006) group sample, which is designed to identify galaxy systems that each occupy a single dark matter halo. We estimate group masses from their abundances, and measure their relative large-scale bias as a function of mass. Our measurements are in agreement with the theoretical halo bias function, given a standard LCDM cosmological model, and they tend to favor a low value of the power spectrum amplitude sigma_8. We search for a residual dependence of clustering on other group properties at fixed mass, and find the strongest signal for central galaxy color in high mass groups. Massive groups with less red central galaxies are more biased on large scales than similar mass groups with redder central galaxies. We show that this effect is unlikely to be caused by errors in our mass estimates, and is most likely observational evidence of recent theoretical findings that halo bias depends on a ``second parameter'' other than mass, such as age or concentration. To compare with the data, we study the bias of massive halos in N-body simulations and quantify the strength of the relation between halo bias and concentration at fixed mass. In addition to confirming a non-trivial prediction of the LCDM cosmological model, these results have important implications for the role that environment plays in shaping galaxy properties.

We identify galaxy groups and clusters in volume-limited samples of the SDSS redshift survey, using a redshift-space friends-of-friends algorithm. We optimize the friends-of-friends linking lengths to recover galaxy systems that occupy the same dark matter halos, using a set of mock catalogs created by populating halos of N-body simulations with galaxies. Extensive tests with these mock catalogs show that no combination of perpendicular and line-of-sight linking lengths is able to yield groups and clusters that simultaneously recover the true halo multiplicity function, projected size distribution, and velocity dispersion. We adopt a linking length combination that yields, for galaxy groups with ten or more members: a group multiplicity function that is unbiased with respect to the true halo multiplicity function; an unbiased median relation between the multiplicities of groups and their associated halos; a spurious group fraction of less than ~1%; a halo completeness of more than ~97%; the correct projected size distribution as a function of multiplicity; and a velocity dispersion distribution that is ~20% too low at all multiplicities. These results hold over a range of mock catalogs that use different input recipes of populating halos with galaxies. We apply our group-finding algorithm to the SDSS data and obtain three group and cluster catalogs for three volume-limited samples that cover 3495.1 square degrees on the sky. We correct for incompleteness caused by fiber collisions and survey edges, and obtain measurements of the group multiplicity function, with errors calculated from realistic mock catalogs. These multiplicity function measurements provide a key constraint on the relation between galaxy populations and dark matter halos.

The present spatial distribution of galaxies in the Universe is non-Gaussian, with 40% skewness in 50 Mpc/h spheres, and remarkably little is known about the information encoded in it about cosmological parameters beyond the power spectrum. In this work we present an attempt to bridge this gap by studying the bispectrum, paying particular attention to a joint analysis with the power spectrum and their combination with CMB data. We address the covariance properties of the power spectrum and bispectrum including the effects of beat coupling that lead to interesting cross-correlations, and discuss how baryon acoustic oscillations break degeneracies. We show that the bispectrum has significant information on cosmological parameters well beyond its power in constraining galaxy bias, and when combined with the power spectrum is more complementary than combining power spectra of different samples of galaxies, since non-Gaussianity provides a somewhat different direction in parameter space. In the framework of flat cosmological models we show that most of the improvement of adding bispectrum information corresponds to parameters related to the amplitude and effective spectral index of perturbations, which can be improved by almost a factor of two. Moreover, we demonstrate that the expected statistical uncertainties in sigma8 of a few percent are robust to relaxing the dark energy beyond a cosmological constant.

We study the impact of setting initial conditions in numerical simulations using the standard procedure based on the Zel'dovich approximation (ZA). As it is well known from perturbation theory, ZA initial conditions have incorrect second and higher-order growth and therefore excite long-lived transients in the evolution of the statistical properties of density and velocity fields. We also study the improvement brought by using more accurate initial conditions based on second-order Lagrangian perturbation theory (2LPT). We show that 2LPT initial conditions reduce transients significantly and thus are much more appropriate for numerical simulations devoted to precision cosmology. Using controlled numerical experiments with ZA and 2LPT initial conditions we show that simulations started at redshift z_i=49 using the ZA underestimate the power spectrum in the nonlinear regime by about 2,4,8 % at z=0,1,3 respectively, whereas the mass function of dark matter halos is underestimated by 5% at m=10^15 M_sun/h (z=0) and 10% at m=2x10^14M_sun/h (z=1). The clustering of halos is also affected to the few percent level at z=0. These systematics effects are typically larger than statistical uncertainties in recent mass function and power spectrum fitting formulae extracted from numerical simulations. At large scales, the measured transients in higher-order correlations can be understood from first principle calculations based on perturbation theory.

We show how to estimate the covariance of the power spectrum of a statistically homogeneous and isotropic density field from a single periodic simulation, by applying a set of weightings to the density field, and by measuring the scatter in power spectra between different weightings. We recommend a specific set of 52 weightings containing only combinations of fundamental modes, constructed to yield a minimum variance estimate of the covariance of power. Numerical tests reveal that at nonlinear scales the variance of power estimated by the weightings method substantially exceeds that estimated from a simple ensemble method. We argue that the discrepancy is caused by beat-coupling, in which products of closely spaced Fourier modes couple by nonlinear gravitational growth to the beat mode between them. Beat-coupling appears whenever nonlinear power is measured from Fourier modes with a finite spread of wavevector, and is therefore present in the weightings method but not the ensemble method. Beat-coupling inevitably affects real galaxy surveys, whose Fourier modes have finite width. Surprisingly, the beat-coupling contribution dominates the covariance of power at nonlinear scales, so that, counter-intuitively, it is expected that the covariance of nonlinear power in galaxy surveys is dominated not by small scale structure, but rather by beat-coupling to the largest scales of the survey.

We develop a new formalism to study nonlinear evolution in the growth of large-scale structure, by following the dynamics of gravitational clustering as it builds up in time. This approach is conveniently represented by Feynman diagrams constructed in terms of three objects: the initial conditions (e.g. perturbation spectrum), the vertex (describing non-linearities) and the propagator (describing linear evolution). We show that loop corrections to the linear power spectrum organize themselves into two classes of diagrams: one corresponding to mode-coupling effects, the other to a renormalization of the propagator. Resummation of the latter gives rise to a quantity that measures the memory of perturbations to initial conditions as a function of scale. As a result of this, we show that a well-defined (renormalized) perturbation theory follows, in the sense that each term in the remaining mode-coupling series dominates at some characteristic scale and is subdominant otherwise. This is unlike standard perturbation theory, where different loop corrections can become of the same magnitude in the nonlinear regime. In companion papers we compare the resummation of the propagator with numerical simulations, and apply these results to the calculation of the nonlinear power spectrum. Remarkably, the expressions in renormalized perturbation theory can be written in a way that closely resembles the halo model.

We study the nonlinear propagator, a key ingredient in renormalized perturbation theory (RPT) that allows a well-controlled extension of perturbation theory into the nonlinear regime. We show that it can be thought as measuring the memory of density and velocity fields to their initial conditions. This provides a clean definition of the validity of linear theory, which is shown to be much more restricted than usually recognized in the literature. We calculate the nonlinear propagator in RPT and compare to measurements in numerical simulations, showing remarkable agreement well into the nonlinear regime. We also show that N-body simulations require a rather large volume to recover the correct propagator, due to the missing large-scale modes. Our results for the nonlinear propagator provide an essential element to compute the nonlinear power spectrum in RPT.

We measure the projected correlation function w_p(r_p) from the Sloan Digital Sky Survey for a flux-limited sample of 118,000 galaxies and for a volume limited subset of 22,000 galaxies with absolute magnitude M_r<-21. Both correlation functions show subtle but systematic departures from the best-fit power law, in particular a change in slope at r_p~1-2 Mpc/h. These departures are stronger for the volume-limited sample, which is restricted to relatively luminous galaxies. We show that the inflection point in w_p(r_p) can be naturally explained by contemporary models of galaxy clustering, according to which it marks the transition from a large scale regime dominated by galaxy pairs in separate dark matter halos to a small scale regime dominated by galaxy pairs in the same dark matter halo. For example, given the dark halo population predicted by an inflationary cold dark matter scenario, the projected correlation function of the volume-limited sample can be well reproduced by a model in which the mean number of M_r<-21 galaxies in a halo of mass M>M_1=4.74 X 10^{13}\msun/h is <N>_M=(M/M_1)^{0.89}, with 75% of the galaxies residing in less massive, single-galaxy halos. This physically motivated model has the same number of free parameters as a power law, and it fits the w_p(r_p) data better, with a \chi^2/d.o.f.=0.93 compared to 6.12 (for 10 degrees of freedom, incorporating the covariance of the correlation function errors). Departures from a power-law correlation function encode information about the relation between galaxies and dark matter halos. Higher precision measurements of these departures for multiple classes of galaxies will constrain galaxy bias and provide new tests of the theory of galaxy formation.

We study the luminosity and color dependence of the galaxy 2-point correlation function in the Sloan Digital Sky Survey, starting from a sample of 200,000 galaxies over 2500 deg^2. We concentrate on the projected correlation function w(r_p), which is directly related to the real space \xi(r). The amplitude of w(r_p) grows continuously with luminosity, rising more steeply above the characteristic luminosity L_*. Redder galaxies exhibit a higher amplitude and steeper correlation function at all luminosities. The correlation amplitude of blue galaxies increases continuously with luminosity, but the luminosity dependence for red galaxies is less regular, with bright red galaxies more strongly clustered at large scales and faint red galaxies more strongly clustered at small scales. We interpret these results using halo occupation distribution (HOD) models assuming concordance cosmological parameters. For most samples, an HOD model with two adjustable parameters fits the w(r_p) data better than a power-law, explaining inflections at r_p ~ 1-3 Mpc/h as the transition between the 1-halo and 2-halo regimes of \xi(r). The implied minimum mass for a halo hosting a central galaxy above a luminosity threshold L grows as M_min ~ L at low luminosities and more steeply above L_*. The mass at which an average halo has one satellite galaxy brighter than L is M_1 ~ 23 M_min(L). These results imply a conditional luminosity function (at fixed halo mass) in which central galaxies lie far above a Schechter function extrapolation of the satellite population. HOD models nicely explain the joint luminosity-color dependence of w(r_p) in terms of the color fractions of central and satellite populations as a function of halo mass. The inferred HOD properties are in good qualitative agreement with theoretical predictions.

We investigate the three-point functions of the weak lensing cosmic shear, using both analytic methods and numerical results from N-body simulations. The analytic model, an isolated dark matter halo with a powerlaw profile chosen to fit the effective index at the scale probed, can be used to understand the basic properties of the eight three-point functions observed in simulations. We use this model to construct a single three-point function estimator that "optimally" combines the eight three-point functions. This new estimator is an alternative to Map statistics and provides up to a factor of two improvement in signal to noise compared to previously used combinations of cosmic shear three-point functions.

We study the behavior of the three-point correlation function xi_3 of dark matter and mock galaxies, concentrating on the effects of redshift-space distortions and the determination of galaxy bias parameters in current redshift galaxy surveys. On large scales, redshift space distortions tend to wash out slightly the configuration dependence of the reduced 3-point function Q3 ~ xi_3/xi_2^2. On smaller scales (< 10 Mpc/h), Q3 develops a characteristic U-shape anisotropy between elongated and open triangles due to the effects of velocity dispersion. We show that this shape is quite universal, very weakly dependent on scale, initial spectral index or cosmological parameters and should be detectable in current galaxy surveys even if affected by shot-noise or galaxy bias. We present a detailed method for obtaining constraints on galaxy bias parameters from measurements of Q3 in current galaxy redshift surveys, based on the eigenmode analysis similar to the one developed for the bispectrum. We show that our method recovers the bias parameters introduced into mock galaxies by a HOD prescription and is also able to handle potential systematics in the case when a smaller number than ideal of mock catalogs is used to estimate the covariance matrix. We find that current redshift surveys (e.g. SDSS or 2dFGRS) are just about large enough to get interesting new constraints on bias.

We present the large-scale correlation function measured from a spectroscopic sample of 46,748 luminous red galaxies from the Sloan Digital Sky Survey. The survey region covers 0.72 h^{-3} Gpc^3 over 3816 square degrees and 0.16<z<0.47, making it the best sample yet for the study of large-scale structure. We find a well-detected peak in the correlation function at 100h^{-1} Mpc separation that is an excellent match to the predicted shape and location of the imprint of the recombination-epoch acoustic oscillations on the low-redshift clustering of matter. This detection demonstrates the linear growth of structure by gravitational instability between z=1000 and the present and confirms a firm prediction of the standard cosmological theory. The acoustic peak provides a standard ruler by which we can measure the ratio of the distances to z=0.35 and z=1089 to 4% fractional accuracy and the absolute distance to z=0.35 to 5% accuracy. From the overall shape of the correlation function, we measure the matter density Omega_mh^2 to 8% and find agreement with the value from cosmic microwave background (CMB) anisotropies. Independent of the constraints provided by the CMB acoustic scale, we find Omega_m = 0.273 +- 0.025 + 0.123 (1+w_0) + 0.137 Omega_K. Including the CMB acoustic scale, we find that the spatial curvature is Omega_K=-0.010+-0.009 if the dark energy is a cosmological constant. More generally, our results provide a measurement of cosmological distance, and hence an argument for dark energy, based on a geometric method with the same simple physics as the microwave background anisotropies. The standard cosmological model convincingly passes these new and robust tests of its fundamental properties.

We show that current surveys have at least as much signal to noise in higher-order statistics as in the power spectrum at weakly nonlinear scales. We discuss how one can use this information to determine the mean of the galaxy halo occupation distribution (HOD) using only large-scale information, through galaxy bias parameters determined from the galaxy bispectrum and trispectrum. After introducing an averaged, reasonably fast to evaluate, trispectrum estimator, we show that the expected errors on linear and quadratic bias parameters can be reduced by at least 20-40%. Also, the inclusion of the trispectrum information, which is sensitive to "three-dimensionality" of structures, helps significantly in constraining the mass dependence of the HOD mean. Our approach depends only on adequate modeling of the abundance and large-scale clustering of halos and thus is independent of details of how galaxies are distributed within halos. This provides a consistency check on the traditional approach of using two-point statistics down to small scales, which necessarily makes more assumptions. We present a detailed forecast of how well our approach can be carried out in the case of the SDSS.

We derive the exact relationship, including all non-linearities, between real-space and redshift-space two-point statistics through the pairwise velocity distribution function. We show using numerical simulations that the pairwise velocity PDF is strongly non-Gaussian at all scales, and explain why this is so. We caution that a commonly used ansatz to model the redshift-space power spectrum gives rise to an unphysical distribution of pairwise velocities, and show that it is in general impossible to derive the distribution from measurements of redshift-space clustering. Methods that claim to do this obtain instead something else, whose properties we derive. We provide a general derivation of the large-scale limit of the redshift-space power spectrum and show that it differs from the Kaiser formula by terms that depend on Gaussian and non-Gaussian contributions to the velocity dispersion of large-scale flows. We also show that the large-scale evolution of velocity fields is not well described by linear theory and discuss how this impacts the redshift-space power spectrum. Finally, we stress that using the monopole of the redshift-space power as an indicator of the real-space power spectrum shape can lead to systematic effects in the determination of cosmological parameters; nevertheless a simple procedure is able to recover the large-scale real-space power spectrum rather well.

The nature of the fuel that drives today's cosmic acceleration is an open and tantalizing mystery. The braneworld theory of Dvali, Gabadadze, and Porrati (DGP) provides a context where late-time acceleration is driven not by some energy-momentum component (dark energy), but rather is the manifestation of the excruciatingly slow leakage of gravity off our four-dimensional world into an extra dimension. At the same time, DGP gravity alters the gravitational force law in a specific and dramatic way at cosmologically accessible scales. We derive the DGP gravitational force law in a cosmological setting for spherical perturbations at subhorizon scales and compute the growth of large-scale structure. We find that a residual repulsive force at large distances gives rise to a suppression of the growth of density and velocity perturbations. Explaining the cosmic acceleration in this framework leads to a present day fluctuation power spectrum normalization sigma_8 <= 0.8 at about the two-sigma level, in contrast with observations. We discuss further theoretical work necessary to go beyond our approximations to confirm these results.

We measure cosmological parameters using the three-dimensional power spectrum P(k) from over 200,000 galaxies in the Sloan Digital Sky Survey (SDSS) in combination with WMAP and other data. Our results are consistent with a ``vanilla'' flat adiabatic Lambda-CDM model without tilt (n=1), running tilt, tensor modes or massive neutrinos. Adding SDSS information more than halves the WMAP-only error bars on some parameters, tightening 1 sigma constraints on the Hubble parameter from h~0.74+0.18-0.07 to h~0.70+0.04-0.03, on the matter density from Omega_m~0.25+/-0.10 to Omega_m~0.30+/-0.04 (1 sigma) and on neutrino masses from <11 eV to <0.6 eV (95%). SDSS helps even more when dropping prior assumptions about curvature, neutrinos, tensor modes and the equation of state. Our results are in substantial agreement with the joint analysis of WMAP and the 2dF Galaxy Redshift Survey, which is an impressive consistency check with independent redshift survey data and analysis techniques. In this paper, we place particular emphasis on clarifying the physical origin of the constraints, i.e., what we do and do not know when using different data sets and prior assumptions. For instance, dropping the assumption that space is perfectly flat, the WMAP-only constraint on the measured age of the Universe tightens from t0~16.3+2.3-1.8 Gyr to t0~14.1+1.0-0.9 Gyr by adding SDSS and SN Ia data. Including tensors, running tilt, neutrino mass and equation of state in the list of free parameters, many constraints are still quite weak, but future cosmological measurements from SDSS and other sources should allow these to be substantially tightened.

We consider primordial non-Gaussianity due to quadratic corrections in the gravitational potential parametrized by a non-linear coupling parameter fnl. We study constraints on fnl from measurements of the galaxy bispectrum in redshift surveys. Using estimates for idealized survey geometries of the 2dF and SDSS surveys and realistic ones from SDSS mock catalogs, we show that it is possible to probe |fnl|~100, after marginalization over bias parameters. We apply our methods to the galaxy bispectrum measured from the PSCz survey, and obtain a 2sigma-constraint |fnl|< 1800. We estimate that an all sky redshift survey up to z~1 can probe |fnl|~1. We also consider the use of cluster abundance to constrain fnl and find that in order to be sensitive to |fnl|~100, cluster masses need to be determined with an accuracy of a few percent, assuming perfect knowledge of the mass function and cosmological parameters.

We measure the large-scale real-space power spectrum P(k) using a sample of 205,443 galaxies from the Sloan Digital Sky Survey, covering 2417 square degrees with mean redshift z~0.1. We employ a matrix-based method using pseudo-Karhunen-Loeve eigenmodes, producing uncorrelated minimum-variance measurements in 22 k-bands of both the clustering power and its anisotropy due to redshift-space distortions, with narrow and well-behaved window functions in the range 0.02 h/Mpc < k < 0.3h/Mpc. We pay particular attention to modeling, quantifying and correcting for potential systematic errors, nonlinear redshift distortions and the artificial red-tilt caused by luminosity-dependent bias. Our final result is a measurement of the real-space matter power spectrum P(k) up to an unknown overall multiplicative bias factor. Our calculations suggest that this bias factor is independent of scale to better than a few percent for k<0.1h/Mpc, thereby making our results useful for precision measurements of cosmological parameters in conjunction with data from other experiments such as the WMAP satellite. As a simple characterization of the data, our measurements are well fit by a flat scale-invariant adiabatic cosmological model with h Omega_m =0.201+/- 0.017 and L* galaxy sigma_8=0.89 +/- 0.02 when fixing the baryon fraction Omega_b/Omega_m=0.17 and the Hubble parameter h=0.72; cosmological interpretation is given in a companion paper.

The nature of the fuel that drives today's cosmic acceleration is an open and tantalizing mystery. We entertain the suggestion that the acceleration is not the manifestation of yet another new ingredient in the cosmic gas tank, but rather a signal of our first real lack of understanding of gravitational physics. By requiring that the underlying gravity theory respects Birkhoff's law, we derive the modified gravitational force-law necessary to generate any given cosmology, without reference to the fundamental theory, revealing modifications of gravity at scales typically much smaller than today's horizon. We discuss how through these modifications, the growth of density perturbations, the late-time integrated Sachs--Wolfe effect, and even solar-system measurements may be sensitive to whether today's cosmic acceleration is generated by dark energy or modified gravitational dynamics, and are subject to imminent observational discrimination. We argue how these conclusions can be more generic, and probably not dependent on the validity of Birkhoff's law.

We discuss the determination of the linear power spectrum of dark matter fluctuations from measurements of the transmitted flux Ly-alpha Forest power spectrum. We show that at most scales probed by current measurements, the flux power spectrum is significantly affected by non-linear corrections to the linear dark matter power spectrum due to gravitational clustering. Inferring the linear dark matter power spectrum shape is therefore difficult due to non-linear corrections driving the non-linear power spectrum to a k^{-1.4} shape nearly independent of initial conditions. We argue that some methods used in previous estimates underestimate the uncertainties in the shape of the linear dark matter power spectrum.

We present a method for defining higher-order moments of a spin-2 field on the sky using the transformation properties of these statistics under rotation and parity. For the three-point function of the cosmic shear we show that the eight logically possible combinations of the shear in three points can be divided into two classes, four combinations are even under parity transformations and four are odd. We compute the expected value of the even parity ones in the non-linear regime using the halo model and conclude that on small scales of the four combinations there is one that is expected to carry most of the signal for triangles close to isosceles. On the other hand, for collapsed triangles all four combinations are expected to have roughly the same level of signal although some of the combinations are negative and others positive. We estimate that a survey of a few square degrees area is enough to detect this signal above the noise at arc minute scales.

Recent observations of high redshift quasar spectra reveal long gaps with little flux. A small or no detectable flux does not by itself imply the intergalactic medium (IGM) is neutral. Inferring the average neutral fraction from the observed absorption requires assumptions about clustering of the IGM, which the gravitational instability model supplies. Our most stringent constraint on the neutral fraction at z ~ 6 is derived from the mean Lyman-beta transmission measured from the z=6.28 SDSS quasar of Becker et al. -- the neutral hydrogen fraction at mean density has to be larger than 4.7 times 10^{-4}. This is substantially higher than the neutral fraction of ~ 3-5 times 10^{-5} at z = 4.5 - 5.7, suggesting that dramatic changes take place around or just before z ~ 6, even though current constraints are still consistent with a fairly ionized IGM at z ~ 6. An interesting alternative method to constrain the neutral fraction is to consider the probability of having many consecutive pixels with little flux, which is small unless the neutral fraction is high. This constraint is slightly weaker than the one obtained from the mean transmission. We show that while the derived neutral fraction at a given redshift is sensitive to the power spectrum normalization, the size of the jump around z ~ 6 is not. We caution that systematic uncertainties include spatial fluctuations in the ionizing background, and the continuum placement. Tests are proposed. In particular, the sightline to sightline dispersion in mean transmission might provide a useful diagnostic. We express the dispersion in terms of the transmission power spectrum, and develop a method to calculate the dispersion for spectra that are longer than the typical simulation box.

The angular distribution of galaxies encodes a wealth of information about large scale structure. Ultimately, the Sloan Digital Sky Survey (SDSS) will record the angular positions of order 10^8 galaxies in five bands, adding significantly to the cosmological constraints. This is the first in a series of papers analyzing a rectangular stripe 2.5x90 degrees from early SDSS data. We present the angular correlation function for galaxies in four separate magnitude bins on angular scales ranging from 0.003 degrees to 15 degrees. Much of the focus of this paper is on potential systematic effects. We show that the final galaxy catalog -- with the mask accounting for regions of poor seeing, reddening, bright stars, etc. -- is free from external and internal systematic effects for galaxies brighter than r* = 22. Our estimator of the angular correlation function includes the effects of the integral constraint and the mask. The full covariance matrix of errors in these estimates is derived using mock catalogs with further estimates using a number of other methods.

We compute the angular power spectrum C_l from 1.5 million galaxies in early SDSS data on large angular scales, l<600. The data set covers about 160 square degrees, with a characteristic depth of order 1 Gpc/h in the faintest (21<r<22) of our four magnitude bins. Cosmological interpretations of these results are presented in a companion paper by Dodelson et al (2001). The data in all four magnitude bins are consistent with a simple flat ``concordance'' model with nonlinear evolution and linear bias factors of order unity. Nonlinear evolution is particularly evident for the brightest galaxies. A series of tests suggest that systematic errors related to seeing, reddening, etc., are negligible, which bodes well for the sixtyfold larger sample that the SDSS is currently collecting. Uncorrelated error bars and well-behaved window functions make our measurements a convenient starting point for cosmological model fitting.

We review the formalism and applications of non-linear perturbation theory (PT) to understanding the large-scale structure of the Universe. We first discuss the dynamics of gravitational instability, from the linear to the non-linear regime. This includes Eulerian and Lagrangian PT, non-linear approximations, and a brief description of numerical simulation techniques. We then cover the basic statistical tools used in cosmology to describe cosmic fields, such as correlations functions in real and Fourier space, probability distribution functions, cumulants and generating functions. In subsequent sections we review the use of PT to make quantitative predictions about these statistics according to initial conditions, including effects of possible non Gaussianity of the primordial fields. Results are illustrated by detailed comparisons of PT predictions with numerical simulations. The last sections deal with applications to observations. First we review in detail practical estimators of statistics in galaxy catalogs and related errors, including traditional approaches and more recent developments. Then, we consider the effects of the bias between the galaxy distribution and the matter distribution, the treatment of redshift distortions in three-dimensional surveys and of projection effects in angular catalogs, and some applications to weak gravitational lensing. We finally review the current observational situation regarding statistics in galaxy catalogs and what the future generation of galaxy surveys promises to deliver.

We present initial results for counts in cells statistics of the angular distribution of galaxies in early data from the Sloan Digital Sky Survey (SDSS). We analyze a rectangular stripe $2.5^\circ$ wide, covering approximately 160 sq. degrees, containing over $10^6$ galaxies in the apparent magnitude range $18 < r^\prime < 22$, with areas of bad seeing, contamination from bright stars, ghosts, and high galactic extinction masked out. This survey region, which forms part of the SDSS Early Data Release, is the same as that for which two-point angular clustering statistics have recently been computed. The third and fourth moments of the cell counts, $s_3$ (skewness) and $s_4$ (kurtosis), constitute the most accurate measurements to date of these quantities (for $r^\prime < 21$) over angular scales $0.015^\circ-0.3^\circ$. They display the approximate hierarchical scaling expected from non-linear structure formation models and are in reasonable agreement with the predictions of $\Lambda$-dominated cold dark matter models with galaxy biasing that suppresses higher order correlations at small scales. The results are in general consistent with previous measurements in the APM, EDSGC, and Deeprange surveys. These results suggest that the SDSS imaging data are free of systematics to a high degree and will therefore enable determination of the skewness and kurtosis to 1% and less then 10%, as predicted by Colombi, Szapudi, & Szalay (1998).

We present measurements of parameters of the 3-dimensional power spectrum of galaxy clustering from 222 square degrees of early imaging data in the Sloan Digital Sky Survey. The projected galaxy distribution on the sky is expanded over a set of Karhunen-Loeve eigenfunctions, which optimize the signal-to-noise ratio in our analysis. A maximum likelihood analysis is used to estimate parameters that set the shape and amplitude of the 3-dimensional power spectrum. Our best estimates are Gamma=0.188 +/- 0.04 and sigma_8L = 0.915 +/- 0.06 (statistical errors only), for a flat Universe with a cosmological constant. We demonstrate that our measurements contain signal from scales at or beyond the peak of the 3D power spectrum. We discuss how the results scale with systematic uncertainties, like the radial selection function. We find that the central values satisfy the analytically estimated scaling relation. We have also explored the effects of evolutionary corrections, various truncations of the KL basis, seeing, sample size and limiting magnitude. We find that the impact of most of these uncertainties stay within the 2-sigma uncertainties of our fiducial result.

The Sloan Digital Sky Survey is one of the first multicolor photometric and spectroscopic surveys designed to measure the statistical properties of galaxies within the local Universe. In this Letter we present some of the initial results on the angular 2-point correlation function measured from the early SDSS galaxy data. The form of the correlation function, over the magnitude interval 18<r*<22, is shown to be consistent with results from existing wide-field, photographic-based surveys and narrower CCD galaxy surveys. On scales between 1 arcminute and 1 degree the correlation function is well described by a power-law with an exponent of ~ -0.7. The amplitude of the correlation function, within this angular interval, decreases with fainter magnitudes in good agreement with analyses from existing galaxy surveys. There is a characteristic break in the correlation function on scales of approximately 1-2 degrees. On small scales, < 1', the SDSS correlation function does not appear to be consistent with the power-law form fitted to the 1'< theta <0.5 deg data. With a data set that is less than 2% of the full SDSS survey area, we have obtained high precision measurements of the power-law angular correlation function on angular scales 1' < theta < 1 deg, which are robust to systematic uncertainties. Because of the limited area and the highly correlated nature of the error covariance matrix, these initial results do not yet provide a definitive characterization of departures from the power-law form at smaller and larger angles. In the near future, however, the area of the SDSS imaging survey will be sufficient to allow detailed analysis of the small and large scale regimes, measurements of higher-order correlations, and studies of angular clustering as a function of redshift and galaxy type.

Early photometric data from the Sloan Digital Sky Survey (SDSS) contain angular positions for 1.5 million galaxies. In companion papers, the angular correlation function $w(\theta)$ and 2D power spectrum $C_l$ of these galaxies are presented. Here we invert Limber's equation to extract the 3D power spectrum from the angular results. We accomplish this using an estimate of $dn/dz$, the redshift distribution of galaxies in four different magnitude slices in the SDSS photometric catalog. The resulting 3D power spectrum estimates from $w(\theta)$ and $C_l$ agree with each other and with previous estimates over a range in wavenumbers $0.03 < k/{\rm h Mpc}^{-1} < 1$. The galaxies in the faintest magnitude bin ($21 < \rstar < 22$, which have median redshift $z_m=0.43$) are less clustered than the galaxies in the brightest magnitude bin ($18 < \rstar < 19$ with $z_m=0.17$), especially on scales where nonlinearities are important. The derived power spectrum agrees with that of Szalay et al. (2001) who go directly from the raw data to a parametric estimate of the power spectrum. The strongest constraints on the shape parameter $\Gamma$ come from the faintest galaxies (in the magnitude bin $21 < \rstar < 22$), from which we infer $\Gamma = 0.14^{+0.11}_{-0.06}$ (95% C.L.).

Current models of galaxy formation applied to understanding the large-scale structure of the universe have two parts. The first is an accurate solution of the equations of motion for the dark matter due to gravitational clustering. The second consists of making physically reasonable approximations to the behavior of baryons inside dark matter halos. The first uses large, computationally intensive, $n$-body simulations. We argue that because the second step is, at least at present, uncertain, it is possible to obtain similar galaxy distributions without solving the first step exactly. We describe an algorithm which is several orders of magnitude faster than n-body simulations, but which is, nevertheless, rather accurate. The algorithm combines perturbation theory with virialized halo models of the nonlinear density and velocity fields. For two- and three-point statistics the resulting fields are exact on large scales, and rather accurate well into the nonlinear regime, particularly for two-point statistics in real and redshift space. We then show how to use this algorithm to generate mock galaxy distributions from halo occupation numbers. As a first application, we show that it provides a good description of the clustering of galaxies in the PSCz survey. We also discuss applications to the estimation of non-Gaussian contributions to error bars and covariance matrix of the power spectrum, in real and redshift space, for galaxies and dark matter. The results for the latter show good agreement with simulations, supporting the use of our method to constrain cosmological parameters from upcoming galaxy surveys.

We consider gravitational clustering from primoridal non-Gaussian fluctuations provided by a $\chi^2$ model, as motivated by some models of inflation. The emphasis is in signatures that can be used to constrain this type of models from large-scale structure galaxy surveys. Non-Gaussian initial conditions provide additional non-linear couplings otherwise forbidden by symmetry that cause non-linear gravitational corrections to become important at larger scales than in the Gaussian case. In fact, the lack of hierarchical scaling in the initial conditions is partially restored by gravitational evolution at scales $k> 0.1$ h/Mpc. However, the bispectrum shows much larger amplitude and residual scale dependence not present in evolution from Gaussian initial conditions that can be used to test this model against observations. We include the effects of biasing and redshift distortions essential to compare this model with galaxy redshift surveys. We also discuss the effects of primordial non-Gaussianity on the redshift-space power spectrum and show that it changes the shape of the quadrupole to monopole ratio through non-linear corrections to infall velocities.

We compute the bispectrum for the \IRAS PSCz catalog and find that the galaxy distribution displays the characteristic signature of gravity. Assuming Gaussian initial conditions, we obtain galaxy biasing parameters $1/b_1=1.20^{+0.18}_{-0.19}$ and $b_2/b_1^2=-0.42\pm0.19$, with no sign of scale-dependent bias for $k\leq 0.3$ h/Mpc. These results impose stringent constraints on non-Gaussian initial conditions. For dimensional scaling models with $\chi^2_N$ statistics, we find N>49, which implies a constraint on primordial skewness $B_3<0.35$.

A simple model of how objects of different masses stream towards each other as they cluster gravitationally is described. The model shows how the mean streaming velocity of dark matter particles is related to the motions of the parent dark matter haloes. It also provides a reasonably accurate description of how the pairwise velocity dispersion of dark matter particles differs from that of the parent haloes. The analysis is then extended to describe the streaming motions of galaxies. This shows explicitly that the streaming motions measured in a given galaxy sample depend on how the sample was selected, and shows how to account for this dependence on sample selection. In addition,we show that the pairwise dispersion should also depend on sample type. Our model predicts that, on small scales, redshift space distortions should affect red galaxies more strongly than blue.

As pointed out in previous studies, the measurement of the skewness of the convergence field $\kappa$ will be useful in breaking the degeneracy among the cosmological parameters constrained from weak lensing observations. The combination of shot noise and finite survey volume implies that such a measurement is likely to be done in a range of intermediate scales ($0.5'$ to 20') where neither perturbation theory nor the hierarchical ansatz apply. Here we explore the behavior of the skewness of $\kappa$ at these intermediate scales, based on results for the non-linear evolution of the mass bispectrum. We combined different ray-tracing simulations to test our predictions, and we find that our calculations describe accurately the transition from the weakly non-linear to the strongly non-linear regime. We show that the single lens-plane approximation remains accurate even in the non-linear regime, and we explicitly calculate the corrections to this approximation. We also discuss the prospects of measuring the skewness in upcoming weak lensing surveys.

We present a fitting formula for the non-linear evolution of the bispectrum in CDM models, obtained from measurements in high resolution numerical simulations. The formula interpolates between the perturbative and highly non-linear regimes, and generalizes previous results obtained for scale-free initial conditions.

We write the correlation function of dark matter particles, xi(r), as the sum of two terms - one which accounts for nonlinear evolution, and dominates on small scales, and another which is essentially the term from linear theory, and dominates on large scales. We use models of the number and spatial distribution of haloes and halo density profiles to describe the nonlinear term and its evolution. The result provides a good description of the evolution of xi(r) in simulations. We then use this decomposition to provide simple and accurate models of how the single particle velocity dispersion evolves with time, and how the first and second moments of the pairwise velocity distribution depend on scale. The key idea is to use the simple physics of linear theory on large scales, the simple physics of the virial theorem on small scales, and our model for the correlation function to tell us how to weight the two types of contributions (linear and nonlinear) to the pairwise velocity statistics. When incorporated into the streaming model, our results will allow a simple accurate description of redshift-space distortions over the entire range of linear to highly nonlinear regimes.

We describe a new approach to gravitational instability in large-scale structure, where the equations of motion are written and solved as in field theory in terms of Feynman diagrams. The basic objects of interest are the propagator (which propagates solutions forward in time), the vertex (which describes non-linear interactions between waves) and a source with prescribed statistics which describes the effect of initial conditions. We show that loop corrections renormalize these quantities, and discuss applications of this formalism to a better understanding of gravitational instability and to improving non-linear perturbation theory in the transition to the non-linear regime. We also consider the role of vorticity creation due to shell-crossing and show using N-body simulations that at small (virialized) scales the velocity field reaches equipartition, i.e. the vorticity power spectrum is about twice the divergence power spectrum.

The bispectrum is the lowest-order statistic sensitive to the shape of structures generated by gravitational instability and is a potentially powerful probe of galaxy biasing and the Gaussianity of primordial fluctuations. Although the evolution of the bispectrum is well understood theoretically from non-linear perturbation theory and numerical simulations, applications to galaxy surveys require a number of issues to be addressed. In this paper we consider the effect on the bispectrum of stochastic non-linear biasing, radial redshift distortions, non-Gaussian initial conditions, survey geometry and sampling. We find that: 1) bias stochasticity does not affect the use of the bispectrum to recover the mean biasing relation between galaxies and mass, at least for models in which the scatter is uncorrelated at large scales. 2) radial redshift distortions do not change significantly the monopole power spectrum and bispectrum compared to their plane-parallel values. 3) survey geometry leads to finite volume effects which must be taken into account in current surveys before comparison with theoretical predictions can be made. 4) sparse sampling and survey geometry correlate different triangles leading to a breakdown of the Gaussian likelihood approximation. We develop a likelihood analysis using bispectrum eigenmodes, calculated by Monte Carlo realizations of mock surveys generated with second-order Lagrangian perturbation theory and checked against N-body simulations. In a companion paper we apply these results to the analysis of the bispectrum of IRAS galaxies.

We study galaxy clustering using halo models, where gravitational clustering is described in terms of dark matter halos. At small scales, clustering statistics are dominated by halo density profiles, whereas at large scales, correlations are the result of combining non-linear perturbation theory with halo biasing. Galaxies are assumed to follow the dark matter profiles, and galaxy formation efficiency is given by the number of galaxies as a function of halo mass. This approach leads to generic predictions: the galaxy power spectrum shows a power-law behavior even though the dark matter does not, and the galaxy higher-order correlations show smaller amplitudes at small scales than their dark matter counterparts, as observed in galaxy catalogs. We find that requiring to fit both the second and third order moments of the APM galaxies provides a strong constraint on galaxy formation models. The data at large scales require that galaxy formation be relatively efficient at small masses, m =10^10 Msun/h, whereas data at smaller scales require that the number of galaxies in a halo scale as the mass to the 0.8th power in the high-mass limit. These constraints are independent of those derived from the luminosity function or Tully-Fisher relation. We also predict the power spectrum, bispectrum, and higher-order moments of the mass density field. Although halo models agree well with measurements of the mass power spectrum and the higher order Sp parameters in N-body simulations, the model assumption that halos are spherical leads to disagreement in the configuration dependence of the bispectrum at small scales. We stress the importance of finite volume effects in higher-order statistics and show how they can be estimated in this approach.

We compute the bispectrum for the galaxy distribution in the IRAS QDOT, 2Jy, and 1.2Jy redshift catalogs for wavenumbers 0.05<k<0.2 h/Mpc and compare the results with predictions from gravitational instability in perturbation theory. Taking into account redshift space distortions, nonlinear evolution, the survey selection function, and discreteness and finite volume effects, all three catalogs show evidence for the dependence of the bispectrum on configuration shape predicted by gravitational instability. Assuming Gaussian initial conditions and local biasing parametrized by linear and non-linear bias parameters b_1 and b_2, a likelihood analysis yields 1/b_1 = 1.32^{+0.36}_{-0.58}, 1.15^{+0.39}_{-0.39} and b_2/b_1^2=-0.57^{+0.45}_{-0.30}, -0.50^{+0.31}_{-0.51}, for the for the 2Jy and 1.2Jy samples, respectively. This implies that IRAS galaxies trace dark matter increasingly weakly as the density contrast increases, consistent with their being under-represented in clusters. In a model with chi^2 non-Gaussian initial conditions, the bispectrum displays an amplitude and scale dependence different than that found in the Gaussian case; if IRAS galaxies do not have bias b_1> 1 at large scales, \chi^2 non-Gaussian initial conditions are ruled out at the 95% confidence level. The IRAS data do not distinguish between Lagrangian or Eulerian local bias.

Gravitational clustering is an intrinsically non-linear process that generates significant non-Gaussian signatures in the density field. We consider how these affect power spectrum determinations from galaxy and weak-lensing surveys. Non-Gaussian effects not only increase the individual error bars compared to the Gaussian case but, most importantly, lead to non-trivial cross-correlations between different band-powers. We calculate the power-spectrum covariance matrix in non-linear perturbation theory (weakly non-linear regime), in the hierarchical model (strongly non-linear regime), and from numerical simulations in real and redshift space. We discuss the impact of these results on parameter estimation from power spectrum measurements and their dependence on the size of the survey and the choice of band-powers. We show that the non-Gaussian terms in the covariance matrix become dominant for scales smaller than the non-linear scale, depending somewhat on power normalization. Furthermore, we find that cross-correlations mostly deteriorate the determination of the amplitude of a rescaled power spectrum, whereas its shape is less affected. In weak lensing surveys the projection tends to reduce the importance of non-Gaussian effects. Even so, for background galaxies at redshift z=1, the non-Gaussian contribution rises significantly around l=1000, and could become comparable to the Gaussian terms depending upon the power spectrum normalization and cosmology. The projection has another interesting effect: the ratio between non-Gaussian and Gaussian contributions saturates and can even decrease at small enough angular scales if the power spectrum of the 3D field falls faster than 1/k^2.

We consider the long-standing problem of predicting the hierarchical clustering amplitudes $S_p$ in the strongly non-linear regime of gravitational evolution. N-body results for the non-linear evolution of the bispectrum (the Fourier transform of the three-point density correlation function) suggest a physically motivated ansatz that yields the strongly non-linear behavior of the skewness, $S_3$, starting from leading-order perturbation theory. When generalized to higher-order ($p>3$) polyspectra or correlation functions, this ansatz leads to a good description of non-linear amplitudes in the strongly non-linear regime for both scale-free and cold dark matter models. Furthermore, these results allow us to provide a general fitting formula for the non-linear evolution of the bispectrum that interpolates between the weakly and strongly non-linear regimes, analogous to previous expressions for the power spectrum.

The bispectrum provides a characteristic signature of gravitational instability that can be used to probe the Gaussianity of the initial conditions and the bias of the galaxy distribution. We study how this signature is affected by redshift distortions using perturbation theory and high-resolution numerical simulations. We obtain perturbative results for the multipole expansion of the redshift-space bispectrum which provide a natural way to break the degeneracy between bias and $\Omega$ present in measurements of the redshift-space power spectrum. We propose a phenomenological model that incorporates the perturbative results and also describes the bispectrum in the transition to the non-linear regime. We stress the importance of non-linear effects and show that inaccurate treatment of these can lead to significant discrepancies in the determination of bias from galaxy redshift surveys. At small scales we find that the bispectrum monopole exhibits a strong configuration dependence that reflects the velocity dispersion of clusters. Therefore, the hierarchical model for the three-point function does not hold in redshift-space.

I discuss the evolution of the redshift-space bispectrum via perturbation theory (PT) and large high-resolution numerical simulations. At large scales, we give the multipole expansion of the bispectrum in PT, which provides a natural way to break the degeneracy between bias and $\Omega$ present in measurements of the power spectrum distortions. At intermediate scales, we propose a simple phenomenological model to take into account non-linear effects. N-body results show that at small scales the perturbative shape of the bispectrum monopole in redshift-space is preserved, breaking the hierarchical form valid in the absence of distortions.

Hydrogen atoms in the Universe that initially formed at redshift z = 1100 were reionized at 5 < z < 60. It is likely that this transition proceeded through a mixed phase of partial ionization. We develop an analytic approach to calculating the effect on the cosmic microwave background (CMB) of Thomson scattering off this inhomogeneous distribution of free electrons. We study two models of the reionization process, one of which associates ionized patches with overdense regions of the smoothed density field. We find the generated anisotropy to have a power spectrum which peaks at angular scales corresponding to the extent of the ionized regions, and has a width that reflects the correlations between them. There is considerable uncertainty in the amplitude, which depends on how efficiently collapsed objects can reionize the surrounding medium. We calculate the effect of neglecting inhomogeneous reionization in the determination of cosmological parameters from high resolution CMB maps and find that it may be a significant source of systematic error.

The standard procedure to generate initial conditions (IC) in numerical simulations is to use the Zel'dovich approximation (ZA). Although the ZA correctly reproduces the linear growing modes of density and velocity perturbations, non-linear growth is inaccurately represented because of the ZA failure to conserve momentum. This implies that it takes time for the actual dynamics to establish the correct statistical properties of density and velocity fields. We extend perturbation theory (PT) to include transients as non-linear excitations of decaying modes caused by the IC. We focus on higher-order statistics of the density contrast and velocity divergence, characterized by the S_p and T_p parameters. We find that the time-scale of transients is determined, at a given order p, by the spectral index n. The skewness factor S_3 (T_3) attains 10% accuracy only after a=6 (a=15) for n=0, whereas higher (lower) n demands more (less) expansion away from the IC. These requirements become much more stringent as p increases. An Omega=0.3 model requires a factor of two larger expansion than an Omega=1 model to reduce transients by the same amount. The predicted transients in S_p are in good agreement with numerical simulations. More accurate IC can be achieved by using 2nd order Lagrangian PT (2LPT), which reproduces growing modes up to 2nd order and thus eliminates transients in the skewness. We show that for p>3 this reduces the required expansion by more than an order of magnitude compared to the ZA. Setting up 2LPT IC only requires minimal, inexpensive changes to ZA codes. We suggest simple steps for its implementation.

The bispectrum, the three-point function of density fluctuations in Fourier space, is the lowest order statistic that carries information about the spatial coherence of large-scale structures. For Gaussian initial conditions, when the density fluctuation amplitude is small, tree-level (leading order) perturbation theory predicts a characteristic dependence of the bispectrum on the shape of the triangle formed by the three wave vectors. This configuration dependence provides a signature of gravitational instability, and departures from it in galaxy catalogs can be interpreted as due to bias, that is, non-gravitational effects. On the other hand, N-body simulations indicate that the reduced three-point function becomes relatively shape-independent in the strongly non-linear regime. In order to understand this non-linear transition and assess the domain of reliability of shape-dependence as a probe of bias, we calculate the one-loop (next-to-leading order) corrections to the bispectrum in perturbation theory. We compare these results with measurements in numerical simulations with scale-free and CDM initial power spectra. We find that the one-loop corrections account very well for the departures from the tree-level results measured in numerical simulations on weakly non-linear scales. In this regime, the reduced bispectrum qualitatively retains its tree-level shape, but the amplitude can change significantly. At smaller scales, the reduced bispectrum in the simulations starts to flatten, an effect which can be partially understood from the one-loop results. In the strong clustering regime, where perturbation theory breaks down entirely, the simulation results confirm that the reduced bispectrum has almost no dependence on triangle shape, in rough agreement with the hierarchical ansatz.

I discuss new results concerning the evolution of the bispectrum due to gravitational instability from gaussian initial conditions using one-loop perturbation theory (PT). Particular attention is paid to the transition from weakly non-linear scales to the non-linear regime at small scales. Comparison with numerical simulations is made to assess the regime of validity of the perturbative approach.

We consider one-loop corrections to the bispectrum and skewness of cosmological density fluctuations induced by gravitational evolution. As has been established by comparison with numerical simulations, tree-level perturbation theory (PT) describes these quantities at the largest scales. One-loop PT provides a tool to probe the transition to the non-linear regime on smaller scales. In this work, we find that, as a function of spectral index n, the one-loop bispectrum follows a pattern analogous to that of the one-loop power spectrum, which shows a change in behavior at a critical index n_c = -1.4, where non-linear corrections vanish. For the bispectrum, for n less than n_c, one-loop corrections increase the configuration dependence of the leading order contribution; for n greater than n_c, one-loop corrections tend to cancel the configuration dependence of the tree-level bispectrum, in agreement with known results from n=-1 numerical simulations. A similar situation is shown to hold for the Zel'dovich approximation (ZA), where n_c = -1.75. Using dimensional regularization, we obtain explicit analytic expressions for the one-loop bispectrum for n=-2 initial power spectra, for both the exact dynamics of gravitational instability and the ZA. We also compute the skewness factor, including local averaging of the density field, for n=-2: S_3(R) = 4.02 + 3.83 sigma^2(R) for gaussian smoothing and S_3(R) = 3.86 + 3.18 sigma^2(R) for top-hat smoothing, where sigma^2(R) is the variance of the density field fluctuations smoothed over a window of radius R. Comparison with fully non-linear numerical simulations implies that, for n < -1, one-loop PT can extend our understanding of nonlinear clustering down to scales where the transition to the stable clustering regime begins.

We calculate the lowest-order non-linear contributions to the power spectrum, two-point correlation function, and smoothed variance of the density field, for Gaussian initial conditions and scale-free initial power spectra, $P(k) \sim k^n$. These results extend and in some cases correct previous work in the literature on cosmological perturbation theory. Comparing with the scaling behavior observed in N-body simulations, we find that the validity of non-linear perturbation theory depends strongly on the spectral index $n$. For $n<-1$, we find excellent agreement over scales where the variance $\sigma^2(R) \la 10$; however, for $n \geq -1$, perturbation theory predicts deviations from self-similar scaling (which increase with $n$) not seen in numerical simulations. This anomalous scaling suggests that the principal assumption underlying cosmological perturbation theory, that large-scale fields can be described perturbatively even when fluctuations are highly non-linear on small scales, breaks down beyond leading order for spectral indices $n \geq -1$. For $n < -1$, the power spectrum, variance, and correlation function in the scaling regime can be calculated using dimensional regularization.

Using a diagrammatic approach to Eulerian perturbation theory, we analytically calculate the variance and skewness of the density and velocity divergence induced by gravitational evolution from Gaussian initial conditions, including corrections *beyond* leading order. Except for the power spectrum, previous calculations in cosmological perturbation theory have been confined to leading order (tree level)-we extend these to include loop corrections. For scale-free initial power spectra, the one-loop variance \sigma^2 = \sigma^2_l + 1.82 \sigma^4_l and the skewness S_3 = 34/7 + 9.8 \sigma^2_l, where \sigma_l is the rms fluctuation of the linear density field. We also compute loop corrections to the variance, skewness, and kurtosis for several non-linear approximation schemes, where the calculation can be easily generalized to 1-point cumulants of higher order and arbitrary number of loops. We find that the Zel'dovich approximation gives the best approximation to the loop corrections of exact perturbation theory, followed by the Linear Potential approximation (LPA) and the Frozen Flow approximation (FFA), in qualitative agreement with the relative behavior of tree-level results. In LPA and FFA, loop corrections are infrared divergent for spectral indices n < 0; this is related to the breaking of Galilean invariance in these schemes.

We study the prospects for the current microlensing searches, which have recently discovered several candidates, to yield useful information about the flattening of the Galaxy dark matter halo. Models of HI warps and N-body simulations of galaxy formation suggest that disks commonly form in oblate halos with a tilt between the disk and halo symmetry axes. The microlensing optical depth for the Large Magellanic Cloud depends sensitively on the disk-halo tilt angle in the Milky Way. We conclude that a much larger spread in values for $\tau(LMC)$ is consistent with rotation curve constraints than previously thought, and that the ratio $\tau(SMC)/\tau(LMC)$ of the optical depth to the Small and Large Magellanic clouds is not a clean test of halo flattening.