Bringing a high-dimensional dataset into science-ready shape is a formidable challenge that often necessitates data compression. Compression has accordingly become a key consideration for contemporary cosmology, affecting public data releases, and reanalyses searching for new physics. However, data compression optimized for a particular model can suppress signs of new physics, or even remove them altogether. We therefore provide a solution for exploring new physics \emph{during} data compression. In particular, we store additional agnostic compressed data points, selected to enable precise constraints of non-standard physics at a later date. Our procedure is based on the maximal compression of the MOPED algorithm, which optimally filters the data with respect to a baseline model. We select additional filters, based on a generalised principal component analysis, which are carefully constructed to scout for new physics at high precision and speed. We refer to the augmented set of filters as MOPED-PC. They enable an analytic computation of Bayesian evidences that may indicate the presence of new physics, and fast analytic estimates of best-fitting parameters when adopting a specific non-standard theory, without further expensive MCMC analysis. As there may be large numbers of non-standard theories, the speed of the method becomes essential. Should no new physics be found, then our approach preserves the precision of the standard parameters. As a result, we achieve very rapid and maximally precise constraints of standard and non-standard physics, with a technique that scales well to large dimensional datasets.

In the light of the discovery of numerous (almost) dark galaxies from the ALFALAFA and LITTLE THINGS surveys, we revisit the predictions of Jimenez et al. 1997, based on the Toomre stability of rapidly-spinning gas disks. We have updated the predictions for $\Lambda$CDM with parameters given by Planck18, computing the expected number densities of dark objects, and their spin parameter and mass distributions. Comparing with the data is more challenging, but where the spins are more reliably determined, the spins are close to the threshold for disks to be stable according to the Toomre criterion, where the expected number density is highest, and reinforces the concept that there is a bias in the formation of luminous galaxies based on the spin of their parent halo.

In this paper, we propose a Gaussian Process (GP) emulator for the calculation of a) tomographic weak lensing band-power spectra, and b) coefficients of summary data massively compressed with the MOPED algorithm. In the former case cosmological parameter inference is accelerated by a factor of $\sim 10$-$30$ compared to explicit calls to the Boltzmann solver CLASS when applied to KiDS-450 weak lensing data. Much larger gains will come with future data, where with MOPED compression, the speed up can be up to a factor of $\sim 10^3$ when the common Limber approximation is used. Furthermore, the GP opens up the possibility of dropping the Limber approximation, without which the theoretical calculations may be unfeasibly slow. A potential advantage of GPs is that an error on the emulated function can be computed and this uncertainty incorporated into the likelihood. If speed is of the essence, then the mean of the Gaussian Process can be used and the uncertainty ignored. We compute the Kullback-Leibler divergence between the emulator likelihood and the CLASS likelihood, and on the basis of this and from analysing the uncertainties on the parameters, we find that in this case, the inclusion of the GP uncertainty does not justify the extra computational expense in the test application. For future weak lensing surveys such as Euclid and Legacy Survey of Space and Telescope (LSST), the number of summary statistics will be large, up to $\sim 10^{4}$. The speed of MOPED is determined by the number of parameters, not the number of summary data, so the gains are very large. In the non-Limber case, the speed-up can be a factor of $\sim 10^5$, provided that a fast way to compute the theoretical MOPED coefficients is available. The GP presented here provides such a fast mechanism and enables MOPED to be employed.

Existing cosmological simulation methods lack a high degree of parallelism due to the long-range nature of the gravitational force, which limits the size of simulations that can be run at high resolution. To solve this problem, we propose a new, perfectly parallel approach to simulate cosmic structure formation, based on the spatial COmoving Lagrangian Acceleration (sCOLA) framework. Building upon a hybrid analytical and numerical description of particles' trajectories, our algorithm allows an efficient tiling of a cosmological volume, where the dynamics within each tile is computed independently. As a consequence, the degree of parallelism is equal to the number of tiles. We optimise the accuracy of sCOLA by the use of a buffer region around tiles, and of appropriate Dirichlet boundary conditions around sCOLA boxes. As a result, we show that cosmological simulations at the degree of accuracy required for the analysis of the next generation of surveys can be run in drastically reduced wall-clock times and with very low memory requirements. The perfect scalability of our algorithm unlocks profoundly new possibilities of computing larger and higher-resolution cosmological simulations, taking advantage of a variety of hardware architectures.

We propose a new, likelihood-free approach to inferring the primordial matter power spectrum and cosmological parameters from arbitrarily complex forward models of galaxy surveys where all relevant statistics can be determined from numerical simulations, i.e. black-boxes. Our approach, which we call simulator expansion for likelihood-free inference (SELFI), builds upon approximate Bayesian computation using a novel effective likelihood, and upon the linearisation of black-box models around an expansion point. Consequently, we obtain simple "filter equations" for an effective posterior of the primordial power spectrum, and a straightforward scheme for cosmological parameter inference. We demonstrate that the workload is computationally tractable, fixed a priori, and perfectly parallel. As a proof of concept, we apply our framework to a realistic synthetic galaxy survey, with a data model accounting for physical structure formation and incomplete and noisy galaxy observations. In doing so, we show that the use of non-linear numerical models allows the galaxy power spectrum to be safely fitted up to at least $k_\mathrm{max} = 0.5$ $h$/Mpc, outperforming state-of-the-art backward-modelling techniques by a factor of $\sim 5$ in the number of modes used. The result is an unbiased inference of the primordial matter power spectrum across the entire range of scales considered, including a high-fidelity reconstruction of baryon acoustic oscillations. It translates into an unbiased and robust inference of cosmological parameters. Our results pave the path towards easy applications of likelihood-free simulation-based inference in cosmology. We have made our code pySELFI and our data products publicly available at http://pyselfi.florent-leclercq.eu.

Future cosmological galaxy surveys such as the Large Synoptic Survey Telescope (LSST) will photometrically observe very large numbers of galaxies. Without spectroscopy, the redshifts required for the analysis of these data will need to be inferred using photometric redshift techniques that are scalable to large sample sizes. The high number density of sources will also mean that around half are blended. We present a Bayesian photometric redshift method for blended sources that uses Gaussian mixture models to learn the joint flux-redshift distribution from a set of unblended training galaxies, and Bayesian model comparison to infer the number of galaxies comprising a blended source. The use of Gaussian mixture models renders both of these applications computationally efficient and therefore suitable for upcoming galaxy surveys.

We propose a method to probe the homogeneity of a general universe, without assuming symmetry. We show that isotropy can be tested at remote locations on the past lightcone by comparing the line-of-sight and transverse expansion rates, using the time dependence of the polarization of Cosmic Microwave Background photons that have been inverse-Compton scattered by the hot gas in massive clusters of galaxies. This probes a combination of remote transverse and parallel components of the expansion rate of the metric, and we may use radial baryon acoustic oscillations or cosmic clocks to measure the parallel expansion rate. Thus we can test remote isotropy, which is a key requirement of a homogeneous universe. We provide explicit formulas that connect observables and properties of the metric.

Pan-STARRS1 has carried out a set of distinct synoptic imaging sky surveys including the $3\pi$ Steradian Survey and the Medium Deep Survey in 5 bands ($grizy_{P1}$). The mean 5$\sigma$ point source limiting sensitivities in the stacked 3$\pi$ Steradian Survey in $grizy_{P1}$ are (23.3, 23.2, 23.1, 22.3, 21.4) respectively. The upper bound on the systematic uncertainty in the photometric calibration across the sky is 7-12 millimag depending on the bandpass. The systematic uncertainty of the astrometric calibration using the Gaia frame comes from a comparison of the results with Gaia: the standard deviation of the mean and median residuals ($ \Delta ra, \Delta dec $) are (2.3, 1.7) milliarcsec, and (3.1, 4.8) milliarcsec respectively. The Pan-STARRS system and the design of the PS1 surveys are described and an overview of the resulting image and catalog data products and their basic characteristics are described together with a summary of important results. The images, reduced data products, and derived data products from the Pan-STARRS1 surveys are available to the community from the Mikulski Archive for Space Telescopes (MAST) at STScI.

Cosmic Microwave Background experiments from COBE to Planck, have launched cosmology into an era of precision science, where many cosmological parameters are now determined to the percent level. Next generation telescopes, focussing on the cosmological 21cm signal from neutral hydrogen, will probe enormous volumes in the low-redshift Universe, and have the potential to determine dark energy properties and test modifications of Einstein's gravity. We study the 21cm bispectrum due to gravitational collapse as well as the contribution by line of sight perturbations in the form of the lensing-ISW bispectrum at low-redshifts ($z \sim 0.35-3$), targeted by upcoming neutral hydrogen intensity mapping experiments. We compute the expected bispectrum amplitudes and use a Fisher forecast model to compare power spectrum and bispectrum observations of intensity mapping surveys by CHIME, MeerKAT and SKA-mid. We find that combined power spectrum and bispectrum observations have the potential to decrease errors on the cosmological parameters by an order of magnitude compared to Planck. Finally, we compute the contribution of the lensing-ISW bispectrum, and find that, unlike for the cosmic microwave background analyses, it can safely be ignored for 21cm bispectrum observations.

Photometric redshifts are necessary for enabling large-scale multicolour galaxy surveys to interpret their data and constrain cosmological parameters. While the increased depth of future surveys such as the Large Synoptic Survey Telescope (LSST) will produce higher precision constraints, it will also increase the fraction of sources that are blended. In this paper, we present a Bayesian photometric redshift method for blended sources with an arbitrary number of intrinsic components. This method generalises existing template-based Bayesian photometric redshift (BPZ) methods, and produces joint posterior distributions for the component redshifts that allow uncertainties to be propagated in a principled way. Using Bayesian model comparison, we infer the probability that a source is blended and the number of components that it contains. We extend our formalism to the case where sources are blended in some bands and resolved in others. Applying this to the combination of LSST- and Euclid-like surveys, we find that the addition of resolved photometry results in a significant improvement in the reduction of outliers over the fully-blended case. We make available blendz, a Python implementation of our method.

We use Dataflow Engines (DFE) to construct an efficient Wiener filter of noisy and incomplete image data, and to quickly draw probabilistic samples of the compatible true underlying images from the Wiener posterior. Dataflow computing is a powerful approach using reconfigurable hardware, which can be deeply pipelined and is intrinsically parallel. The unique Wiener-filtered image is the minimum-variance linear estimate of the true image (if the signal and noise covariances are known) and the most probable true image (if the signal and noise are Gaussian distributed). However, many images are compatible with the data with different probabilities, given by the analytic posterior probability distribution referred to as the Wiener posterior. The DFE code also draws large numbers of samples of true images from this posterior, which allows for further statistical analysis. Naive computation of the Wiener-filtered image is impractical for large datasets, as it scales as $n^3$, where $n$ is the number of pixels. We use a messenger field algorithm, which is well suited to a DFE implementation, to draw samples from the Wiener posterior, that is, with the correct probability we draw samples of noiseless images that are compatible with the observed noisy image. The Wiener-filtered image can be obtained by a trivial modification of the algorithm. We demonstrate a lower bound on the speed-up, from drawing 10$^5$ samples of a 128$^2$ image, of 11.3 ${\pm}$ 0.8 with 8 DFEs in a 1U MPC-X box when compared with a 1U server presenting 32 CPU threads. We also discuss a potential application in astronomy, to provide better dark matter maps and improved determination of the parameters of the Universe.

Current cosmological constraints on the scalar spectral index of primordial fluctuations $n_{\rm s}$ in the $\Lambda$CDM model have excluded the minimal scale-invariant Harrison-Zel'dovich model ($n_{\rm s}=1$; hereafter HZ) at high significance, providing support for inflation. In recent years, however, some tensions have emerged between different cosmological datasets that, if not due to systematics, could indicate the presence of new physics beyond the $\Lambda$CDM model. In the light of these developments, we evaluate the Bayesian evidence against HZ in different data combinations and model extensions. Considering only the Planck temperature data, we find inconclusive evidence against HZ when including variations in the neutrino number $N_{\rm eff}$ and/or the Helium abundance $Y_{\rm He}$. Adding the Planck polarization data, on the other hand, yields strong evidence against HZ in the extensions we considered. Perhaps most interestingly, Planck temperature data combined with local measurements of the Hubble constant give as the most probable model an HZ spectrum, with additional neutrinos. However, with the inclusion of polarisation, standard $\Lambda$CDM is once again preferred, but the HZ model with extra neutrinos is not strongly disfavored. The possibility of fully ruling out the HZ spectrum is therefore ultimately connected with the solution to current tensions between cosmological datasets. If these tensions are confirmed by future data, then new physical mechanisms could be at work and an HZ spectrum could still offer a valid alternative.

Given the precision of current neutrino data, priors still impact noticeably the constraints on neutrino masses and their hierarchy. To avoid our understanding of neutrinos being driven by prior assumptions, we construct a prior that is mathematically minimally informative. Using the constructed uninformative prior, we find that the normal hierarchy is favoured but with inconclusive posterior odds of 5.1:1. Better data is hence needed before the neutrino masses and their hierarchy can be well constrained. We find that the next decade of cosmological data should provide conclusive evidence if the normal hierarchy with negligible minimum mass is correct, and if the uncertainty in the sum of neutrino masses drops below 0.025 eV. On the other hand, if neutrinos obey the inverted hierarchy, achieving strong evidence will be difficult with the same uncertainties. Our uninformative prior was constructed from principles of the Objective Bayesian approach. The prior is called a reference prior and is minimally informative in the specific sense that the information gain after collection of data is maximised. The prior is computed for the combination of neutrino oscillation data and cosmological data and still applies if the data improve.

We investigate whether a Gaussian likelihood, as routinely assumed in the analysis of cosmological data, is supported by simulated survey data. We define test statistics, based on a novel method that first destroys Gaussian correlations in a dataset, and then measures the non-Gaussian correlations that remain. This procedure flags pairs of datapoints which depend on each other in a non-Gaussian fashion, and thereby identifies where the assumption of a Gaussian likelihood breaks down. Using this diagnostic, we find that non-Gaussian correlations in the CFHTLenS cosmic shear correlation functions are significant. With a simple exclusion of the most contaminated datapoints, the posterior for $s_8$ is shifted without broadening, but we find no significant reduction in the tension with $s_8$ derived from Planck Cosmic Microwave Background data. However, we also show that the one-point distributions of the correlation statistics are noticeably skewed, such that sound weak lensing data sets are intrinsically likely to lead to a systematically low lensing amplitude being inferred. The detected non-Gaussianities get larger with increasing angular scale such that for future wide-angle surveys such as Euclid or LSST, with their very small statistical errors, the large-scale modes are expected to be increasingly affected. The shifts in posteriors may then not be negligible and we recommend that these diagnostic tests be run as part of future analyses.

Non-linear gravitational collapse introduces non-Gaussian statistics into the matter fields of the late Universe. As the large-scale structure is the target of current and future observational campaigns, one would ideally like to have the full probability density function of these non-Gaussian fields. The only viable way we see to achieve this analytically, at least approximately and in the near future, is via the Edgeworth expansion. We hence rederive this expansion for Fourier modes of non-Gaussian fields and then continue by putting it into a wider statistical context than previously done. We show that in its original form, the Edgeworth expansion only works if the non-Gaussian signal is averaged away. This is counterproductive, since we target the parameter-dependent non-Gaussianities as a signal of interest. We hence alter the analysis at the decisive step and now provide a roadmap towards a controlled and unadulterated analysis of non-Gaussianities in structure formation (with the Edgeworth expansion). Our central result is that, although the Edgeworth expansion has pathological properties, these can be predicted and avoided in a careful manner. We also show that, despite the non-Gaussianity coupling all modes, the Edgeworth series may be applied to any desired subset of modes, since this is equivalent (to the level of the approximation) to marginalising over the exlcuded modes. In this first paper of a series, we restrict ourselves to the sampling properties of the Edgeworth expansion, i.e.~how faithfully it reproduces the distribution of non-Gaussian data. A follow-up paper will detail its Bayesian use, when parameters are to be inferred.

We show how the massive data compression algorithm MOPED can be used to reduce, by orders of magnitude, the number of simulated datasets that are required to estimate the covariance matrix required for the analysis of gaussian-distributed data. This is relevant when the covariance matrix cannot be calculated directly. The compression is especially valuable when the covariance matrix varies with the model parameters. In this case, it may be prohibitively expensive to run enough simulations to estimate the full covariance matrix throughout the parameter space. This compression may be particularly valuable for the next-generation of weak lensing surveys, such as proposed for Euclid and LSST, for which the number of summary data (such as band power or shear correlation estimates) is very large, $\sim 10^4$, due to the large number of tomographic redshift bins that the data will be divided into. In the pessimistic case where the covariance matrix is estimated separately for all points in an MCMC analysis, this may require an unfeasible $10^9$ simulations. We show here that MOPED can reduce this number by a factor of 1000, or a factor of $\sim 10^6$ if some regularity in the covariance matrix is assumed, reducing the number of simulations required to a manageable $10^3$, making an otherwise intractable analysis feasible.

We compute the Bayesian Evidence for models considered in the main analysis of Planck cosmic microwave background data. By utilising carefully-defined nearest-neighbour distances in parameter space, we reuse the Monte Carlo Markov Chains already produced for parameter inference to compute Bayes factors $B$ for many different model-dataset combinations. Standard 6-parameter flat $\Lambda$CDM model is favoured over all other models considered, with curvature being mildly favoured only when CMB lensing is not included. Many alternative models are strongly disfavoured by the data, including primordial correlated isocurvature models ($\ln B=-7.8$), non-zero scalar-to-tensor ratio ($\ln B=-4.3$), running of the spectral index ($\ln B = -4.7$), curvature ($\ln B=-3.6$), non-standard numbers of neutrinos ($\ln B=-3.1$), non-standard neutrino masses ($\ln B=-3.2$), non-standard lensing potential ($\ln B=-4.6$), evolving dark energy ($\ln B=-3.2$), sterile neutrinos ($\ln B=-6.9$), and extra sterile neutrinos with a non-zero scalar-to-tensor ratio ($\ln B=-10.8$). Other models are less strongly disfavoured with respect to flat $\Lambda$CDM. As with all analyses based on Bayesian Evidence, the final numbers depend on the widths of the parameter priors. We adopt the priors used in the Planck analysis, while performing a prior sensitivity analysis. Our quantitative conclusion is that extensions beyond the standard cosmological model are disfavoured by Planck data. Only when newer Hubble constant measurements are included does $\Lambda$CDM become disfavoured, and only mildly, compared with a dynamical dark energy model ($\ln B\sim +2$).

We compute the power spectrum at one-loop order in standard perturbation theory for the matter density field to which a standard Lagrangian Baryonic acoustic oscillation (BAO) reconstruction technique is applied. The BAO reconstruction method corrects the bulk motion associated with the gravitational evolution using the inverse Zel'dovich approximation (ZA) for the smoothed density field. We find that the overall amplitude of one-loop contributions in the matter power spectrum substantially decrease after reconstruction. The reconstructed power spectrum thereby approaches the initial linear spectrum when the smoothed density field is close enough to linear, i.e., the smoothing scale $R_s$ larger than around 10$h^{-1}$Mpc. On smaller $R_s$,however, the deviation from the linear spectrum becomes significant on large scales ($k\lt R_s^{-1}$) due to the nonlinearity in the smoothed density field, and the reconstruction is inaccurate. Compared with N-body simulations, we show that the reconstructed power spectrum at one loop order agrees with simulations better than the unreconstructed power spectrum. We also calculate the tree-level bispectrum in standard perturbation theory to investigate non-Gaussianity in the reconstructed matter density field. We show that the amplitude of the bispectrum significantly decreases for small $k$ after reconstruction and that the tree-level bispectrum agrees well with N-body results in the weakly nonlinear regime.

Einstein's theory of General Relativity implies that energy, i.e. matter, curves space-time and thus deforms lightlike geodesics, giving rise to gravitational lensing. This phenomenon is well understood in the case of the Schwarzschild metric, and has been accurately described in the past; however, lensing in the Kerr space-time has received less attention in the literature despite potential practical observational applications. In particular, lensing in such space is not expressible as the gradient of a scalar potential and as such is a source of curl-like signatures and an asymmetric shear pattern. In this paper, we develop a differentiable lensing map in the Kerr metric, reworking and extending previous approaches. By using standard tools of weak gravitational lensing, we isolate and quantify the distortion that is uniquely induced by the presence of angular momentum in the metric. We apply this framework to the distortion induced by a Kerr-like foreground object on a distribution of background of sources. We verify that the new unique lensing signature is orders of magnitude below current observational bounds for a range of lens configurations.

We apply two Bayesian hierarchical inference schemes to infer shear power spectra, shear maps and cosmological parameters from the CFHTLenS weak lensing survey - the first application of this method to data. In the first approach, we sample the joint posterior distribution of the shear maps and power spectra by Gibbs sampling, with minimal model assumptions. In the second approach, we sample the joint posterior of the shear maps and cosmological parameters, providing a new, accurate and principled approach to cosmological parameter inference from cosmic shear data. As a first demonstration on data we perform a 2-bin tomographic analysis to constrain cosmological parameters and investigate the possibility of photometric redshift bias in the CFHTLenS data. Under the baseline $\Lambda$CDM model we constrain $S_8 = \sigma_8(\Omega_\mathrm{m}/0.3)^{0.5} = 0.67 ^{\scriptscriptstyle+ 0.03 }_{\scriptscriptstyle- 0.03 }$ $(68\%)$, consistent with previous CFHTLenS analysis but in tension with Planck. Adding neutrino mass as a free parameter we are able to constrain $\sum m_\nu < 4.6\mathrm{eV}$ (95%) using CFHTLenS data alone. Including a linear redshift dependent photo-$z$ bias $\Delta z = p_2(z - p_1)$, we find $p_1=-0.25 ^{\scriptscriptstyle+ 0.53 }_{\scriptscriptstyle- 0.60 }$ and $p_2 = -0.15 ^{\scriptscriptstyle+ 0.17 }_{\scriptscriptstyle- 0.15 }$, and tension with Planck is only alleviated under very conservative prior assumptions. Neither the non-minimal neutrino mass or photo-$z$ bias models are significantly preferred by the CFHTLenS (2-bin tomography) data.

In this paper we discuss the commonly-used limiting cases, or approximations, for two-point cosmic shear statistics. We discuss the most prominent assumptions in this statistic: the flat-sky (small angle limit), the Limber (Bessel-to-delta function limit) and the Hankel transform (large l-mode limit) approximations; that the vast majority of cosmic shear results to date have used simultaneously. We find that the combined effect of these approximations can suppress power by >1% on scales of l<40. A fully non-approximated cosmic shear study should use a spherical-sky, non-Limber-approximated power spectrum analysis; and a transform involving Wigner small-d matrices in place of the Hankel transform. These effects, unaccounted for, would constitute at least 11% of the total budget for systematic effects for a power spectrum analysis of a Euclid-like experiment; but they are unnecessary.

In this paper, we present a method for computing the marginal likelihood, also known as the model likelihood or Bayesian evidence, from Markov Chain Monte Carlo (MCMC), or other sampled posterior distributions. In order to do this, one needs to be able to estimate the density of points in parameter space, and this can be challenging in high numbers of dimensions. Here we present a Bayesian analysis, where we obtain the posterior for the marginal likelihood, using $k$th nearest-neighbour distances in parameter space, using the Mahalanobis distance metric, under the assumption that the points in the chain (thinned if required) are independent. We generalise the algorithm to apply to importance-sampled chains, where each point is assigned a weight. We illustrate this with an idealised posterior of known form with an analytic marginal likelihood, and show that for chains of length $\sim 10^5$ points, the technique is effective for parameter spaces with up to $\sim 20$ dimensions. We also argue that $k=1$ is the optimal choice, and discuss failure modes for the algorithm. In a companion paper (Heavens et al. 2017) we apply the technique to the main MCMC chains from the 2015 Planck analysis of cosmic background radiation data, to infer that quantitatively the simplest 6-parameter flat $\Lambda$CDM standard model of cosmology is preferred over all extensions considered.

Parameter inference with an estimated covariance matrix systematically loses information due to the remaining uncertainty of the covariance matrix. Here, we quantify this loss of precision and develop a framework to hypothetically restore it, which allows to judge how far away a given analysis is from the ideal case of a known covariance matrix. We point out that it is insufficient to estimate this loss by debiasing a Fisher matrix as previously done, due to a fundamental inequality that describes how biases arise in non-linear functions. We therefore develop direct estimators for parameter credibility contours and the figure of merit. We apply our results to DES Science Verification weak lensing data, detecting a 10% loss of information that increases their credibility contours. No significant loss of information is found for KiDS. For a Euclid-like survey, with about 10 nuisance parameters we find that 2900 simulations are sufficient to limit the systematically lost information to 1%, with an additional uncertainty of about 2%. Without any nuisance parameters 1900 simulations are sufficient to only lose 1% of information. We also derive an estimator for the Fisher matrix of the unknown true covariance matrix, two estimators of its inverse with different physical meanings, and an estimator for the optimally achievable figure of merit. The formalism here quantifies the gains to be made by running more simulated datasets, allowing decisions to be made about numbers of simulations in an informed way.

Measurements of the power spectrum from large-scale structure surveys have to date assumed an equal-time approximation, where the full cross-correlation power spectrum of the matter density field evaluated at different times (or distances) has been approximated either by the power spectrum at a fixed time, or in an improved fashion, by a geometric mean $P(k; r_1, r_2)=[P(k; r_1) P(k; r_2)]^{1/2}$. In this paper we investigate the expected impact of the geometric mean ansatz, and present an application in assessing the impact on weak gravitational lensing cosmological parameter inference, using a perturbative unequal-time correlator. As one might expect, we find that the impact of this assumption is greatest at large separations in redshift $\Delta z > 0.3$ where the change in the amplitude of the matter power spectrum can be as much as $10$ percent for $k > 5h$Mpc$^{-1}$. However, of more concern is that the corrections for small separations, where the clustering is not close to zero, may not be negligibly small. In particular, we find that for a Euclid- or LSST-like weak lensing experiment the assumption of equal-time correlators may result in biased predictions of the cosmic shear power spectrum, and that the impact is strongly dependent on the amplitude of the intrinsic alignment signal. To compute unequal-time correlations to sufficient accuracy will require advances in either perturbation theory to high $k$-modes, or extensive use of simulations.

We investigate our knowledge of early universe cosmology by exploring how much additional energy density can be placed in different components beyond those in the $\Lambda$CDM model. To do this we use a method to separate early- and late-universe information enclosed in observational data, thus markedly reducing the model-dependency of the conclusions. We find that the 95\% credibility regions for extra energy components of the early universe at recombination are: non-accelerating additional fluid density parameter $\Omega_{\rm MR} < 0.006$ and extra radiation parameterised as extra effective neutrino species $2.3 < N_{\rm eff} < 3.2$ when imposing flatness. Our constraints thus show that even when analyzing the data in this largely model-independent way, the possibility of hiding extra energy components beyond $\Lambda$CDM in the early universe is seriously constrained by current observations. We also find that the standard ruler, the sound horizon at radiation drag, can be well determined in a way that does not depend on late-time Universe assumptions, but depends strongly on early-time physics and in particular on additional components that behave like radiation. We find that the standard ruler length determined in this way is $r_{\rm s} = 147.4 \pm 0.7$ Mpc if the radiation and neutrino components are standard, but the uncertainty increases by an order of magnitude when non-standard dark radiation components are allowed, to $r_{\rm s} = 150 \pm 5$ Mpc.

The Planck full mission cosmic microwave background(CMB) temperature and E-mode polarization maps are analysed to obtain constraints on primordial non-Gaussianity(NG). Using three classes of optimal bispectrum estimators - separable template-fitting (KSW), binned, and modal - we obtain consistent values for the local, equilateral, and orthogonal bispectrum amplitudes, quoting as our final result from temperature alone fNL^local=2.5+\-5.7, fNL^equil=-16+\-70 and fNL^ortho=-34+\-33(68%CL). Combining temperature and polarization data we obtain fNL^local=0.8+\-5.0, fNL^equil=-4+\-43 and fNL^ortho=-26+\-21 (68%CL). The results are based on cross-validation of these estimators on simulations, are stable across component separation techniques, pass an extensive suite of tests, and are consistent with Minkowski functionals based measurements. The effect of time-domain de-glitching systematics on the bispectrum is negligible. In spite of these test outcomes we conservatively label the results including polarization data as preliminary, owing to a known mismatch of the noise model in simulations and the data. Beyond fNL estimates, we present model-independent reconstructions of the CMB bispectrum and derive constraints on early universe scenarios that generate NG, including general single-field and axion inflation, initial state modifications, parity-violating tensor bispectra, and directionally dependent vector models. We also present a wide survey of scale-dependent oscillatory bispectra, and we look for isocurvature NG. Our constraint on the local primordial trispectrum amplitude is gNL^local=(-9.0+\-7.7)x10^4 (68%CL), and we perform an analysis of additional trispectrum shapes. The global picture is one of consistency with the premises of the LambdaCDM cosmology, namely that the structure we observe today was sourced by adiabatic, passive, Gaussian, and primordial seed perturbations.[abridged]

Assuming the existence of standard rulers, standard candles and standard clocks, requiring only the cosmological principle, a metric theory of gravity, a smooth expansion history, and using state-of-the-art observations, we determine the length of the "low-redshift standard ruler". The data we use are a compilation of recent Baryon acoustic oscillation data (relying on the standard ruler), Type 1A supernov\ae\ (as standard candles), ages of early type galaxies (as standard clocks) and local determinations of the Hubble constant (as a local anchor of the cosmic distance scale). In a standard $\Lambda$CDM cosmology the "low-redshift standard ruler" coincides with the sound horizon at radiation drag, which can also be determined --in a model dependent way-- from CMB observations. However, in general, the two quantities need not coincide. We obtain constraints on the length of the low-redshift standard ruler: $r^h_{\rm s}=101.0 \pm 2.3 h^{-1}$ Mpc, when using only Type 1A supernov\ae\ and Baryon acoustic oscillations, and $r_{\rm s}=150.0\pm 4.7 $ Mpc when using clocks to set the Hubble normalisation, while $r_{\rm s}=141.0\pm 5.5 $ Mpc when using the local Hubble constant determination (using both yields $r_{\rm s}=143.9\pm 3.1 $ Mpc). The low-redshift determination of the standard ruler has an error which is competitive with the model-dependent determination from cosmic microwave background measurements made with the {\em Planck} satellite, which assumes it is the sound horizon at the end of baryon drag.

Fisher matrices play an important role in experimental design and in data analysis. Their primary role is to make predictions for the inference of model parameters - both their errors and covariances. In this short review, I outline a number of extensions to the simple Fisher matrix formalism, covering a number of recent developments in the field. These are: (a) situations where the data (in the form of (x,y) pairs) have errors in both x and y; (b) modifications to parameter inference in the presence of systematic errors, or through fixing the values of some model parameters; (c) Derivative Approximation for LIkelihoods (DALI) - higher-order expansions of the likelihood surface, going beyond the Gaussian shape approximation; (d) extensions of the Fisher-like formalism, to treat model selection problems with Bayesian evidence.

Euclid is a European Space Agency medium class mission selected for launch in 2020 within the Cosmic Vision 2015 2025 program. The main goal of Euclid is to understand the origin of the accelerated expansion of the universe. Euclid will explore the expansion history of the universe and the evolution of cosmic structures by measuring shapes and redshifts of galaxies as well as the distribution of clusters of galaxies over a large fraction of the sky. Although the main driver for Euclid is the nature of dark energy, Euclid science covers a vast range of topics, from cosmology to galaxy evolution to planetary research. In this review we focus on cosmology and fundamental physics, with a strong emphasis on science beyond the current standard models. We discuss five broad topics: dark energy and modified gravity, dark matter, initial conditions, basic assumptions and questions of methodology in the data analysis. This review has been planned and carried out within Euclid's Theory Working Group and is meant to provide a guide to the scientific themes that will underlie the activity of the group during the preparation of the Euclid mission.

We present an optimized variant of the halo model, designed to produce accurate matter power spectra well into the non-linear regime for a wide range of cosmological models. To do this, we introduce physically motivated free parameters into the halo-model formalism and fit these to data from high-resolution $N$-body simulations. For a variety of $\Lambda$CDM and $w$CDM models the halo-model power is accurate to $\simeq 5$ per cent for $k\leq 10h\,\mathrm{Mpc}$ and $z\leq 2$. An advantage of our new halo model is that it can be adapted to account for the effects of baryonic feedback on the power spectrum. We demonstrate this by fitting the halo model to power spectra from the OWLS hydrodynamical simulation suite via parameters that govern halo internal structure. We are able to fit all feedback models investigated at the 5 per cent level using only two free parameters, and we place limits on the range of these halo parameters for feedback models investigated by the OWLS simulations. Accurate predictions to high-$k$ are vital for weak lensing surveys, and these halo parameters could be considered nuisance parameters to marginalize over in future analyses to mitigate uncertainty regarding the details of feedback. Finally, we investigate how lensing observables predicted by our model compare to those from simulations and from HALOFIT for a range of $k$-cuts and feedback models and quantify the angular scales at which these effects become important. Code to calculate power spectra from the model presented in this paper can be found at https://github.com/alexander-mead/hmcode

We study the implications of Planck data for models of dark energy (DE) and modified gravity (MG), beyond the cosmological constant scenario. We start with cases where the DE only directly affects the background evolution, considering Taylor expansions of the equation of state, principal component analysis and parameterizations related to the potential of a minimally coupled DE scalar field. When estimating the density of DE at early times, we significantly improve present constraints. We then move to general parameterizations of the DE or MG perturbations that encompass both effective field theories and the phenomenology of gravitational potentials in MG models. Lastly, we test a range of specific models, such as k-essence, f(R) theories and coupled DE. In addition to the latest Planck data, for our main analyses we use baryonic acoustic oscillations, type-Ia supernovae and local measurements of the Hubble constant. We further show the impact of measurements of the cosmological perturbations, such as redshift-space distortions and weak gravitational lensing. These additional probes are important tools for testing MG models and for breaking degeneracies that are still present in the combination of Planck and background data sets. All results that include only background parameterizations are in agreement with LCDM. When testing models that also change perturbations (even when the background is fixed to LCDM), some tensions appear in a few scenarios: the maximum one found is \sim 2 sigma for Planck TT+lowP when parameterizing observables related to the gravitational potentials with a chosen time dependence; the tension increases to at most 3 sigma when external data sets are included. It however disappears when including CMB lensing.

There is currently a discrepancy in the measured value of the amplitude of matter clustering, parameterised using $\sigma_8$, inferred from galaxy weak lensing, and cosmic microwave background data, which could be an indication of new physics, such as massive neutrinos or a modification to the gravity law, or baryon feedback. In this paper we make the assumption that the cosmological parameters are well determined by Planck, and use weak lensing data to investigate the implications for baryon feedback and massive neutrinos, as well as possible contributions from intrinsic alignments and biases in photometric redshifts. We apply a non-parametric approach to model the baryonic feedback on the dark matter clustering, which is flexible enough to reproduce the OWLS and Illustris simulation results. The statistic we use, 3D cosmic shear, is a method that extracts cosmological information from weak lensing data using a spherical-Bessel function power spectrum approach. We analyse the CFHTLenS weak lensing data and, assuming best fit cosmological parameters from the Planck CMB experiment, find that there is no evidence for baryonic feedback on the dark matter power spectrum, but there is evidence for a bias in the photometric redshifts in the CFHTLenS data, consistent with a completely independent analysis by Choi et al. (2015), based on spectroscopic redshifts; and that these conclusions are robust to assumptions about the intrinsic alignment systematic. We also find an upper limit on the sum of neutrino masses conditional on other $\Lambda$CDM parameters being fixed, of $< 0.28$ eV ($1\sigma$).

Despite its continued observational successes, there is a persistent (and growing) interest in extending cosmology beyond the standard model, $\Lambda$CDM. This is motivated by a range of apparently serious theoretical issues, involving such questions as the cosmological constant problem, the particle nature of dark matter, the validity of general relativity on large scales, the existence of anomalies in the CMB and on small scales, and the predictivity and testability of the inflationary paradigm. In this paper, we summarize the current status of $\Lambda$CDM as a physical theory, and review investigations into possible alternatives along a number of different lines, with a particular focus on highlighting the most promising directions. While the fundamental problems are proving reluctant to yield, the study of alternative cosmologies has led to considerable progress, with much more to come if hopes about forthcoming high-precision observations and new theoretical ideas are fulfilled.

To accomplish correct Bayesian inference from weak lensing shear data requires a complete statistical description of the data. The natural framework to do this is a Bayesian Hierarchical Model, which divides the chain of reasoning into component steps. Starting with a catalogue of shear estimates in tomographic bins, we build a model that allows us to sample simultaneously from the the underlying tomographic shear fields and the relevant power spectra (E-mode, B-mode, and E-B, for auto- and cross-power spectra). The procedure deals easily with masked data and intrinsic alignments. Using Gibbs sampling and messenger fields, we show with simulated data that the large (over 67000-)dimensional parameter space can be efficiently sampled and the full joint posterior probability density function for the parameters can feasibly be obtained. The method correctly recovers the underlying shear fields and all of the power spectra, including at levels well below the shot noise.

Lensing of the Cosmic Microwave Background (CMB) changes the morphology of pattern of temperature fluctuations, so topological descriptors such as Minkowski Functionals can probe the gravity model responsible for the lensing. We show how the recently introduced two-to-two and three-to-one kurt-spectra (and their associated correlation functions), which depend on the power spectrum of the lensing potential, can be used to probe modified gravity theories such as $f({R})$ theories of gravity and quintessence models. We also investigate models based on effective field theory, which include the constant-$\Omega$ model, and low-energy Ho\vrava theories. Estimates of the cumulative signal-to-noise for detection of lensing-induced morphology changes, reaches ${\cal O}(10^3)$ for the future planned CMB polarization mission COrE$^{+}$. Assuming foreground removal is possible to $\ell_{max}=3000$, we show that many modified gravity theories can be rejected with a high level of significance, making this technique comparable in power to galaxy weak lensing or redshift surveys. These topological estimators are also useful in distinguishing {\em lensing} from other scattering secondaries at the level of the four-point function or trispectrum. Examples include the kinetic Sunyaev-Zel'dovich (kSZ) effect which shares, with lensing, a lack of spectral distortion. We also discuss the complication of foreground contamination from unsubtracted point sources.

We present the first measurement of individual cluster mass estimates using weak lensing size and flux magnification. Using data from the HST-STAGES survey of the A901/902 supercluster we detect the four known groups in the supercluster at high significance using magnification alone. We discuss the application of a fully Bayesian inference analysis, and investigate a broad range of potential systematics in the application of the method. We compare our results to a previous weak lensing shear analysis of the same field finding the recovered signal-to-noise of our magnification-only analysis to range from 45% to 110% of the signal-to-noise in the shear-only analysis. On a case-by-case basis we find consistent magnification and shear constraints on cluster virial radius, and finding that for the full sample, magnification constraints to be a factor $0.77 \pm 0.18$ lower than the shear measurements.

We develop a Bayesian hierarchical modelling approach for cosmic shear power spectrum inference, jointly sampling from the posterior distribution of the cosmic shear field and its (tomographic) power spectra. Inference of the shear power spectrum is a powerful intermediate product for a cosmic shear analysis, since it requires very few model assumptions and can be used to perform inference on a wide range of cosmological models \emph{a posteriori} without loss of information. We show that joint posterior for the shear map and power spectrum can be sampled effectively by Gibbs sampling, iteratively drawing samples from the map and power spectrum, each conditional on the other. This approach neatly circumvents difficulties associated with complicated survey geometry and masks that plague frequentist power spectrum estimators, since the power spectrum inference provides prior information about the field in masked regions at every sampling step. We demonstrate this approach for inference of tomographic shear $E$-mode, $B$-mode and $EB$-cross power spectra from a simulated galaxy shear catalogue with a number of important features; galaxies distributed on the sky and in redshift with photometric redshift uncertainties, realistic random ellipticity noise for every galaxy and a complicated survey mask. The obtained posterior distributions for the tomographic power spectrum coefficients recover the underlying simulated power spectra for both $E$- and $B$-modes.

When inferring parameters from a Gaussian-distributed data set by computing a likelihood, a covariance matrix is needed that describes the data errors and their correlations. If the covariance matrix is not known a priori, it may be estimated and thereby becomes a random object with some intrinsic uncertainty itself. We show how to infer parameters in the presence of such an estimated covariance matrix, by marginalising over the true covariance matrix, conditioned on its estimated value. This leads to a likelihood function that is no longer Gaussian, but rather an adapted version of a multivariate t-distribution, which has the same numerical complexity as the multivariate Gaussian. As expected, marginalisation over the true covariance matrix improves inference when compared with Hartlap et al.'s method, which uses an unbiased estimate of the inverse covariance matrix but still assumes that the likelihood is Gaussian.

We present the first cosmological measurement derived from a galaxy density field subject to a `clipping' transformation. By enforcing an upper bound on the galaxy number density field in the Galaxy and Mass Assembly survey (GAMA), contributions from the nonlinear processes of virialisation and galaxy bias are greatly reduced. This leads to a galaxy power spectrum which is easier to model, without calibration from numerical simulations. We develop a theoretical model for the power spectrum of a clipped field in redshift space, which is exact for the case of anisotropic Gaussian fields. Clipping is found to extend the applicability of the conventional Kaiser prescription by more than a factor of three in wavenumber, or a factor of thirty in terms of the number of Fourier modes. By modelling the galaxy power spectrum on scales k < 0.3 h/Mpc and density fluctuations $\delta_g < 4$ we measure the normalised growth rate $f\sigma_8(z = 0.18) = 0.29 \pm 0.10$.

Euclid is a European Space Agency medium class mission selected for launch in 2019 within the Cosmic Vision 2015-2025 programme. The main goal of Euclid is to understand the origin of the accelerated expansion of the Universe. Euclid will explore the expansion history of the Universe and the evolution of cosmic structures by measuring shapes and redshifts of galaxies as well as the distribution of clusters of galaxies over a large fraction of the sky. Although the main driver for Euclid is the nature of dark energy, Euclid science covers a vast range of topics, from cosmology to galaxy evolution to planetary research. In this review we focus on cosmology and fundamental physics, with a strong emphasis on science beyond the current standard models. We discuss five broad topics: dark energy and modified gravity, dark matter, initial conditions, basic assumptions and questions of methodology in the data analysis. This review has been planned and carried out within Euclid's Theory Working Group and is meant to provide a guide to the scientific themes that will underlie the activity of the group during the preparation of the Euclid mission.

Weak lensing can be observed through a number of effects on the images of distant galaxies; their shapes are sheared, their sizes and fluxes (magnitudes) are magnified and their positions on the sky are modified by the lensing field. Galaxy shapes probe the shear field whilst size, magnitude and number density probe the convergence field. Both contain cosmological information. In this paper we are concerned with the magnification of the size and magnitude of individual galaxies as a probe of cosmic convergence. We develop a Bayesian approach for inferring the convergence field from a measured size, magnitude and redshift and demonstrate that the inference on convergence requires detailed knowledge of the joint distribution of intrinsic sizes and magnitudes. We build a simple parameterised model for the size-magnitude distribution and estimate this distribution for CFHTLenS galaxies. In light of the measured distribution, we show that the typical dispersion on convergence estimation is ~0.8, compared to ~0.38 for shear. We discuss the possibility of physical systematics for magnification (similar to intrinsic alignments for shear) and compute the expected gains in the Dark Energy Figure-of-Merit (FoM) from combining magnification with shear for different scenarios regarding systematics: when accounting for intrinsic alignments but no systematics on the magnification signal, including magnification could improve the FoM by upto a factor of ~2.5, whilst when accounting for physical systematics in both shear and magnification we anticipate a gain between ~25% and ~65%. In addition to the statistical gains, the fact that cosmic shear and magnification are subject to different systematics makes magnification an attractive complement to any cosmic shear analysis.

The European Space Agency's Planck satellite, dedicated to studying the early Universe and its subsequent evolution, was launched 14~May 2009 and scanned the microwave and submillimetre sky continuously between 12~August 2009 and 23~October 2013. In February~2015, ESA and the Planck Collaboration released the second set of cosmology products based on data from the entire Planck mission, including both temperature and polarization, along with a set of scientific and technical papers and a web-based explanatory supplement. This paper gives an overview of the main characteristics of the data and the data products in the release, as well as the associated cosmological and astrophysical science results and papers. The science products include maps of the cosmic microwave background (CMB), the thermal Sunyaev-Zeldovich effect, and diffuse foregrounds in temperature and polarization, catalogues of compact Galactic and extragalactic sources (including separate catalogues of Sunyaev-Zeldovich clusters and Galactic cold clumps), and extensive simulations of signals and noise used in assessing the performance of the analysis methods and assessment of uncertainties. The likelihood code used to assess cosmological models against the Planck data are described, as well as a CMB lensing likelihood. Scientific results include cosmological parameters deriving from CMB power spectra, gravitational lensing, and cluster counts, as well as constraints on inflation, non-Gaussianity, primordial magnetic fields, dark energy, and modified gravity.

In this paper we present a power spectrum formalism that combines the full three-dimensional information from the galaxy ellipticity field, with information from the cosmic microwave background (CMB). We include in this approach galaxy cosmic shear and galaxy intrinsic alignments, CMB deflection, CMB temperature and CMB polarisation data; including the inter-datum power spectra between all quantities. We apply this to forecasting cosmological parameter errors for CMB and imaging surveys for Euclid-like, Planck, ACTPoL, and CoRE-like experiments. We show that the additional covariance between the CMB and ellipticity measurements can improve dark energy equation of state measurements by 15%, and the combination of cosmic shear and the CMB, from Euclid-like and CoRE-like experiments, could in principle measure the sum of neutrino masses with an error of 0.003 eV.

This paper presents the first application of 3D cosmic shear to a wide-field weak lensing survey. 3D cosmic shear is a technique that analyses weak lensing in three dimensions using a spherical harmonic approach, and does not bin data in the redshift direction. This is applied to CFHTLenS, a 154 square degree imaging survey with a median redshift of 0.7 and an effective number density of 11 galaxies per square arcminute usable for weak lensing. To account for survey masks we apply a 3D pseudo-Cl approach on weak lensing data, and to avoid uncertainties in the highly non-linear regime, we separately analyse radial wave numbers k<=1.5h/Mpc and k<=5.0h/Mpc, and angular wavenumbers l~400-5000. We show how one can recover 2D and tomographic power spectra from the full 3D cosmic shear power spectra and present a measurement of the 2D cosmic shear power spectrum, and measurements of a set of 2-bin and 6-bin cosmic shear tomographic power spectra; in doing so we find that using the 3D power in the calculation of such 2D and tomographic power spectra from data naturally accounts for a minimum scale in the matter power spectrum. We use 3D cosmic shear to constrain cosmologies with parameters OmegaM, OmegaB, sigma8, h, ns, w0, wa. For a non-evolving dark energy equation of state, and assuming a flat cosmology, lensing combined with WMAP7 results in h=0.78+/-0.12, OmegaM=0.252+/-0.079, sigma8=0.88+/-0.23 and w=-1.16+/-0.38 using only scales k<=1.5h/Mpc. We also present results of lensing combined with first year Planck results, where we find no tension with the results from this analysis, but we also find no significant improvement over the Planck results alone. We find evidence of a suppression of power compared to LCDM on small scales 1.5 < k < 5.0 h/Mpc in the lensing data, which is consistent with predictions of the effect of baryonic feedback on the matter power spectrum.

We measure the length of the Baryon Acoustic Oscillation (BAO) feature, and the expansion rate of the recent Universe, from low-redshift data only, almost model-independently. We make only the following minimal assumptions: homogeneity and isotropy; a metric theory of gravity; a smooth expansion history, and the existence of standard candles (supernov\ae) and a standard BAO ruler. The rest is determined by the data, which are compilations of recent BAO and Type IA supernova results. Making only these assumptions, we find for the first time that the standard ruler has length $103.9 \pm 2.3\, h^{-1}$ Mpc. The value is a measurement, in contrast to the model-dependent theoretical prediction determined with model parameters set by Planck data ($99.3 \pm 2.1 \, h^{-1}$ Mpc). The latter assumes $\Lambda$CDM, and that the ruler is the sound horizon at radiation drag. Adding passive galaxies as standard clocks or a local Hubble constant measurement allows the absolute BAO scale to be determined ($142.8\pm 3.7$ Mpc), and in the former case the additional information makes the BAO length determination more precise ($101.9\pm 1.9 \, h^{-1}\,$Mpc). The inverse curvature radius of the Universe is weakly constrained and consistent with zero, independently of the gravity model, provided it is metric. We find the effective number of relativistic species to be $N_{\rm eff} = 3.53\pm 0.32$, independent of late-time dark energy or gravity physics.

The Fisher Information Matrix formalism is extended to cases where the data is divided into two parts (X,Y), where the expectation value of Y depends on X according to some theoretical model, and X and Y both have errors with arbitrary covariance. In the simplest case, (X,Y) represent data pairs of abscissa and ordinate, in which case the analysis deals with the case of data pairs with errors in both coordinates, but X can be any measured quantities on which Y depends. The analysis applies for arbitrary covariance, provided all errors are gaussian, and provided the errors in X are small, both in comparison with the scale over which the expected signal Y changes, and with the width of the prior distribution. This generalises the Fisher Matrix approach, which normally only considers errors in the `ordinate' Y. In this work, we include errors in X by marginalising over latent variables, effectively employing a Bayesian hierarchical model, and deriving the Fisher Matrix for this more general case. The methods here also extend to likelihood surfaces which are not gaussian in the parameter space, and so techniques such as DALI (Derivative Approximation for Likelihoods) can be generalised straightforwardly to include arbitrary gaussian data error covariances. For simple mock data and theoretical models, we compare to Markov Chain Monte Carlo experiments, illustrating the method with cosmological supernova data. We also include the new method in the Fisher4Cast software.

The ESA's Planck satellite, dedicated to studying the early Universe and its subsequent evolution, was launched 14 May 2009 and has been scanning the microwave and submillimetre sky continuously since 12 August 2009. This paper gives an overview of the mission and its performance, the processing, analysis, and characteristics of the data, the scientific results, and the science data products and papers in the release. The science products include maps of the CMB and diffuse extragalactic foregrounds, a catalogue of compact Galactic and extragalactic sources, and a list of sources detected through the SZ effect. The likelihood code used to assess cosmological models against the Planck data and a lensing likelihood are described. Scientific results include robust support for the standard six-parameter LCDM model of cosmology and improved measurements of its parameters, including a highly significant deviation from scale invariance of the primordial power spectrum. The Planck values for these parameters and others derived from them are significantly different from those previously determined. Several large-scale anomalies in the temperature distribution of the CMB, first detected by WMAP, are confirmed with higher confidence. Planck sets new limits on the number and mass of neutrinos, and has measured gravitational lensing of CMB anisotropies at greater than 25 sigma. Planck finds no evidence for non-Gaussianity in the CMB. Planck's results agree well with results from the measurements of baryon acoustic oscillations. Planck finds a lower Hubble constant than found in some more local measures. Some tension is also present between the amplitude of matter fluctuations derived from CMB data and that derived from SZ data. The Planck and WMAP power spectra are offset from each other by an average level of about 2% around the first acoustic peak.

We show how cross-correlating a high redshift external tracer field, such as the 21cm neutral hydrogen distribution and product maps involving Cosmic Microwave Background (CMB) temperature and polarisation fields, that probe mixed bispectrum involving these fields, can help to determine the reionization history of the Universe, beyond what can be achieved from cross-spectrum analysis. Taking clues from recent studies for the detection of primordial non-Gaussianity (Munshi & Heavens 2010), we develop a set of estimators that can study reionization using a power spectrum associated with the bispectrum (or skew-spectrum). We use the matched filtering inherent in this method to investigate different reionization histories. We check to what extent they can be used to rule out various models of reionization and study cross contamination from different sources such as the lensing of the CMB. The estimators can be fine-tuned to optimize study of a specific reionization history. We consider three different types of tracers in our study, namely: proto-galaxies; 21cm maps of neutral hydrogen; quasars. We also consider four alternative models of reionization. We find that the cumulative signal-to-noise (S/N) for detection at $\ell_{max}=2000$ can reach $\mathcal{O}(70)$ for cosmic variance limited all-sky experiments. Combining 100GHz, 143GHz and 217GHz channels of the Planck experiment, we find that the (S/N) lies in the range $\mathcal{O}(5)-\mathcal{O}(35)$. The (S/N) depends on the specific choice of a tracer field, and multiple tracers can be effectively used to map out the entire reionization history with reasonable S/N. Contamination from weak lensing is investigated and found to be negligible, and the effects of Thomson scattering from patchy reionizationare also considered.

We use the optimised skew-spectrum as well as the skew-spectra associated with the Minkowski Functionals (MFs) to test the possibility of using the cross-correlation of the Integrated Sachs-Wolfe effect (ISW) and lensing of the cosmic microwave background (CMB) radiation to detect deviations in the theory of gravity away from General Relativity (GR). We find that the although both statistics can put constraints on modified gravity, the optimised skew-spectra are especially sensitive to the parameter $\rm B_0$ that denotes the the {\em Compton wavelength} of the scalaron at the present epoch. We investigate three modified gravity theories, namely: the Post-Parametrised Friedmanian (PPF) formalism; the Hu-Sawicki (HS) model; and the Bertschinger-Zukin (BZ) formalism. Employing a likelihood analysis for an experimental setup similar to ESA's Planck mission, we find that, assuming GR to be the correct model, we expect the constraints from the first two skew-spectra, $S_{\ell}^{(0)}$ and $S_{\ell}^{(1)}$, to be the same: $\rm B_0<0.45$ at $95%$ confidence level (CL), and $\rm B_0<0.67$ at $99%$ CL in the BZ model. The third skew-spectrum does not give any meaningful constraint. We find that the optimal skew-spectrum provides much more powerful constraint, giving $\rm B_0<0.071$ at $95%$ CL and $\rm B_0<0.15$ at $99%$ CL, which is essentially identical to what can be achieved using the full bispectrum.

We give an analytical form for the weighted correlation function of peaks in a Gaussian random field. In a cosmological context, this approach strictly describes the formation bias and is the main result here. Nevertheless, we show its validity and applicability to the evolved cosmological density field and halo field, using Gaussian random field realisations and dark matter N-body numerical simulations. Using this result from peak theory we compute the bias of peaks (and dark matter halos) and show that it reproduces results from the simulations at the ${\mathcal O}(10\%)$ level. Our analytical formula for the bias predicts a scale-dependent bias with two characteristics: a broad band shape which, however, is most affected by the choice of weighting scheme and evolution bias, and a more robust, narrow feature localised at the BAO scale, an effect that is confirmed in simulations. This scale-dependent bias smooths the BAO feature but, conveniently, does not move it. We provide a simple analytic formula to describe this effect. We envision that our analytic solution will be of use for galaxy surveys that exploit galaxy clustering.

Weak lensing alters the size of images with a similar magnitude to the distortion due to shear. Galaxy size probes the convergence field, and shape the shear field, both of which contain cosmological information. We show the gains expected in the Dark Energy Figure of Merit if galaxy size information is used in combination with galaxy shape. In any normal analysis of cosmic shear, galaxy sizes are also studied, so this is extra statistical information comes for free and is currently unused. There are two main results in this letter: firstly, we show that size measurement can be made uncorrelated with ellipticity measurement, thus allowing the full statistical gain from the combination, provided that $\sqrt{Area}$ is used as a size indicator; secondly, as a proof of concept, we show that when the relevant modes are noise-dominated, as is the norm for lensing surveys, the gains are substantial, with improvements of about 68% in the Figure of Merit expected when systematic errors are ignored. An approximate treatment of such systematics such as intrinsic alignments and size-magnitude correlations respectively suggests that a much better improvement in the Dark Energy Figure of Merit of even a factor of ~4 may be achieved.

We outline the prospects for performing pioneering radio weak gravitational lensing analyses using observations from a potential forthcoming JVLA Sky Survey program. A large-scale survey with the JVLA can offer interesting and unique opportunities for performing weak lensing studies in the radio band, a field which has until now been the preserve of optical telescopes. In particular, the JVLA has the capacity for large, deep radio surveys with relatively high angular resolution, which are the key characteristics required for a successful weak lensing study. We highlight the potential advantages and unique aspects of performing weak lensing in the radio band. In particular, the inclusion of continuum polarisation information can greatly reduce noise in weak lensing reconstructions and can also remove the effects of intrinsic galaxy alignments, the key astrophysical systematic effect that limits weak lensing at all wavelengths. We identify a VLASS "deep fields" program (total area ~10-20 square degs), to be conducted at L-band and with high-resolution (A-array configuration), as the optimal survey strategy from the point of view of weak lensing science. Such a survey will build on the unique strengths of the JVLA and will remain unsurpassed in terms of its combination of resolution and sensitivity until the advent of the Square Kilometre Array. We identify the best fields on the JVLA-accessible sky from the point of view of overlapping with existing deep optical and near infra-red data which will provide crucial redshift information and facilitate a host of additional compelling multi-wavelength science.

The Planck nominal mission cosmic microwave background (CMB) maps yield unprecedented constraints on primordial non-Gaussianity (NG). Using three optimal bispectrum estimators, separable template-fitting (KSW), binned, and modal, we obtain consistent values for the primordial local, equilateral, and orthogonal bispectrum amplitudes, quoting as our final result fNL^local= 2.7+/-5.8, fNL^equil= -42+/-75, and fNL^ortho= -25+\-39 (68% CL statistical). NG is detected in the data; using skew-C_l statistics we find a nonzero bispectrum from residual point sources, and the ISW-lensing bispectrum at a level expected in the LambdaCDM scenario. The results are based on comprehensive cross-validation of these estimators on Gaussian and non-Gaussian simulations, are stable across component separation techniques, pass an extensive suite of tests, and are confirmed by skew-C_l, wavelet bispectrum and Minkowski functional estimators. Beyond estimates of individual shape amplitudes, we present model-independent, 3-dimensional reconstructions of the Planck CMB bispectrum and thus derive constraints on early-Universe scenarios that generate primordial NG, including general single-field models of inflation, excited initial states (non-Bunch-Davies vacua), and directionally-dependent vector models. We provide an initial survey of scale-dependent feature and resonance models. These results bound both general single-field and multi-field model parameter ranges, such as the speed of sound, c_s \geq 0.02 (95% CL), in an effective field theory parametrization, and the curvaton decay fraction r_D \geq 0.15 (95% CL). The Planck data significantly limit the viable parameter space of the ekpyrotic/cyclic scenarios. The amplitude of the 4-point function in the local model tauNL < 2800 (95% CL). These constraints represent the highest precision tests to date of physical mechanisms for the origin of cosmic structure.

In this paper, we motivate the use of galaxy clustering measurements using photometric redshift information, including a contribution from flux magnification, as a probe of cosmology. We present cosmological forecasts when clustering data alone is used, and when clustering is combined with a cosmic shear analysis. We consider two types of clustering analysis: firstly, clustering with only redshift auto-correlations in tomographic redshift bins; secondly, using all available redshift bin correlations. Finally, we consider how inferred cosmological parameters may be biased using each analysis when flux magnification is neglected. Results are presented for a Stage III ground-based survey, and a Stage IV space-based survey modelled with photometric redshift errors, and values for the slope of the luminosity function inferred from CFHTLenS catalogues. We find that combining clustering information with shear can improve constraints on cosmological parameters, giving an improvement to a Dark Energy Task Force-like figure of merit by a factor of 1.33 when only auto-correlations in redshift are used for the clustering analysis, rising to 1.52 when cross-correlations in redshift are also included. The addition of galaxy-galaxy lensing gives further improvement, with increases in figure of merit by a factor of 2.82 and 3.7 for each type of clustering analysis respectively. The presence of flux magnification in a clustering analysis does not significantly affect the precision of cosmological constraints when combined with cosmic shear and galaxy-galaxy lensing. However if magnification is neglected, inferred cosmological parameter values are biased, with biases in some cosmological parameters larger than statistical errors. (Abridged)

We present a method for suppressing contributions from higher-order terms in perturbation theory, greatly increasing the amount of information which may be extracted from the matter power spectrum. In an evolved cosmological density field the highest density regions are responsible for the bulk of the nonlinear power. By suitably down-weighting these problematic regions we find that the one- and two-loop terms are typically reduced in amplitude by ~70 per cent and ~95 per cent respectively, relative to the linear power spectrum. This greatly facilitates modelling the shape of the galaxy power spectrum, potentially increasing the number of useful Fourier modes by more than two orders of magnitude. We provide a demonstration of how this technique allows the galaxy bias and the amplitude of linear matter perturbations sigma_8 to be determined from the power spectrum on conventionally nonlinear scales, 0.1<k<0.7 h/Mpc.

We generalize the concept of the ordinary skew-spectrum to probe the effect of non-Gaussianity on the morphology of Cosmic Microwave Background (CMB) maps in several domains: in real-space (where they are commonly known as cumulant-correlators), and in harmonic and needlet bases. The essential aim is to retain more information than normally contained in these statistics, in order to assist in determining the source of any measured non-Gaussianity, in the same spirit as Munshi & Heavens (2010) skew-spectra were used to identify foreground contaminants to the CMB bispectrum in Planck data. Using a perturbative series to construct the Minkowski Functionals (MFs), we provide a pseudo-Cl based approach in both harmonic and needlet representations to estimate these spectra in the presence of a mask and inhomogeneous noise. Assuming homogeneous noise we present approx- imate expressions for error covariance for the purpose of joint estimation of these spectra. We present specific results for four different models of primordial non-Gaussianity local, equilateral, orthogonal and enfolded models, as well as non-Gaussianity caused by unsubtracted point sources. Closed form results of next-order corrections to MFs too are obtained in terms of a quadruplet of kurt-spectra. We also use the method of modal decomposition of the bispectrum and trispectrum to reconstruct the MFs as an alternative method of reconstruction of morphological properties of CMB maps. Finally, we introduce the odd-parity skew-spectra to probe the odd-parity bispectrum and its impact on the morphology of the CMB sky. Although developed for the CMB, the generic results obtained here can be useful in other areas of cosmology.

We provide an exact expression for the multi-variate joint probability distribution function of non-Gaussian fields primordially arising from local transformations of a Gaussian field. This kind of non-Gaussianity is generated in many models of inflation. We apply our expression to the non- Gaussianity estimation from Cosmic Microwave Background maps and the halo mass function where we obtain analytical expressions. We also provide analytic approximations and their range of validity. For the Cosmic Microwave Background we give a fast way to compute the PDF which is valid up to 7{\sigma} for fNL values (both true and sampled) not ruled out by current observations, which consists of expressing the PDF as a combination of bispectrum and trispectrum of the temperature maps. The resulting expression is valid for any kind of non-Gaussianity and is not limited to the local type. The above results may serve as the basis for a fully Bayesian analysis of the non-Gaussianity parameter.

We present a finely-binned tomographic weak lensing analysis of the Canada-France-Hawaii Telescope Lensing Survey, CFHTLenS, mitigating contamination to the signal from the presence of intrinsic galaxy alignments via the simultaneous fit of a cosmological model and an intrinsic alignment model. CFHTLenS spans 154 square degrees in five optical bands, with accurate shear and photometric redshifts for a galaxy sample with a median redshift of zm =0.70. We estimate the 21 sets of cosmic shear correlation functions associated with six redshift bins, each spanning the angular range of 1.5<theta<35 arcmin. We combine this CFHTLenS data with auxiliary cosmological probes: the cosmic microwave background with data from WMAP7, baryon acoustic oscillations with data from BOSS, and a prior on the Hubble constant from the HST distance ladder. This leads to constraints on the normalisation of the matter power spectrum sigma_8 = 0.799 +/- 0.015 and the matter density parameter Omega_m = 0.271 +/- 0.010 for a flat Lambda CDM cosmology. For a flat wCDM cosmology we constrain the dark energy equation of state parameter w = -1.02 +/- 0.09. We also provide constraints for curved Lambda CDM and wCDM cosmologies. We find the intrinsic alignment contamination to be galaxy-type dependent with a significant intrinsic alignment signal found for early-type galaxies, in contrast to the late-type galaxy sample for which the intrinsic alignment signal is found to be consistent with zero.

We investigate the extent to which cosmic size magnification may be used to com- plement cosmic shear in weak gravitational lensing surveys, with a view to obtaining high-precision estimates of cosmological parameters. Using simulated galaxy images, we find that size estimation can be an excellent complement, finding that unbiased estimation of the convergence field is possible with galaxies with angular sizes larger than the point-spread function (PSF) and signal-to-noise ratio in excess of 10. The statistical power is similar to, but not quite as good as, cosmic shear, and it is subject to different systematic effects. Application to ground-based data will be challeng- ing, with relatively large empirical corrections required to account for with biases for galaxies which are smaller than the PSF, but for space-based data with 0.1 arcsecond resolution, the size distribution of galaxies brighter than i=24 is ideal for accurate estimation of cosmic size magnification.

Observationally confirming spatial homogeneity on sufficiently large cosmological scales is of importance to test one of the underpinning assumptions of cosmology, and is also imperative for correctly interpreting dark energy. A challenging aspect of this is that homogeneity must be probed inside our past lightcone, while observations take place on the lightcone. The star formation history (SFH) in the galaxy fossil record provides a novel way to do this. We calculate the SFH of stacked Luminous Red Galaxy (LRG) spectra obtained from the Sloan Digital Sky Survey. We divide the LRG sample into 12 equal area contiguous sky patches and 10 redshift slices (0.2 < z < 0.5), which correspond to 120 blocks of volume 0.04Gpc3. Using the SFH in a time period which samples the history of the Universe between look-back times 11.5 to 13.4 Gyrs as a proxy for homogeneity, we calculate the posterior distribution for the excess large-scale variance due to inhomogeneity, and find that the most likely solution is no extra variance at all. At 95% credibility, there is no evidence of deviations larger than 5.8%.

Secondary contributions to the anisotropy of the Cosmic Microwave Background (CMB), such as the integrated Sachs-Wolfe (ISW) effect, the thermal Sunyaev-Zel'dovich effect (tSZ), and the effect of gravitational lensing, have distinctive non-Gaussian signatures, and full descriptions therefore require information beyond that contained in their power spectra. In this paper we use the recently introduced skew-spectra associated with the Minkowski Functionals (MF) to probe the topology of CMB maps to probe the secondary non-Gaussianity as a function of beam-smoothing in order to separate various contributions. We devise estimators for these spectra in the presence of a realistic observational masks and present expressions for their covariance as a function of instrumental noise. Specific results are derived for the mixed ISW-lensing and tSZ-lensing bispectra as well as contamination due to point sources for noise levels that correspond to the Planck (143 GHz channel) and EPIC (150 GHz channel) experiments. The cumulative signal to noise ration $S/N$ for one-point generalized skewness-parameters can reach an order of ${\cal O}(10)$ for Planck and two orders of magnitude higher for EPIC, i.e. ${\cal O}(10^3)$. We also find that these three spectra skew-spectra are correlated, having correlation coefficients $r \sim 0.5-1.0$; higher $l$ modes are more strongly correlated. Though the values of $S/N$ increase with decreasing noise, the triplets of skew-spectra that determine the MFs bcome more correlated; the $S/N$ ratios of lensing-induced skew-spectra are smaller compared to that of a frequency-cleaned tSZ map.

We present a method for attaining sub-arcsecond pointing stability during sub- orbital balloon flights, as designed for in the High Altitude Lensing Observatory (HALO) concept. The pointing method presented here has the potential to perform near-space quality optical astronomical imaging at 1-2% of the cost of space-based missions. We also discuss an architecture that can achieve sufficient thermomechanical stability to match the pointing stability. This concept is motivated by advances in the development and testing of Ultra Long Duration Balloon (ULDB) flights which promise to allow observation campaigns lasting more than three months. The design incorporates a multi-stage pointing architecture comprising: a gondola coarse azimuth control system, a multi-axis nested gimbal frame structure with arcsecond stability, a telescope de-rotator to eliminate field rotation, and a fine guidance stage consisting of both a telescope mounted angular rate sensor and guide CCDs in the focal plane to drive a fast-steering mirror. We discuss the results of pointing tests together with a preliminary thermo-mechanical analysis required for sub-arcsecond pointing at high altitude. Possible future applications in the areas of wide-field surveys and exoplanet searches are also discussed.

GRavitational lEnsing Accuracy Testing 2010 (GREAT10) is a public image analysis challenge aimed at the development of algorithms to analyze astronomical images. Specifically, the challenge is to measure varying image distortions in the presence of a variable convolution kernel, pixelization and noise. This is the second in a series of challenges set to the astronomy, computer science and statistics communities, providing a structured environment in which methods can be improved and tested in preparation for planned astronomical surveys. GREAT10 extends upon previous work by introducing variable fields into the challenge. The "Galaxy Challenge" involves the precise measurement of galaxy shape distortions, quantified locally by two parameters called shear, in the presence of a known convolution kernel. Crucially, the convolution kernel and the simulated gravitational lensing shape distortion both now vary as a function of position within the images, as is the case for real data. In addition, we introduce the "Star Challenge" that concerns the reconstruction of a variable convolution kernel, similar to that in a typical astronomical observation. This document details the GREAT10 Challenge for potential participants. Continually updated information is also available from http://www.greatchallenges.info.

A large fraction of the information collected by cosmological surveys is simply discarded to avoid lengthscales which are difficult to model theoretically. We introduce a new technique which enables the extraction of useful information from the bispectrum of galaxies well beyond the conventional limits of perturbation theory. Our results strongly suggest that this method increases the range of scales where the relation between the bispectrum and power spectrum in tree-level perturbation theory may be applied, from k_max ~ 0.1 h/Mpc to ~ 0.7 h/Mpc. This leads to correspondingly large improvements in the determination of galaxy bias. Since the clipped matter power spectrum closely follows the linear power spectrum, there is the potential to use this technique to probe the growth rate of linear perturbations and confront theories of modified gravity with observation.

Euclid is a space-based survey mission from the European Space Agency designed to understand the origin of the Universe's accelerating expansion. It will use cosmological probes to investigate the nature of dark energy, dark matter and gravity by tracking their observational signatures on the geometry of the universe and on the cosmic history of structure formation. The mission is optimised for two independent primary cosmological probes: Weak gravitational Lensing (WL) and Baryonic Acoustic Oscillations (BAO). The Euclid payload consists of a 1.2 m Korsch telescope designed to provide a large field of view. It carries two instruments with a common field-of-view of ~0.54 deg2: the visual imager (VIS) and the near infrared instrument (NISP) which contains a slitless spectrometer and a three bands photometer. The Euclid wide survey will cover 15,000 deg2 of the extragalactic sky and is complemented by two 20 deg2 deep fields. For WL, Euclid measures the shapes of 30-40 resolved galaxies per arcmin2 in one broad visible R+I+Z band (550-920 nm). The photometric redshifts for these galaxies reach a precision of dz/(1+z) < 0.05. They are derived from three additional Euclid NIR bands (Y, J, H in the range 0.92-2.0 micron), complemented by ground based photometry in visible bands derived from public data or through engaged collaborations. The BAO are determined from a spectroscopic survey with a redshift accuracy dz/(1+z) =0.001. The slitless spectrometer, with spectral resolution ~250, predominantly detects Ha emission line galaxies. Euclid is a Medium Class mission of the ESA Cosmic Vision 2015-2025 programme, with a foreseen launch date in 2019. This report (also known as the Euclid Red Book) describes the outcome of the Phase A study.

The standard Friedmann model of cosmology is based on the Copernican Principle, i.e. the assumption of a homogeneous background on which structure forms via perturbations. Homogeneity underpins both general relativistic and modified gravity models and is central to the way in which we interpret observations of the CMB and the galaxy distribution. It is therefore important to probe homogeneity via observations. We describe a test based on the fossil record of distant galaxies: if we can reconstruct key intrinsic properties of galaxies as functions of proper time along their worldlines, we can compare such properties at the same proper time for our galaxy and others. We achieve this by computing the lookback time using radial Baryon Acoustic Oscillations, and the time along galaxy world line using stellar physics, allowing us to probe homogeneity, in principle anywhere inside the past light cone. Agreement in the results would be an important consistency test -- although it would not in itself prove homogeneity. Any significant deviation in the results however would signal a breakdown of homogeneity.

In these lectures I give an overview of gravitational lensing, concentrating on theoretical aspects, including derivations of some of the important results. Topics covered include the determination of surface mass densities of intervening lenses, as well as the statistical analysis of distortions of galaxy images by general inhomogeneities (cosmic shear), both in 2D projection on the sky, and in 3D where source distance information is available. 3D mass reconstruction and the shear ratio test are also considered, and the sensitivity of observables to Dark Energy is used to show how its equation of state may be determined using weak lensing. Finally, the article considers the prospect of testing Einstein's General Relativity with weak lensing, exploiting the differences in growth rates of perturbations in different models.} \abstract{In these lectures I give an overview of gravitational lensing, concentrating on theoretical aspects, including derivations of some of the important results. Topics covered include the determination of surface mass densities of intervening lenses, as well as the statistical analysis of distortions of galaxy images by general inhomogeneities (cosmic shear), both in 2D projection on the sky, and in 3D where source distance information is available. 3D mass reconstruction and the shear ratio test are also considered, and the sensitivity of observables to Dark Energy is used to show how its equation of state may be determined using weak lensing. Finally, the article considers the prospect of testing Einstein's General Relativity with weak lensing, exploiting the differences in growth rates of perturbations in different models.

We investigate the extent to which the pure magnification effect of gravitational lensing can be extracted from galaxy clustering statistics, by a nulling method which aims to eliminate terms arising from the intrinsic clustering of galaxies. The aim is to leave statistics which are free from the uncertainties of galaxy bias. We find that nulling can be done effectively, leaving data which are relatively insensitive to uncertainties in galaxy bias and its evolution, leading to cosmological parameter estimation which is effectively unbiased. This advantage comes at the expense of increased statistical errors, which are in some cases large, but it offers a robust alternative analysis method to cosmic shear for cosmological imaging surveys designed for weak lensing studies, or to full modelling of the clustering signal including magnification effects.

We present parameter estimation forecasts for future 3D cosmic shear surveys for a class of Unified Dark Matter (UDM) models, where a single scalar field mimics both Dark Matter (DM) and Dark Energy (DE). These models have the advantage that they can describe the dynamics of the Universe with a single matter component providing an explanation for structure formation and cosmic acceleration. A crucial feature of the class of UDM models we use in this work is characterised by a parameter, c_inf (in units of the speed of light c=1), that is the value of the sound speed at late times, and on which structure formation depends. We demonstrate that the properties of the DM-like behaviour of the scalar field can be estimated with very high precision with large-scale, fully 3D weak lensing surveys. We found that 3D weak lensing significantly constrains c_inf, and we find minimal errors 0.00003, for the fiducial value c_inf=0.001, and 0.000026, for c_inf=0.012. Moreover, we compute the Bayesian evidence for UDM models over the LCDM model as a function of c_inf. For this purpose, we can consider the LCDM model as a UDM model with c_inf=0. We find that the expected evidence clearly shows that the survey data would unquestionably favour UDM models over the LCDM model, for the values c_inf>0.001.

Fisher Information Matrix methods are commonly used in cosmology to estimate the accuracy that cosmological parameters can be measured with a given experiment, and to optimise the design of experiments. However, the standard approach usually assumes both data and parameter estimates are Gaussian-distributed. Further, for survey forecasts and optimisation it is usually assumed the power-spectra covariance matrix is diagonal in Fourier-space. But in the low-redshift Universe, non-linear mode-coupling will tend to correlate small-scale power, moving information from lower to higher-order moments of the field. This movement of information will change the predictions of cosmological parameter accuracy. In this paper we quantify this loss of information by comparing naive Gaussian Fisher matrix forecasts with a Maximum Likelihood parameter estimation analysis of a suite of mock weak lensing catalogues derived from N-body simulations, based on the SUNGLASS pipeline, for a 2-D and tomographic shear analysis of a Euclid-like survey. In both cases we find that the 68% confidence area of the Omega_m-sigma_8 plane increases by a factor 5. However, the marginal errors increase by just 20 to 40%. We propose a new method to model the effects of nonlinear shear-power mode-coupling in the Fisher Matrix by approximating the shear-power distribution as a multivariate Gaussian with a covariance matrix derived from the mock weak lensing survey. We find that this approximation can reproduce the 68% confidence regions of the full Maximum Likelihood analysis in the Omega_m-sigma_8 plane to high accuracy for both 2-D and tomographic weak lensing surveys. Finally, we perform a multi-parameter analysis of Omega_m, sigma_8, h, n_s, w_0 and w_a to compare the Gaussian and non-linear mode-coupled Fisher matrix contours. (Abridged)

Here we present a number of improvements to weak lensing 3D power spectrum analysis, 3D cosmic shear, that uses the shape and redshift information of every galaxy to constrain cosmological parameters. We show how photometric redshift probability distributions for individual galaxies can be directly included in this statistic with no averaging. We also include the Limber approximation, considerably simplifying full 3D cosmic shear analysis, and we investigate its range of applicability. Finally we show the relationship between weak lensing tomography and the 3D cosmic shear field itself; the steps connecting them being the Limber approximation, a harmonic-space transform and a discretisation in wavenumber. Each method has its advantages: 3D cosmic shear analysis allows straightforward inclusion of all relevant modes, thus ensuring minimum error bars, and direct control of the range of physical wavenumbers probed, to avoid the uncertain highly nonlinear regime. On the other hand, tomography is more convenient for checking systematics through direct investigation of the redshift dependence of the signal. Finally, for tomography, we suggest that the angular modes probed should be redshift-dependent, to recover some of the 3D advantages.

We introduce a collection of statistics appropriate for the study of spinorial quantities defined in three dimensions, focussing on applications to cosmological weak gravitational lensing studies in 3D. In particular, we concentrate on power spectra associated with three- and four-point statistics, which have the advantage of compressing a large number of typically very noisy modes into a convenient data set. It has been shown previously by \cite{MuHe09} that, for non--Gaussianity studies in the microwave background, such compression can be lossless for certain purposes, so we expect the statistics we define here to capture the bulk of the cosmological information available in these higher-order statistics. We consider the effects of a sky mask and noise, and use Limber's approximation to show how, for high-frequency angular modes, confrontation of the statistics with theory can be achieved efficiently and accurately. We focus on scalar and spinorial fields including convergence, shear and flexion of 3D weak lensing, but many of the results apply for general spin fields.

Comparing clustering of differently biased tracers of the dark matter distribution offers the opportunity to reduce the cosmic variance error in the measurement of certain cosmological parameters. We develop a formalism that includes bias non-linearities and stochasticity. Our formalism is general enough that can be used to optimise survey design and tracers selection and optimally split (or combine) tracers to minimise the error on the cosmologically interesting quantities. Our approach generalises the one presented by McDonald & Seljak (2009) of circumventing sample variance in the measurement of $f\equiv d \ln D/d\ln a$. We analyse how the bias, the noise, the non-linearity and stochasticity affect the measurements of $Df$ and explore in which signal-to-noise regime it is significantly advantageous to split a galaxy sample in two differently-biased tracers. We use N-body simulations to find realistic values for the parameters describing the bias properties of dark matter haloes of different masses and their number density. We find that, even if dark matter haloes could be used as tracers and selected in an idealised way, for realistic haloes, the sample variance limit can be reduced only by up to a factor $\sigma_{2tr}/\sigma_{1tr}\simeq 0.6$. This would still correspond to the gain from a three times larger survey volume if the two tracers were not to be split. Before any practical application one should bear in mind that these findings apply to dark matter haloes as tracers, while realistic surveys would select galaxies: the galaxy-host halo relation is likely to introduce extra stochasticity, which may reduce the gain further.

We present a series of colour evolution models for Luminous Red Galaxies (LRGs) in the 7th spectroscopic data release of the Sloan Digital Sky Survey (SDSS), computed using the full-spectrum fitting code VESPA on high signal-to-noise stacked spectra. The colour-evolution models are computed as a function of colour, luminosity and redshift, and we do not a-priori assume that LRGs constitute a uniform population of galaxies in terms of stellar evolution. By computing star-formation histories from the fossil record, the measured stellar evolution of the galaxies is decoupled from the survey's selection function, which also evolves with redshift. We present these evolutionary models computed using three different sets of Stellar Population Synthesis (SPS) codes. We show that the traditional fiducial model of purely passive stellar evolution of LRGs is broadly correct, but it is not sufficient to explain the full spectral signature. We also find that higher-order corrections to this model are dependent on the SPS used, particularly when calculating the amount of recent star formation. The amount of young stars can be non-negligible in some cases, and has important implications for the interpretation of the number density of LRGs within the selection box as a function of redshift. Dust extinction, however, is more robust to the SPS modelling: extinction increases with decreasing luminosity, increasing redshift, and increasing r-i colour. We are making the colour evolution tracks publicly available at http://www.icg.port.ac.uk/~tojeiror/lrg_evolution/.

A new cosmic shear analysis pipeline SUNGLASS (Simulated UNiverses for Gravitational Lensing Analysis and Shear Surveys) is introduced. SUNGLASS is a pipeline that rapidly generates simulated universes for weak lensing and cosmic shear analysis. The pipeline forms suites of cosmological N-body simulations and performs tomographic cosmic shear analysis using line-of-sight integration through these simulations while saving the particle lightcone information. Galaxy shear and convergence catalogues with realistic 3D galaxy redshift distributions are produced for the purposes of testing weak lensing analysis techniques and generating covariance matrices for data analysis and cosmological parameter estimation. We present a suite of fast medium resolution simulations with shear and convergence maps for a generic 100 square degree survey out to a redshift of z = 1.5, with angular power spectra agreeing with the theory to better than a few percent accuracy up to l = 10^3 for all source redshifts up to z = 1.5 and wavenumbers up to l = 2000 for the source redshifts z > 1.1. At higher wavenumbers, there is a failure of the theoretical lensing power spectrum reflecting the known discrepancy of the Smith et al. (2003) fitting formula at high physical wavenumbers. A two-parameter Gaussian likelihood analysis of sigma_8 and Omega_m is also performed on the suite of simulations, demonstrating that the cosmological parameters are recovered from the simulations and the covariance matrices are stable for data analysis. We find no significant bias in the parameter estimation at the level of ~ 0.02. The SUNGLASS pipeline should be an invaluable tool in weak lensing analysis.

Minimum-variance estimators for the parameter fnl that quantifies local-model non-Gaussianity can be constructed from the cosmic microwave background (CMB) bispectrum (three-point function) and also from the trispectrum (four-point function). Some have suggested that a comparison between the estimates for the values of fnl from the bispectrum and trispectrum allow a consistency test for the model. But others argue that the saturation of the Cramer-Rao bound by the bispectrum estimator implies that no further information on fnl can be obtained from the trispectrum. Here we elaborate the nature of the correlation between the bispectrum and trispectrum estimators for fnl. We show that the two estimators become statistically independent in the limit of large number of CMB pixels and thus that the trispectrum estimator does indeed provide additional information on fnl beyond that obtained from the bispectrum. We explain how this conclusion is consistent with the Cramer-Rao bound. Our discussion of the Cramer-Rao bound may be of interest to those doing Fisher-matrix parameter-estimation forecasts or data analysis in other areas of physics as well.

Using light curves and host galaxy spectra of 101 Type Ia supernovae (SNe Ia) with redshift $z \lesssim 0.3$ from the SDSS Supernova Survey (SDSS-SN), we derive the SN Ia rate as a function of progenitor age (the delay time distribution, or DTD). We use the VESPA stellar population synthesis algorithm to analyze the SDSS spectra of all galaxies in the field searched by SDSS-SN, giving us a reference sample of 77,000 galaxies for our SN Ia hosts. Our method does not assume any a priori shape for the DTD and therefore is minimally parametric. We present the DTD in physical units for high stretch (luminous, slow declining) and low stretch (subluminous, fast declining) supernovae in three progenitor age bins. We find strong evidence of two progenitor channels: one that produces high stretch SNe Ia $\lesssim 400$ Myr after the birth of the progenitor system, and one that produces low stretch SNe Ia with a delay $\gtrsim 2.4$ Gyr. We find that each channel contributes roughly half of the Type Ia rate in our reference sample. We also construct the average spectra of high stretch and low stretch SN Ia host galaxies, and find that the difference of these spectra looks like a main sequence B star with nebular emission lines indicative of star formation. This supports our finding that there are two populations of SNe Ia, and indicates that the progenitors of high stretch SNe are at the least associated with very recent star formation in the last few tens of Myr. Our results provide valuable constraints for models of Type Ia progenitors and may help improve the calibration of SNe Ia as standard candles.

We outline the expected constraints on non-Gaussianity from the cosmic microwave background (CMB) with current and future experiments, focusing on both the third (f_{NL}) and fourth-order (g_{NL} and \tau_{NL}) amplitudes of the local configuration or non-Gaussianity. The experimental focus is the skewness (two-to-one) and kurtosis (two-to-two and three-to-one) power spectra from weighted maps. In adition to a measurement of \tau_{NL} and g_{NL} with WMAP 5-year data, our study provides the first forecasts for future constraints on g_{NL}. We describe how these statistics can be corrected for the mask and cut-sky through a window function, bypassing the need to compute linear terms that were introduced for the previous-generation non-Gaussianity statistics, such as the skewness estimator. We discus the ratio A_{NL} = \tau_{NL}/(6f_{NL}/5)^2 as an additional test of single-field inflationary models and discuss the physical significance of each statistic. Using these estimators with WMAP 5-Year V+W-band data out to l_{max}=600 we constrain the cubic order non-Gaussianity parameters \tau_{NL}, and g_{NL} and find -7.4 < g_{NL}/10^5 < 8.2 and -0.6 < \tau_{NL}/10^4 < 3.3 improving the previous COBE-based limit on \tau_{NL} < 10^8 nearly four orders of magnitude with WMAP.

A large sub-mm survey with Herschel will enable many exciting science opportunities, especially in an era of wide-field optical and radio surveys and high resolution cosmic microwave background experiments. The Herschel-SPIRE Legacy Survey (HSLS), will lead to imaging data over 4000 sq. degrees at 250, 350, and 500 micron. Major Goals of HSLS are: (a) produce a catalog of 2.5 to 3 million galaxies down to 26, 27 and 33 mJy (50% completeness; 5 sigma confusion noise) at 250, 350 and 500 micron, respectively, in the southern hemisphere (3000 sq. degrees) and in an equatorial strip (1000 sq. degrees), areas which have extensive multi-wavelength coverage and are easily accessible from ALMA. Two thirds of the of the sources are expected to be at z > 1, one third at z > 2 and about a 1000 at z > 5. (b) Remove point source confusion in secondary anisotropy studies with Planck and ground-based CMB data. (c) Find at least 1200 strongly lensed bright sub-mm sources leading to a 2% test of general relativity. (d) Identify 200 proto-cluster regions at z of 2 and perform an unbiased study of the environmental dependence of star formation. (e) Perform an unbiased survey for star formation and dust at high Galactic latitude and make a census of debris disks and dust around AGB stars and white dwarfs.

In these lectures I cover a number of topics in cosmological data analysis. I concentrate on general techniques which are common in cosmology, or techniques which have been developed in a cosmological context. In fact they have very general applicability, for problems in which the data are interpreted in the context of a theoretical model, and thus lend themselves to a Bayesian treatment. We consider the general problem of estimating parameters from data, and consider how one can use Fisher matrices to analyse survey designs before any data are taken, to see whether the survey will actually do what is required. We outline numerical methods for estimating parameters from data, including Monte Carlo Markov Chains and the Hamiltonian Monte Carlo method. We also look at Model Selection, which covers various scenarios such as whether an extra parameter is preferred by the data, or answering wider questions such as which theoretical framework is favoured, using General Relativity and braneworld gravity as an example. These notes are not a literature review, so there are relatively few references.

We measure two higher-order power spectra involving weighted cubic and squared temperature anisotropy maps from WMAP 5-year data to study the trispectrum generated by primordial non-Gaussianity. Using these measurements combined with Gaussian and noise simulations, we constrain the cubic order non-Gaussianity parameters \tau_{NL}, and g_{NL}. With V+W-band data out to l_{max}=600, we find -7.4 < g_{\rm NL}/10^5 < 8.2 and -0.6 < \tau_{\rm NL}/10^4 < 3.3 improving the previous COBE-based limit on \tau_{\rm NL} < 10^8 nearly four orders of magnitude with WMAP. We find that the ratio of trispectrum to bispectrum amplitude as captured by the ratio of \tau_{\rm Nl}/(6f_{\rm NL}/5)^2 ranges from -3 to 21 at the 95% confidence level.

Owing to their more extensive sky coverage and tighter control on systematic errors, future deep weak lensing surveys should provide a better statistical picture of the dark matter clustering beyond the level of the power spectrum. In this context, the study of non-Gaussianity induced by gravity can help tighten constraints on the background cosmology by breaking parameter degeneracies, as well as throwing light on the nature of dark matter, dark energy or alternative gravity theories. Analysis of the shear or flexion properties of such maps is more complicated than the simpler case of the convergence due to the spinorial nature of the fields involved. Here we develop analytical tools for the study of higher-order statistics such as the bispectrum (or trispectrum) directly using such maps at different source redshift. The statistics we introduce can be constructed from cumulants of the shear or flexions, involving the cross-correlation of squared and cubic maps at different redshifts. Typically, the low signal-to-noise ratio prevents recovery of the bispectrum or trispectrum mode by mode. We define power spectra associated with each multi- spectra which compresses some of the available information of higher order multispectra. We show how these can be recovered from a noisy observational data even in the presence of arbitrary mask, which introduces mixing between Electric (E-type) and Magnetic (B-type) polarization, in an unbiased way. We also introduce higher order cross-correlators which can cross-correlate lensing shear with different tracers of large scale structures.

We explore a systematic approach to the analysis of primordial non-Gaussianity using fluctuations in temperature and polarization of the Cosmic Microwave Background (CMB). Following Munshi & Heavens (2009), we define a set of power-spectra as compressed forms of the bispectrum and trispectrum derived from CMB temperature and polarization maps; these spectra compress the information content of the corresponding full multispectra and can be useful in constraining early Universe theories. We generalize the standard pseudo-C_l estimators in such a way that they apply to these spectra involving both spin-0 and spin-2 fields, developing explicit expressions which can be used in the practical implementation of these estimators. While these estimators are suboptimal, they are nevertheless unbiased and robust hence can provide useful diagnostic tests at a relatively small computational cost. We next consider approximate inverse-covariance weighting of the data and construct a set of near-optimal estimators based on that approach. Instead of combining all available information from the entire set of mixed bi- or trispectra, i.e multispectra describing both temperature and polarization information, we provide analytical constructions for individual estimators, associated with particular multispectra. The bias and scatter of these estimators can be computed using Monte-Carlo techniques. Finally, we provide estimators which are completely optimal for arbitrary scan strategies and involve inverse covariance weighting; we present the results of an error analysis performed using a Fisher-matrix formalism at both the one-point and two-point level.

Weak gravitational lensing on a cosmological scales can provide strong constraints both on the nature of dark matter and the dark energy equation of state. Most current weak lensing studies are restricted to (two-dimensional) projections, but tomographic studies with photometric redshifts have started, and future surveys offer the possibility of probing the evolution of structure with redshift. In future we will be able to probe the growth of structure in 3D and put tighter constraints on cosmological models than can be achieved by the use of galaxy redshift surveys alone. Earlier studies in this direction focused mainly on evolution of the 3D power spectrum, but extension to higher-order statistics can lift degeneracies as well as providing information on primordial non-gaussianity. We present analytical results for specific higher-order descriptors, the bispectrum and trispectrum, as well as collapsed multi-point statistics derived from them, i.e. cumulant correlators. We also compute quantities we call the power spectra associated with the bispectrum and trispectrum, the Fourier transforms of the well-known cumulant correlators. We compute the redshift dependence of these objects and study their performance in the presence of realistic noise and photometric redshift errors.

A survey that can cover the sky in optical bands over wide fields to faint magnitudes with a fast cadence will enable many of the exciting science opportunities of the next decade. The Large Synoptic Survey Telescope (LSST) will have an effective aperture of 6.7 meters and an imaging camera with field of view of 9.6 deg^2, and will be devoted to a ten-year imaging survey over 20,000 deg^2 south of +15 deg. Each pointing will be imaged 2000 times with fifteen second exposures in six broad bands from 0.35 to 1.1 microns, to a total point-source depth of r~27.5. The LSST Science Book describes the basic parameters of the LSST hardware, software, and observing plans. The book discusses educational and outreach opportunities, then goes on to describe a broad range of science that LSST will revolutionize: mapping the inner and outer Solar System, stellar populations in the Milky Way and nearby galaxies, the structure of the Milky Way disk and halo and other objects in the Local Volume, transient and variable objects both at low and high redshift, and the properties of normal and active galaxies at low and high redshift. It then turns to far-field cosmological topics, exploring properties of supernovae to z~1, strong and weak lensing, the large-scale distribution of galaxies and baryon oscillations, and how these different probes may be combined to constrain cosmological models and the physics of dark energy.

Several recent papers have suggested that the cosmological constant Lambda directly influences the gravitational deflection of light. We place this problem in a cosmological context, deriving an expression for the linear potentials which control the cosmological bending of light, finding that it has no explicit dependence on the cosmological constant. To explore the physical origins of the apparent Lambda-dependent potential that appears in the static Kottler metric, we highlight the two classical effects which lead to the aberration of light. The first relates to the observer's motion relative to the source, and encapsulates the familiar concept of angular-diameter distance. The second term, which has proved to be the source of debate, arises from cosmic acceleration, but is rarely considered since it vanishes for photons with radial motion. This apparent form of light-bending gives the appearance of curved geodesics even within a flat and homogeneous universe. However this cannot be construed as a real lensing effect, since its value depends on the observer's frame of reference. Our conclusion is thus that standard results for gravitational lensing in a universe containing Lambda do not require modification, with any influence of Lambda being restricted to negligible high-order terms.

In this non-specialist review I look at how weak lensing can provide information on the dark sector of the Universe. The review concentrates on what can be learned about Dark Matter, Dark Energy and Dark Gravity, and why. On Dark Matter, results on the confrontation of theoretical profiles with observation are reviewed, and measurements of neutrino masses discussed. On Dark Energy, the interest is whether this could be Einstein's cosmological constant, and prospects for high-precision studies of the equation of state are considered. On Dark Gravity, we consider the exciting prospects for future weak lensing surveys to distinguish General Relativity from extra-dimensional or other gravity theories.

Cosmic microwave background studies of non-Gaussianity involving higher-order multispectra can distinguish between early universe theories that predict nearly identical power spectra. However, the recovery of higher-order multispectra is difficult from realistic data due to their complex response to inhomogeneous noise and partial sky coverage, which are often difficult to model analytically. A traditional alternative is to use one-point cumulants of various orders, which collapse the information present in a multispectrum to one number. The disadvantage of such a radical compression of the data is a loss of information as to the source of the statistical behaviour. A recent study by Munshi & Heavens (2009) has shown how to define the skew spectrum (the power spectra of a certain cubic field, related to the bispectrum) in an optimal way and how to estimate it from realistic data. The skew spectrum retains some of the information from the full configuration-dependence of the bispectrum, and can contain all the information on non-Gaussianity. In the present study, we extend the results of the skew spectrum to the case of two degenerate power-spectra related to the trispectrum. We also explore the relationship of these power-spectra and cumulant correlators previously used to study non-Gaussianity in projected galaxy surveys or weak lensing surveys. We construct nearly optimal estimators for quick tests and generalise them to estimators which can handle realistic data with all their complexity in a completely optimal manner. We show how these higher-order statistics and the related power spectra are related to the Taylor expansion coefficients of the potential in inflation models, and demonstrate how the trispectrum can constrain both the quadratic and cubic terms.

We measure the skewness power spectrum of the Cosmic Microwave Background (CMB) anisotropies optimized for a detection of the secondary bispectrum generated by the correlation of the CMB lensing potential with integrated Sachs-Wolfe effect and the Sunyaev-Zel'dovich effect. The covariance of our measurements is generated by Monte-Carlo simulations of Gaussian CMB fields with noise properties consistent with Wilkinson Microwave Anisotropy Probe (WMAP) 5-year data. When interpreting multi-frequency measurements we also take into account the confusion resulting from unresolved radio point sources. We analyze Q, V and W-band WMAP 5-year raw and foreground-cleaned maps using the KQ75 mask out to l_max=600. We find no significant evidence for a non-zero non-Gaussian signal from the lensing-secondary correlation in all three bands and we constrain the overall amplitude of the cross power spectrum between CMB lensing potential and the sum of SZ and ISW fluctuations to be 0.42 \pm 0.86 and 1.19 \pm 0.86 in combined V and W-band raw and foreground-cleaned maps provided by the WMAP team, respectively. The point source amplitude at the bispectrum level measured with this skewness power spectrum is higher than previous measurements of point source non-Gaussianity. We also consider an analysis where we also account for the primordial non-Gaussianity in addition to lensing-secondary bispectrum and point sources. The focus of this paper is on secondary anisotropies. Consequently the estimator is not optimised for primordial non-Gaussianity and the limit we find on local non-Gaussianity from the foreground-cleaned V+W maps is f_NL = -13 \pm 62, when marginalized over point sources and lensing-ISW/SZ constributions to the total bispectrum.

We applied the VESPA algorithm to the Sloan Digital Sky Survey final data release of the Main Galaxies and Luminous Red Galaxies samples. The result is a catalogue of stellar masses, detailed star formation and metallicity histories and dust content of nearly 800,000 galaxies. We make the catalogue public via a T-SQL database, which is described in detail in this paper. We present the results using a range of stellar population and dust models, and will continue to update the catalogue as new and improved models are made public. The data and documentation are currently online, and can be found at http://www-wfau.roe.ac.uk/vespa/. We also present a brief exploration of the catalogue, and show that the quantities derived are robust: luminous red galaxies can be described by one to three populations, whereas a main galaxy sample galaxy needs on average two to five; red galaxies are older and less dusty; the dust values we recover are well correlated with measured Balmer decrements and star formation rates are also in agreement with previous measurements.

We present the results of the GREAT08 Challenge, a blind analysis challenge to infer weak gravitational lensing shear distortions from images. The primary goal was to stimulate new ideas by presenting the problem to researchers outside the shear measurement community. Six GREAT08 Team methods were presented at the launch of the Challenge and five additional groups submitted results during the 6 month competition. Participants analyzed 30 million simulated galaxies with a range in signal to noise ratio, point-spread function ellipticity, galaxy size, and galaxy type. The large quantity of simulations allowed shear measurement methods to be assessed at a level of accuracy suitable for currently planned future cosmic shear observations for the first time. Different methods perform well in different parts of simulation parameter space and come close to the target level of accuracy in several of these. A number of fresh ideas have emerged as a result of the Challenge including a re-examination of the process of combining information from different galaxies, which reduces the dependence on realistic galaxy modelling. The image simulations will become increasingly sophisticated in future GREAT challenges, meanwhile the GREAT08 simulations remain as a benchmark for additional developments in shear measurement algorithms.

We present the first lensing mass measurements of Sunyaev-Zel'dovich (SZ) selected clusters. Using optical imaging from the Southern Cosmology Survey (SCS), we present weak lensing masses for three clusters selected by their SZ emission in the South Pole Telescope survey (SPT). We confirm that the SZ selection procedure is successful in detecting mass concentrations. We also study the weak lensing signals from 38 optically-selected clusters in ~8 square degrees of the SCS survey. We fit Navarro, Frenk and White (NFW) profiles and find that the SZ clusters have amongst the largest masses, as high as 5x10^14 Msun. Using the best fit masses for all the clusters, we analytically calculate the expected SZ integrated Y parameter, which we find to be consistent with the SPT observations.

We develop optimised estimators of two sorts of power spectra for fields defined on the sky, in the presence of partial sky coverage. The first is the cross-power spectrum of two fields on the sky; the second is the skew spectrum of three fields. These can probe the Integrated Sachs Wolfe Effect (ISW) at large angular scales and the Sunyaev Z\'eldovich (SZ) effect from hot gas in clusters at small angular scales. The skew spectrum, recently introduced by Munshi & Heavens (2009), is an optimised statistic which can be tuned to study a particular form of non-Gaussianity, such as may arise in the early Universe, but which retains information on the nature of non-Gaussianity. In this paper we develop the mathematical formalism for the skew spectrum of 3 different fields. When applied to the CMB, this allows us to explore the contamination of the skew spectrum by secondary sources of CMB fluctuations. Considering the three-point function, the study of the bispectrum provides valuable information regarding cross-correlation of secondaries with lensing of CMB with much higher significance compared to just the study involving CMB sky alone. We use our analytical models to study specific cases of cosmological interest which include cross-correlating CMB with various large scale tracers to probe ISW and SZ effects for cross spectral analysis and use the formalism to study the signal-to-noise ratio for detection of the weak lensing of the CMB by cross-correlating it with different tracers as well as point sources for CMB experiments such as Planck (abridged).

The combination of current large scale structure and cosmic microwave background (CMB) anisotropies data can place strong constraints on the sum of the neutrino masses. Here we show that future cosmic shear experiments, in combination with CMB constraints, can provide the statistical accuracy required to answer questions about differences in the mass of individual neutrino species. Allowing for the possibility that masses are non-degenerate we combine Fisher matrix forecasts for a weak lensing survey like Euclid with those for the forthcoming Planck experiment. Under the assumption that neutrino mass splitting is described by a normal hierarchy we find that the combination Planck and Euclid will possibly reach enough sensitivity to put a constraint on the mass of a single species. Using a Bayesian evidence calculation we find that such future experiments could provide strong evidence for either a normal or an inverted neutrino hierachy. Finally we show that if a particular neutrino hierachy is assumed then this could bias cosmological parameter constraints, for example the dark energy equation of state parameter, by > 1\sigma, and the sum of masses by 2.3\sigma.

We study the extent to which one can distinguish primordial non--Gaussianity (NG) arising from adiabatic and isocurvature perturbations. We make a joint analysis of different NG models based on various inflationary scenarios: local-type and equilateral-type NG from adiabatic perturbations and local-type and quadratic-type NG from isocurvature perturbations together with a foreground contamination by point sources. We separate the Fisher information of the bispectrum of CMB temperature and polarization maps by l for the skew spectrum estimator introduced by Munshi & Heavens (2009) to study the scale dependence of the signal-to-noise ratio of different NG components and their correlations. We find that the adiabatic and the isocurvature modes are strongly correlated, though the phase difference of acoustic oscillations helps to distinguish them. The correlation between local-and equilateral-type is weak, but the two isocurvature modes are too strongly correlated to be discriminated. Point source contamination, to the extent to which it can be regarded as white noise, can be almost completely separated from the primordial components for l>100. Including correlations among the different components, we find that the errors of the NG parameters increase by 20-30% for the WMAP 5-year observation, but 5% for Planck observations.

The GRavitational lEnsing Accuracy Testing 2008 (GREAT08) Challenge focuses on a problem that is of crucial importance for future observations in cosmology. The shapes of distant galaxies can be used to determine the properties of dark energy and the nature of gravity, because light from those galaxies is bent by gravity from the intervening dark matter. The observed galaxy images appear distorted, although only slightly, and their shapes must be precisely disentangled from the effects of pixelisation, convolution and noise. The worldwide gravitational lensing community has made significant progress in techniques to measure these distortions via the Shear TEsting Program (STEP). Via STEP, we have run challenges within our own community, and come to recognise that this particular image analysis problem is ideally matched to experts in statistical inference, inverse problems and computational learning. Thus, in order to continue the progress seen in recent years, we are seeking an infusion of new ideas from these communities. This document details the GREAT08 Challenge for potential participants. Please visit http://www.great08challenge.info for the latest information.

We address the dual challenge of estimating deviations from Gaussianity arising in models of the Early Universe, whilst retaining information necessary to assess whether a detection of non-Gaussianity is primordial. We do this by constructing a new statistic, the bispectrum-related power spectrum, which is constructed from a map of the Cosmic Microwave Background. The estimator is optimised for primordial non-Gaussianity detection, but can also be useful in distinguishing primordial non-Gaussianity from secondary non-Gaussianity, such as may arise from unsubtracted point sources, or residuals from component separation. Extending earlier studies we present unbiased non-Gaussianity estimators optimised for partial sky coverage and inhomogeneous noise associated with realistic scan strategies, but which retain the ability to assess foreground contamination.

Much of the science case for the next generation of deep, wide-field optical/infrared surveys has been driven by the further study of dark energy. This is a laudable goal (and the subject of a companion white paper by Zhan et al.). However, one of the most important lessons of the current generation of surveys is that the interesting science questions at the end of the survey are quite different than they were when the surveys were being planned. The current surveys succeeded in this evolving terrain by being very general tools that could be applied to a number of very fundamental measurements. Likewise, the accessibility of the data enabled the broader cosmological and astronomical community to generate more science than the survey collaborations could alone. With that in mind, we should consider some of the basic physical and cosmological questions that surveys like LSST and JDEM-Wide will be able to address.

The coming decade will be an exciting period for dark energy research, during which astronomers will address the question of what drives the accelerated cosmic expansion as first revealed by type Ia supernova (SN) distances, and confirmed by later observations. The mystery of dark energy poses a challenge of such magnitude that, as stated by the Dark Energy Task Force (DETF), "nothing short of a revolution in our understanding of fundamental physics will be required to achieve a full understanding of the cosmic acceleration." The lack of multiple complementary precision observations is a major obstacle in developing lines of attack for dark energy theory. This lack is precisely what next-generation surveys will address via the powerful techniques of weak lensing (WL) and baryon acoustic oscillations (BAO) -- galaxy correlations more generally -- in addition to SNe, cluster counts, and other probes of geometry and growth of structure. Because of their unprecedented statistical power, these surveys demand an accurate understanding of the observables and tight control of systematics. This white paper highlights the opportunities, approaches, prospects, and challenges relevant to dark energy studies with wide-deep multiwavelength photometric redshift surveys. Quantitative predictions are presented for a 20000 sq. deg. ground-based 6-band (ugrizy) survey with 5-sigma depth of r~27.5, i.e., a Stage 4 survey as defined by the DETF.

The availability of high-quality spectra for a large number of galaxies in the SDSS survey allows for a more sophisticated extraction of information about their stellar populations than, e.g., the luminosity weighted age. Indeed, sophisticated and robust techniques to fully analyze galaxy spectra have now reached enough maturity as to trust their results and findings. By reconstructing the star formation and metallicity history of galaxies from the SDSS fossil record and analyzing how it relates to its environment, we have learned how to classify galaxies: to first order the evolution of a galaxy is determined by its present stellar mass, which in turn seems to be governed by the merger rate of dark halos.

We use the VESPA algorithm and spectra from the Sloan Digital Sky Survey to investigate the star formation history of the host galaxies of 257 Type Ia supernovae. We find 5$\sigma$ evidence for a short-lived population of progenitors with lifetimes of less than 180 Myr, indicating a Type Ia supernova channel arising from stars in the mass range $\sim$3.5-8 $M_\odot$. As standardizeable candles, Type Ia supernovae play an important role in determining the expansion history of the Universe, but to be useful for future cosmological surveys, the peak luminosity needs to be free of uncorrected systematic effects at the level of 1-2%. If the different progenitor routes lead to supernovae with even moderately small differences in properties, then these need to be corrected for separately, or they could lead to a systematic bias in future supernovae surveys, as the prompt route is likely to increase in importance at high redshift. VESPA analysis of hosts could be a valuable tool in this, by identifying which progenitor route is most likely.

Using the SDSS spectroscopic sample, we estimate the dark matter halo spin parameter lambda for ~53,000 disk galaxies for which MOPED star formation histories are available. We investigate the relationship between spin and total stellar mass, star formation history, and environment. First, we find a clear anti-correlation between stellar mass and spin, with low mass galaxies generally having high dark matter spins. Second, galaxies which have formed more than ~5% of their stars in the last 0.2 Gyr have more broadly distributed and typically higher spins (including a significant fraction with lambda > 0.1) than galaxies which formed a large fraction of their stars more than 10 Gyr ago. Finally, we find little or no correlation between the value of spin of the dark halo and environment as determined both by proximity to a new cluster catalog and a marked correlation study. This agrees well with the predictions from linear hierarchical torquing theory and numerical simulations.

In this brief review I consider the advances made in weak gravitational lensing over the last 8 years, concentrating on the large scales - cosmic shear. I outline the theoretical developments, observational status, and the challenges which cosmic shear must overcome to realise its full potential. Finally I consider the prospects for probing Dark Energy and extra-dimensional gravity theories with future experiments.

We present the time evolution of the stellar metallicity for SDSS galaxies, a sample that spans five orders of magnitude in stellar mass (10^7 - 10^{12} Msun). Assuming the BC03 stellar population models, we find that more massive galaxies are more metal-rich than less massive ones at all redshifts; the mass-metallicity relation is imprinted in galaxies from the epoch of formation. For galaxies with present stellar masses > 10^{10} Msun, the time evolution of stellar metallicity is very weak, with at most 0.2-0.3 dex over a Hubble time- for this reason the mass-metallicity relation evolves little with redshift. However, for galaxies with present stellar masses < 10^{10} Msun, the evolution is significant, with metallicity increasing by more than a decade from redshift 3 to the present. By being able to recover the metallicity history, we have managed to identify the origin of a recent discrepancy between the metallicity recovered from nebular lines and absorption lines. As expected, we show that the young population dominates the former while the old population the latter. We have investigated the dependence on the stellar models used and find that older stellar population synthesis codes do not produce a clear result. Finally, we have explored the relationship between cluster environment and metallicity, and find a strong correlation in the sense that galaxies in high density regions have high metallicity.

Using the star formation rates from the SDSS galaxy sample, extracted using the MOPED algorithm, and the empirical Kennicutt law relating star formation rate to gas density, we calculate the time evolution of the gas fraction as a function of the present stellar mass. We show how the gas-to-stars ratio varies with stellar mass, finding good agreement with previous results for smaller samples at the present epoch. For the first time we show clear evidence for progressive gas loss with cosmic epoch, especially in low-mass systems. We find that galaxies with small stellar masses have lost almost all of their cold baryons over time, whereas the most massive galaxies have lost little. Our results also show that the most massive galaxies have evolved faster and turned most of their gas into stars at an early time, thus strongly supporting a downsizing scenario for galaxy evolution.

We extend the Bayesian model fitting shape measurement method presented in Miller et al. (2007) and use the method to estimate the shear from the Shear TEsting Programme simulations (STEP). The method uses a fast model fitting algorithm which uses realistic galaxy profiles and analytically marginalises over the position and amplitude of the model by doing the model fitting in Fourier space. This is used to find the full posterior probability in ellipticity so that the shear can be estimated in a fully Bayesian way. The Bayesian shear estimation allows measurement bias arising from the presence of random noise to be removed. In this paper we introduce an iterative algorithm that can be used to estimate the intrinsic ellipticity prior and show that this is accurate and stable. By using the method to estimate the shear from the STEP1 simulations we find the method to have a shear bias of m ~ 0.005 and a variation in shear offset with PSF type of sigma_c ~ 0.0002. These values are smaller than for any method presented in the STEP1 publication that behaves linearly with shear. Using the method to estimate the shear from the STEP2 simulations we find than the shear bias and offset are m ~ 0.002 and c ~ -0.0007 respectively. In addition we find that the bias and offset are stable to changes in magnitude and size of the galaxies. Such biases should yield any cosmological constraints from future weak lensing surveys robust to systematic effects in shape measurement.

In this paper we investigate the potential of 3D cosmic shear to constrain massive neutrino parameters. We find that if the total mass is substantial (near the upper limits from LSS, but setting aside the Ly alpha limit for now), then 3D cosmic shear + Planck is very sensitive to neutrino mass and one may expect that a next generation photometric redshift survey could constrain the number of neutrinos N_nu and the sum of their masses m_nu to an accuracy of dN_nu ~ 0.08 and dm_nu ~ 0.03 eV respectively. If in fact the masses are close to zero, then the errors weaken to dN_nu ~ 0.10 and dm_nu~0.07 eV. In either case there is a factor 4 improvement over Planck alone. We use a Bayesian evidence method to predict joint expected evidence for N_nu and m_nu. We find that 3D cosmic shear combined with a Planck prior could provide `substantial' evidence for massive neutrinos and be able to distinguish `decisively' between many competing massive neutrino models. This technique should `decisively' distinguish between models in which there are no massive neutrinos and models in which there are massive neutrinos with |N_nu-3| > 0.35 and m_nu > 0.25 eV. We introduce the notion of marginalised and conditional evidence when considering evidence for individual parameter values within a multi-parameter model.

We present an investigation into the potential effect of systematics inherent in multi-band wide field surveys on the dark energy equation of state determination for two 3D weak lensing methods. The weak lensing methods are a geometric shear-ratio method and 3D cosmic shear. The analysis here uses an extension of the Fisher matrix framework to jointly include photometric redshift systematics, shear distortion systematics and intrinsic alignments. We present results for DUNE and Pan-STARRS surveys. We show that assuming systematic parameters are fixed, but possibly biased, results in potentially large biases in dark energy parameters. We quantify any potential bias by defining a Bias Figure of Merit. We also show the effect on the dark energy Figure of Merit of marginalising over each systematic parameter individually. We find that the largest effect on the Figure of Merit comes from uncertainty in the photometric redshift systematic parameters. These can reduce the Figure of Merit by up to a factor of 2 to 4 in both 3D weak lensing methods, if no informative prior on the systematic parameters is applied. Shear distortion systematics have a smaller overall effect. Intrinsic alignment effects can reduce the Figure of Merit by up to a further factor of 2. This, however, is a worst case scenario. By including prior information on systematic parameters the Figure of Merit can be recovered to a large extent. We conclude that, as a rule of thumb, given a realistic current understanding of intrinsic alignments and photometric redshifts, then including all three primary systematic effects reduces the Figure of Merit by at most a factor of 2, but that in reality this factor should be much less. [abridged]

The principles of measuring the shapes of galaxies by a model-fitting approach are discussed in the context of shape-measurement for surveys of weak gravitational lensing. It is argued that such an approach should be optimal, allowing measurement with maximal signal-to-noise, coupled with estimation of measurement errors. The distinction between likelihood-based and Bayesian methods is discussed. Systematic biases in the Bayesian method may be evaluated as part of the fitting process, and overall such an approach should yield unbiased shear estimation without requiring external calibration from simulations. The principal disadvantage of model-fitting for large surveys is the computational time required, but here an algorithm is presented that enables large surveys to be analysed in feasible computation times. The method and algorithm is tested on simulated galaxies from the Shear TEsting Program (STEP).

We introduce VErsatile SPectral Analysis (VESPA): a new method which aims to recover robust star formation and metallicity histories from galactic spectra. VESPA uses the full spectral range to construct a galaxy history from synthetic models. We investigate the use of an adaptative parametrization grid to recover reliable star formation histories on a galaxy-by-galaxy basis. Our goal is robustness as opposed to high resolution histories, and the method is designed to return high time resolution only where the data demand it. In this paper we detail the method and we present our findings when we apply VESPA to synthetic and real Sloan Digital Sky Survey (SDSS) spectroscopic data. We show that the number of parameters that can be recovered from a spectrum depends strongly on the signal-to-noise, wavelength coverage and presence or absence of a young population. For a typical SDSS sample of galaxies, we can normally recover between 2 to 5 stellar populations. We find very good agreement between VESPA and our previous analysis of the SDSS sample with MOPED.

The Fisher matrix approach (Fisher 1935) allows one to calculate in advance how well a given experiment will be able to estimate model parameters, and has been an invaluable tool in experimental design. In the same spirit, we present here a method to predict how well a given experiment can distinguish between different models, regardless of their parameters. From a Bayesian viewpoint, this involves computation of the Bayesian evidence. In this paper, we generalise the Fisher matrix approach from the context of parameter fitting to that of model testing, and show how the expected evidence can be computed under the same simplifying assumption of a gaussian likelihood as the Fisher matrix approach for parameter estimation. With this `Laplace approximation' all that is needed to compute the expected evidence is the Fisher matrix itself. We illustrate the method with a study of how well upcoming and planned experiments should perform at distinguishing between Dark Energy models and modified gravity theories. In particular we consider the combination of 3D weak lensing, for which planned and proposed wide-field multi-band imaging surveys will provide suitable data, and probes of the expansion history of the Universe, such as proposed supernova and baryonic acoustic oscillations surveys. We find that proposed large-scale weak lensing surveys from space should be able readily to distinguish General Relativity from modified gravity models.

We present the first application of the 3D cosmic shear method developed in Heavens et al. (2006) and the geometric shear-ratio analysis developed in Taylor et al. (2006), to the COMBO-17 data set. 3D cosmic shear has been used to analyse galaxies with redshift estimates from two random COMBO-17 fields covering 0.52 square degrees in total, providing a conditional constraint in the (sigma_8, Omega_m) plane as well as a conditional constraint on the equation of state of dark energy, parameterised by a constant w= p/rho c^2. The (sigma_8, Omega_m) plane analysis constrained the relation between sigma_8 and Omega_m to be sigma_8(Omega_m/0.3)^{0.57 +- 0.19}=1.06 +0.17 -0.16, in agreement with a 2D cosmic shear analysis of COMBO-17. The 3D cosmic shear conditional constraint on w using the two random fields is w=-1.27 +0.64 -0.70. The geometric shear-ratio analysis has been applied to the A901/2 field, which contains three small galaxy clusters. Combining the analysis from the A901/2 field, using the geometric shear-ratio analysis, and the two random fields, using 3D cosmic shear, w is conditionally constrained to w=-1.08 +0.63 -0.58. The errors presented in this paper are shown to agree with Fisher matrix predictions made in Heavens et al. (2006) and Taylor et al. (2006). When these methods are applied to large datasets, as expected soon from surveys such as Pan-STARRS and VST-KIDS, the dark energy equation of state could be constrained to an unprecedented degree of accuracy.

We present the results of a MOPED analysis of ~3 x 10^5 galaxy spectra from the Sloan Digital Sky Survey Data Release Three (SDSS DR3), with a number of improvements in data, modelling and analysis compared with our previous analysis of DR1. The improvements include: modelling the galaxies with theoretical models at a higher spectral resolution of 3\AA; better calibrated data; an extended list of excluded emission lines, and a wider range of dust models. We present new estimates of the cosmic star formation rate, the evolution of stellar mass density and the stellar mass function from the fossil record. In contrast to our earlier work the results show no conclusive peak in the star formation rate out to a redshift around 2 but continue to show conclusive evidence for `downsizing' in the SDSS fossil record. The star formation history is now in good agreement with more traditional instantaneous measures. The galaxy stellar mass function is determined over five decades of mass, and an updated estimate of the current stellar mass density is presented. We also investigate the systematic effects of changes in the stellar population modelling, the spectral resolution, dust modelling, sky lines, spectral resolution and the change of data set. We find that the main changes in the results are due to the improvements in the calibration of the SDSS data, changes in the initial mass function and the theoretical models used.

We carry out a Bayesian model selection analysis of different dark energy parametrizations using the recent luminosity distance data of high redshift supernovae from Riess et al. 2007 and from the new ESSENCE Supernova Survey. Including complementary cosmological datasets, we found substantial evidence ($\Delta \ln (E) \sim 1$) against a time-varying dark energy equation of state parameter, and against phantom dark energy models. We find a small preference for a standard cosmological constant over accelerating non-phantom models where w is constant, but allowed to vary in the range -1 to -0.33.

Weak gravitational lensing is responsible for the shearing and magnification of the images of high-redshift sources due to the presence of intervening matter. The distortions are due to fluctuations in the gravitational potential, and are directly related to the distribution of matter and to the geometry and dynamics of the Universe. As a consequence, weak gravitational lensing offers unique possibilities for probing the Dark Matter and Dark Energy in the Universe. In this review, we summarise the theoretical and observational state of the subject, focussing on the statistical aspects of weak lensing, and consider the prospects for weak lensing surveys in the future. Weak gravitational lensing surveys are complementary to both galaxy surveys and cosmic microwave background (CMB) observations as they probe the unbiased non-linear matter power spectrum at modest redshifts. Most of the cosmological parameters are accurately estimated from CMB and large-scale galaxy surveys, so the focus of attention is shifting to understanding the nature of Dark Matter and Dark Energy. On the theoretical side, recent advances in the use of 3D information of the sources from photometric redshifts promise greater statistical power, and these are further enhanced by the use of statistics beyond two-point quantities such as the power spectrum. The use of 3D information also alleviates difficulties arising from physical effects such as the intrinsic alignment of galaxies, which can mimic weak lensing to some extent. (Abridged)

We present parameter estimation forecasts for present and future 3D cosmic shear surveys. We demonstrate that, in conjunction with results from cosmic microwave background (CMB) experiments, the properties of dark energy can be estimated with very high precision with large-scale, fully 3D weak lensing surveys. In particular, a 5-band, 10,000 square degree ground-based survey to a median redshift of zm=0.7 could achieve 1-$\sigma$ marginal statistical errors, in combination with the constraints expected from the CMB Planck Surveyor, of $\Delta$w0=0.108 and $\Delta$wa=0.099 where we parameterize w by w(a)=w0+wa(1-a) where a is the scale factor. Such a survey is achievable with a wide-field camera on a 4 metre class telescope. The error on the value of w at an intermediate pivot redshift of z=0.368 is constrained to $\Delta$w(z=0.368)=0.0175. We compare and combine the 3D weak lensing constraints with the cosmological and dark energy parameters measured from planned Baryon Acoustic Oscillation (BAO) and supernova Type Ia experiments, and find that 3D weak lensing significantly improves the marginalized errors. A combination of 3D weak lensing, CMB and BAO experiments could achieve $\Delta$w0=0.037 and $\Delta$wa=0.099. Fully 3D weak shear analysis avoids the loss of information inherent in tomographic binning, and we show that the sensitivity to systematic errors is much less. In conjunction with the fact that the physics of lensing is very soundly based, this analysis demonstrates that deep, wide-angle 3D weak lensing surveys are extremely promising for measuring dark energy properties.

We adapt the Jain--Taylor (2003) shear-ratio geometric lensing method to measure the dark energy equation of state, and its time derivative from dark matter haloes in cosmologies with arbitrary spatial curvature. The full shear-ratio covariance matrix is calculated for lensed sources, including the intervening large-scale structure and photometric redshift errors as additional sources of noise, and a maximum likelihood method for applying the test is presented. Combining with the expected results from the CMB we design an optimal survey for probing dark energy. A targeted survey imaging 60 of the largest clusters in a hemisphere with 5-band optical photometric redshifts to a median galaxy depth of zm=0.9 could measure w0 to a marginal 1-sigma error of $\Delta$w0=0.5. We marginalize over all other parameters including wa, where the equation of state is parameterized in terms of scale factor a as w(a)=w0+wa(1-a). For higher accuracy a large-scale photometric redshift survey is required. Such a near-future 5-band survey covering 10,000 square degrees to z_m=0.7 could measure w0 to $\Delta$w0=0.075 and $\Delta$wa=0.33. A stronger combined constraint is measured at a pivot redshift zp=0.27 of $\Delta$w(zp)=0.0298. We compare and combine the geometric test with the cosmological and dark energy parameters measured from planned Baryon Acoustic Oscillation (BAO) and supernova Type Ia experiments, and find that the geometric test results combine with a significant reduction in errors due to different degeneracies. A combination of geometric lensing, CMB and BAO experiments could achieve a pivot redshift constraint of $\Delta$w(zp)=0.020 at zp=0.62. Simple relations are presented that show how our lensing results can be scaled to other telescope classes and survey parameters.

We present the first results from a study designed to test whether, given high-quality spectrophotometry spanning the mid-UV--optical wavelength regime, it is possible to distinguish the metal content (Z) and star-formation history (sfh) of individual elliptical galaxies with sufficient accuracy to establish whether their formation history is linked to their detailed morphology and position on the Fundamental Plane. From a detailed analysis of UV-optical spectrophotometry of the `cuspy' elliptical galaxy NGC 3605 and the giant elliptical NGC 5018 we find that: 1) optical spectra with l > 3500 A may not contain sufficient data to robustly uncover all the stellar populations present in individual galaxies, even in such relatively passive objects as ellipticals, 2) the addition of the UV data approaching l = 2500 A holds the key to establishing well-constrained sfhs, from which we can infer a formation and evolution history which is consistent with their photometric properties, 3) despite the superficial similarity of their spectra, the two galaxies have very different `recent' sfhs -- the smaller, cuspy elliptical NGC 3605 contains a high-Z population of age ~= 1 Gyr, and has a position on the fundamental plane typical of the product of a low-z gas-rich merger (most likely at z ~ 0.08), while the giant elliptical NGC 5018, with a sub-solar secondary population, appears to have gained its more recent stars via mass transfer / accretion of gas from its spiral companion, 4) despite these differences in detailed history, more than 85% of the stellar mass of both galaxies is associated with an old (9-12 Gyr) stellar population of near-solar Z. This pilot study provides strong motivation for the construction and analysis of high-quality UV-optical spectra for a substantial sample of ellipticals spanning the Fundamental Plane.

We use the spectra of ~ 22,000 early-type galaxies, selected from the Sloan Digital Sky Survey, to infer the ages, metallicities and star formation histories of these galaxies. We find clear evidence of "downsizing", i.e. galaxies with larger velocity dispersion have older stellar populations. In particular, most early-type galaxies with velocity dispersion exceeding 200 km s-1 formed more than 90% of their current stellar mass at redshift z > 2.5. Therefore, star formation was suppressed around this redshift. We also show that chemical enrichment was rapid, lasting 1-2 Gyr and find evidence that [Fe/H] is sub-solar. We study the robustness of these results by comparing three different approaches: using (i) Lick absorption line indices; (ii) fitting a single-burst stellar population model to the whole spectrum (lines+continuum); and (iii) reconstructing the star formation and metallicity histories in multiple age-bins, providing a method to measure mass-weighted ages and metallicities. We find good agreement between the luminosity-weighted ages and metallicities computed with these three methods.

We investigate the expected correlation between the weak gravitational shear of distant galaxies and the orientation of foreground galaxies, through the use of numerical simulations. This shear-ellipticity correlation can mimic a cosmological weak lensing signal, and is potentially the limiting physical systematic effect for cosmology with future high-precision weak lensing surveys. We find that, if uncorrected, the shear-ellipticity correlation could contribute up to 10% of the weak lensing signal on scales up to 20 arcminutes, for lensing surveys with a median depth z=1. The most massive foreground galaxies are expected to cause the largest correlations, a result also seen in the Sloan Digital Sky Survey. We find that the redshift dependence of the effect is proportional to the lensing efficiency of the foreground, and this offers prospects for removal to high precision, although with some model dependence. The contamination is characterised by a weakly negative B-mode, which can be used as a diagnostic of systematic errors. We also provide more accurate predictions for a second potential source of error, the intrinsic alignment of nearby galaxies. This source of contamination is less important, however, as it can be easily removed with distance information.

We present a mark correlation analysis of the galaxies in the Sloan Digital Sky Survey using weights provided by MOPED. The large size of the sample permits statistically significant statements about how galaxies with different metallicities and star formation histories are spatially correlated. Massive objects formed a larger fraction of their stars at higher redshifts and over shorter timescales than did less massive objects (sometimes called down-sizing). We find that those galaxies which dominated the cosmic star formation at z~3 are predominantly in clusters today, whereas galaxies which dominate the star formation at z~0 inhabit substantially lower mass objects in less dense regions today. Hence, our results indicate that star formation and chemical enrichment occured first in the denser regions of the Universe, and moved to less dense regions at later times.

We present a search for non-Gaussianity in the WMAP first-year data using the two-point correlation function of maxima and minima in the temperature map. We find evidence for non-Gaussianity on large scales, whose origin appears to be associated with unsubstracted foregrounds, but which is not entirely clear. The signal appears to be associated most strongly with cold spots, and is more pronounced in the Southern galactic hemisphere. Removal of the region of sky near the galactic plane, or filtering out large-scale modes removes the signal. Analysis of individual frequency maps shows strongest signal in the 41GHz Q band. A study of difference maps tests the hypothesis that the non-Gaussianity is due to residual foregrounds and noise, but shows no significant detection. We suggest that the detection is due to large-scale residual foregrounds affecting more than one frequency band, but a primordial contribution from the Cosmic Microwave Background cannot be excluded.

We present a comprehensive full-sky 3-dimensional analysis of the weak-lensing fields and their corresponding power spectra. Using the formalism of spin-weight spherical harmonics and spherical Bessel functions, we relate the two-point statistics of the harmonic expansion coefficients of the weak lensing shear and convergence to the power spectrum of the matter density perturbations, and derive small-angle limits. Such a study is relevant in view of the next generation of large-scale weak lensing surveys which will provide distance information about the sources through photometric redshifts. This opens up the possibility of accurate cosmological parameter estimation via weak lensing, with an emphasis on the equation of state of dark energy.

Using the spectroscopic sample of the SDSS DR1 we measure how gas was transformed into stars as a function of time and stellar mass: the baryonic conversion tree (BCT). There is a clear correlation between early star formation activity and present-day stellar mass: the more massive galaxies have formed about 80% of their stars at $z>1$, while for the less massive ones the value is only about 20%. By comparing the BCT to the dark matter merger tree, we find indications that star formation efficiency at $z>1$ had to be about a factor of two higher than today ($\sim 10%$) in galaxies with present-day stellar mass larger than $2 \times 10^{11}M_\odot$, if this early star formation occurred in the main progenitor. Therefore, the LCDM paradigm can accommodate a large number of red objects. On the other hand, in galaxies with present-day stellar mass less than $10^{11}$ M$_{\odot}$, efficient star formation seems to have been triggered at $z \sim 0.2$. We show that there is a characteristic mass (M$_* \sim 10^{10}$ M$_{\odot}$) for feedback efficiency (or lack of star formation). For galaxies with masses lower than this, feedback (or star formation suppression) is very efficient while for higher masses it is not. The BCT, determined here for the first time, should be an important observable with which to confront theoretical

We present the result of a decomposition of the 2dFGRS galaxy overdensity field into an orthonormal basis of spherical harmonics and spherical Bessel functions. Galaxies are expected to directly follow the bulk motion of the density field on large scales, so the absolute amplitude of the observed large-scale redshift-space distortions caused by this motion is expected to be independent of galaxy properties. By splitting the overdensity field into radial and angular components, we linearly model the observed distortion and obtain the cosmological constraint Omega_m^{0.6} sigma_8=0.46+/-0.06. The amplitude of the linear redshift-space distortions relative to the galaxy overdensity field is dependent on galaxy properties and, for L_* galaxies at redshift z=0, we measure beta(L_*,0)=0.58+/-0.08, and the amplitude of the overdensity fluctuations b(L_*,0) sigma_8=0.79+/-0.03, marginalising over the power spectrum shape parameters. Assuming a fixed power spectrum shape consistent with the full Fourier analysis produces very similar parameter constraints.

Using the MOPED algorithm we determine non-parametrically the Stellar Mass Function of 96,545 galaxies from the Sloan Digital Sky Survey data release one. By using the reconstructed spectrum due to starlight we can eliminate contamination from either emission lines or AGN components. Our results give excellent agreement with previous works, but extend their range by more than two decades in mass to 10^{7.5} < M_s/h^{-2}M_\odot < 10^{12}. We present both a standard Schechter fit and a fit modified to include an extra, high-mass contribution, possibly from cluster cD galaxies. The Schechter fit parameters are $\phi^*=(7.7\pm 0.8)\times 10^{-3} h^3 Mpc^{-3}$, $M^*=(7.53 \pm 0.04) \times 10^{10} h^{-2}M_\odot$ and $\alpha=-1.167\pm 0.004$. Our sample also yields an estimate for the contribution from baryons in stars to the critical density of $\Omega_{b*}h=(2.40 \pm 0.04) \times 10^{-3}$, in good agreement with other indicators. Error bars are statistical and a Salpeter IMF is assumed throughout. We find no evolution of the mass function in the redshift range $0.05 < z < 0.34$ indicating that almost all present day stars were already formed at $z \sim 0.34$ with little or no star formation activity since then and that the evolution seen in the luminosity function must be largely due to stellar fading.

We use the LUQAS sample of 27 high resolution, high signal-to-noise QSO absorption spectra (Kim et al. 2004) and the results from Croft et al. (2002) together with a suite of high-resolution hydro-dynamical simulations run with the GADGET-II code, to infer the linear dark matter power spectrum on scales of \~ 0.3-30 Mpc at z=2.125 and at z=2.72.

The determination of the star-formation history of the Universe is a key goal of modern cosmology, as it is crucial to our understanding of how structure in the Universe forms and evolves. A picture has built up over recent years, piece-by-piece, by observing young stars in distant galaxies at different times in the past. These studies indicated that the stellar birthrate peaked some 8 billion years ago, and then declined by a factor of around ten to its present value. Here we report on a new study which obtains the complete star formation history by analysing the fossil record of the stellar populations of 96545 nearby galaxies. Broadly, our results support those derived from high-redshift galaxies elsewhere in the Universe. We find, however, that the peak of star formation was more recent - around 5 billion years ago. Our study also shows that the bigger the stellar mass of the galaxy, the earlier the stars were formed. This striking result indicates a very different formation history for high- and low-mass formation.

We present a determination of the bispectrum of the flux in the Lyman-alpha forest of QSO absorption spectra obtained from LUQAS which consists of spectra observed with the high resolution Echelle spectrograph UVES. Typical errors on the observed bispectrum as obtained from a jack-knife estimator are ~ 50%. For wavenumbers in the range 0.03 s/km <k < 0.1 s/km the observed bispectrum agrees within the errors with that of the synthetic absorption spectra obtained from numerical hydro-simulations of a LCDM model with and without feedback from star formation. Including galactic feedback changes the bispectrum by less than 10%. At smaller wavenumbers the associated metal absorption lines contribute about 50% to the bispectrum and the observed bispectrum exceeds that of the simulations. At wavenumbers k< 0.03 (s/km) second-order perturbation theory applied to the flux spectrum gives a reasonable (errors smaller than 30%) approximation to the bispectra of observed and simulated absorption spectra. The bispectrum of the observed absorption spectra also agrees, within the errors, with that of a randomized set of absorption spectra where a random shift in wavelength has been added to absorption lines identified with VPFIT. This suggests that for a sample of the size presented here, the errors on the bispectrum are too large to discriminate between models with very different 3D distribution of Lyman-alpha absorption. If it were possible to substantially reduce these errors for larger samples of absorption spectra, the bispectrum might become an important statistical tool for probing the growth of gravitational structure in the Universe at redshift z>2.

We present a method for removal of the contaminating effects of intrinsic galaxy alignments, in measurements of cosmic shear from multi-colour weak lensing surveys. The method down-weights pairs which are likely to be close in three dimensions, based on spectroscopic or photometric redshifts. Results are dramatic: the intrinsic contamination of the low-redshift Sloan photometric redshift survey could be 80 times the lensing signal, but this can be almost completely removed, leaving random shot noise errors of the order of 10%. Intrinsic galaxy alignments, although an annoying contaminant for cosmic shear studies, are interesting in their own right, and we therefore present a new observational constraint for their amplitude from the aperture mass B-mode in the Red-Sequence Cluster Survey. The small measured B-mode rules out the published intrinsic alignment models from numerical simulations, which assume no evolution in galaxy clustering.

We estimate and remove the contamination of weak gravitational lensing measurements by the intrinsic alignment of close pairs of galaxies. We do this by investigating both the aperture mass B mode statistic, and the shear correlations of close and distant pairs of galaxies. We re-analyse the COMBO-17 survey, and study published results from the RCS and the VIRMOS-DESCART survey, concluding that the intrinsic alignment effect is at the lower end of the range of theoretical predictions. We also revisit this theoretical issue, and show that misalignment of baryon and halo angular momenta may be an important effect which can reduce the intrinsic ellipticity correlations estimated from numerical simulations to the level that we and the SuperCOSMOS survey observe. We re-examine the cosmological parameter estimation from the COMBO-17 survey, now marginalising over the Hubble constant. Assuming no evolution in galaxy clustering, and marginalising over the intrinsic alignment signal, we find the mass clustering amplitude is reduced by 0.03 to sigma_8(Omega_m / 0.27)^0.6 = 0.71 +/- 0.11. We consider the forthcoming SNAP wide weak lensing survey, and the CFHTLS wide synoptic survey, and expect them to be contaminated on scales >1 arcmin by intrinsic alignments at the level of ~1% and ~2% respectively. Division of the SNAP survey for lensing tomography significantly increases the contamination in the lowest redshift bin to ~7% and possibly higher. Removal of the intrinsic alignment effect by the downweighting of nearby galaxy pairs will therefore be vital for SNAP, (abridged).

Using MOPED we determine non-parametrically the star-formation and metallicity history of over 37,000 high-quality galaxy spectra from the Sloan Digital Sky Survey (SDSS) early data release. We use the entire spectral range, rather than concentrating on specific features, and we estimate the complete star formation history without prior assumptions about its form (by constructing so-called `population boxes'). The main results of this initial study are that the star formation rate in SDSS galaxies has been in decline for ~6 Gyr; a metallicity distribution for star-forming gas which is peaked ~3 Gyr ago at about solar metallicity, inconsistent with closed-box models, but consistent with infall models. We also determine the infall rate of gas in SDSS and show that it has been significant for the last 3 Gyr. We investigate errors using a Monte-Carlo Markov Chain algorithm. Further, we demonstrate that recovering star formation and metallicity histories for such a large sample becomes intractable without data compression methods, particularly the exploration of the likelihood surface. By exploring the whole likelihood surface we show that age-metallicity degeneracies are not as severe as by using only a few spectral features. We find that 65% of galaxies contain a significant old population (with an age of at least 8 Gyr), including recent starburst galaxies, and that over 97% have some stars older than 2 Gyr. It is the first time that the past star formation history has been determined from the fossil record of the present-day spectra of galaxies.

I propose an analysis method, based on spin-spherical harmonics and spherical Bessel functions, for large-scale weak lensing surveys which have source distance information through photometric redshifts. I show that the distance information can significantly reduce statistical errors on cosmological parameters; in particular, 3D lensing analysis offers excellent prospects for constraining the equation of state of the vacuum energy which dominates the energy density of the Universe. I show that the ratio of pressure to energy density could be determined to an accuracy of $\sim 1%$ or better. Having distance information also offers significant advantages in the control of systematic effects such as the intrinsic alignment of galaxies. The case for obtaining photometric redshifts is therefore compelling. A signal-to-noise eigenmode analysis of the modes shows that the modes with highest signal-to-noise correspond quite closely to ignoring the redshift information, but there is significant extra information from a few radial modes. These modes are generally long-wavelength, suggesting that useful information can be gleaned even if the photometric redshifts are relatively inaccurate.

By studying the bispectrum of the galaxy distribution in the 2dF galaxy redhsift survey (2dFGRS), we have shown that 2dFGRS galaxies are unbiased tracers of the mass distribution. This allows us the break the degeneracy intrinsic to power spectrum studies, between the matter density parameter Omega_m and the bias parameter b, and to obtain an accurate measurement of Omega_m: Omega_{m,z_{eff}}=0.27 \pm 0.06, a measurement obtained from the 2dFGRS alone, independently from other data sets. This result has to be interpreted as Omega_m at the effective redshift of the survey z_{eff}=0.17. Extrapolated at z=0 we obtain Omega_m=0.2 \pm 0.06 for the LCDM model. This constraint on the matter density parameter when combined with cosmic microwave background constraints on the flatness of the Universe, show firm evidence that the Universe is dominated by a vacuum energy component.

We describe different methods for estimating the bispectrum of Cosmic Microwave Background data. In particular we construct a minimum variance estimator for the flat-sky limit and compare results with previously-studied frequentist methods. Application to the MAXIMA dataset shows consistency with primordial Gaussianity. Weak quadratic non-Gaussianity is characterised by a tunable parameter $f_{NL}$, corresponding to non-Gaussianity at a level $\sim 10^{-5}f_{NL}$ (ratio of non-Gaussian to Gaussian terms), and we find limits of $|f_{NL}|<950$ for the minimum-variance estimator and $|f_{NL}|<1650$ for the usual frequentist estimator. These are the tightest limits on primordial non-Gaussianity which include the full effects of the radiation transfer function.

Intrinsic alignments of galaxies can mimic to an extent the effects of shear caused by weak gravitational lensing. Previous studies have shown that for shallow surveys with median redshifts z_m = 0.1, the intrinsic alignment dominates the lensing signal. For deep surveys with z_m = 1, intrinsic alignments are believed to be a significant contaminant of the lensing signal, preventing high-precision measurements of the matter power spectrum. In this paper we show how distance information, either spectroscopic or photometric redshifts, can be used to down-weight nearby pairs in an optimised way, to reduce the errors in the shear signal arising from intrinsic alignments. Provided a conservatively large intrinsic alignment is assumed, the optimised weights will essentially remove all traces of contamination. For the Sloan spectroscopic galaxy sample, residual shot noise continues to render it unsuitable for weak lensing studies. However, a dramatic improvement for the slightly deeper Sloan photometric survey is found, whereby the intrinsic contribution, at angular scales greater than 1 arcminute, is reduced from about 80 times the lensing signal to a 10% effect. For deeper surveys such as the COMBO-17 survey with z_m = 0.6, the optimisation reduces the error from a largely systematic 220% error at small angular scales to a much smaller and largely statistical error of only 17% of the expected lensing signal. We therefore propose that future weak lensing surveys be accompanied by the acquisition of photometric redshifts, in order to remove fully the unknown intrinsic alignment errors from weak lensing detections.

We explore whether the rest-frame near-UV spectral region, observable in high-redshift galaxies via optical spectroscopy, contains sufficient information to allow the degeneracy between age and metallicity to be lifted. We do this by testing the ability of evolutionary synthesis models to reclaim the correct metallicity when fitted to the near-UV spectra of F stars of known (sub-solar and super-solar) metallicity. F stars are of particular interest because the rest-frame near-UV spectra of the oldest known elliptical galaxies at z > 1 appear to be dominated by F stars near to the main-sequence turnoff. We find that, in the case of the F stars, where the HST ultraviolet spectra have high signal:noise, model-fitting with metallicity allowed to vary as a free parameter is rather successful at deriving the correct metallicity. As a result, the estimated turnoff ages of these stars yielded by the model fitting are well constrained. Encouraged by this we have fitted these same variable- metallicity models to the deep, optical spectra of the z \simeq 1.5 mJy radio galaxies 53W091 and 53W069 obtained with the Keck telescope. While the age-metallicity degeneracy is not so easily lifted for these galaxies, we find that even when metallicity is allowed as a free parameter, the best estimates of their ages are still \geq 3 Gyr, with ages younger than 2 Gyr now strongly excluded. Furthermore, we find that a search of the entire parameter space of metallicity and star formation history using MOPED (Heavens et al., 2000) leads to the same conclusion. Our results therefore continue to argue strongly against an Einstein-de Sitter universe, and favour a lambda-dominated universe in which star formation in at least these particular elliptical galaxies was completed somewhere in the redshift range z = 3 - 5.

We calculate the bispectrum of the gravitational field fluctuations generated during warm inflation, where dissipation of the vacuum potential during inflation is the mechanism for structure formation. The bispectrum is non--zero because of the self--interaction of the scalar field. We compare the predictions with those of standard, or `supercooled', inflationary models, and consider the detectability of these levels of non--Gaussianity in the bispectrum of the cosmic microwave background. We find that the levels of non--Gaussianity for warm and supercooled inflation are comparable, and over--ridden by the contribution to the bispectrum due to other physical effects. We also conclude that the resulting bispectrum values will be undetectable in the cosmic microwave background for both the MAP and Planck Surveyor satellites.

The statistical properties of a map of the primary fluctuations in the cosmic microwave background (CMB) may be specified to high accuracy by a few thousand power spectra measurements, provided the fluctuations are gaussian, yet the number of parameters relevant for the CMB is probably no more than about 10-20. There is consequently a large degree of redundancy in the power spectrum data. In this paper, we show that the MOPED data compression technique can reduce the CMB power spectrum measurements to about 10-20 numbers (one for each parameter), from which the cosmological parameters can be estimated virtually as accurately as from the complete power spectrum. This offers opportunities for very fast parameter estimation from real and simulated CMB skies, with accurate likelihood calculations at Planck resolution being speeded up by a factor of around five hundred million.

We compute the bispectrum of the 2dF Galaxy Redshift Survey (2dFGRS) and use it to measure the bias parameter of the galaxies. This parameter quantifies the strength of clustering of the galaxies relative to the mass in the Universe. By analysing 80 million triangle configurations in the wavenumber range 0.1 < k < 0.5 h/Mpc (i.e. on scales roughly between 5 and 30 Mpc/h) we find that the linear bias parameter is consistent with unity: b_1=1.04 pm 0.11, and the quadratic (nonlinear) bias is consistent with zero: b_2=-0.054 pm 0.08. Thus, at least on large scales, optically-selected galaxies do indeed trace the underlying mass distribution. The bias parameter can be combined with the 2dFGRS measurement of the redshift distortion parameter beta = Omega_m^{0.6}/b_1, to yield Omega_m = 0.27 pm 0.06 for the matter density of the Universe, a result which is determined entirely from this survey, independently of other datasets. Our measurement of the matter density of the Universe should be interpreted as Omega_m at the effective redshift of the survey (z=0.17).

We present results from a deep mid-IR survey of the Hubble Deep Field South (HDF-S) region performed at 7 and 15um with the CAM instrument on board ISO. We found reliable optical/near-IR associations for 32 of the 35 sources detected in this field by Oliver et al. (2002, Paper I): eight of them were identified as stars, one is definitely an AGN, a second seems likely to be an AGN, too, while the remaining 22 appear to be normal spiral or starburst galaxies. Using model spectral energy distributions (SEDs) of similar galaxies, we compare methods for estimating the star formation rates (SFRs) in these objects, finding that an estimator based on integrated (3-1000um) IR luminosity reproduces the model SFRs best. Applying this estimator to model fits to the SEDs of our 22 spiral and starburst galaxies, we find that they are forming stars at rates of ~1-100 M_sol/yr, with a median value of ~40M_sol/yr, assuming an Einstein - de Sitter universe with a Hubble constant of 50 km/s/Mpc, and star formation taking place according to a Salpeter (1955) IMF across the mass range 0.1-100M_sol. We split the redshift range 0.0<z<0.6 into two equal-volume bins to compute raw estimates of the star formation rate density contributed by these sources, assuming the same cosmology and IMF as above and computing errors based on estimated uncertainties in the SFRs of individual galaxies. We compare these results with other estimates of the SFR density made with the same assumptions, showing them to be consistent with the results of Flores et al. (1999) from their ISO survey of the CFRS 1415+52 field. However, the relatively small volume of our survey means that our SFR density estimates suffer from a large sampling variance, implying that our results, by themselves, do not place tight constraints on the global mean SFR density.

Gravitational lensing causes background galaxy images to become aligned, and the statistical characteristics of the image alignments can then be used to constrain the power spectrum of mass fluctuations. Analyses of gravitational lensing assume that intrinsic galaxy alignments are negligible, but if this assumption does not hold, then the interpretation of image alignments will be in error. As gravitational lensing experiments become more ambitious and seek very low-level alignments arising from lensing by large-scale structure, it becomes more important to estimate the level of intrinsic alignment in the galaxy population. In this article, I review the cluster of independent theoretical studies of this issue, as well as the current observational status. Theoretically, the calculation of intrinsic alignments is by no means straightforward, but some consensus has emerged from the existing works, despite each making very different assumptions. This consensus is that a) intrinsic alignments are a small but non-negligible (< 10%) contaminant of the lensing ellipticity correlation function, for samples with a median redshift z = 1; b) intrinsic alignments dominate the signal for low-redshift samples (z = 0.1), as expected in the SuperCOSMOS lensing survey and the Sloan Digital Sky Survey

Dark matter in the Universe consisting of macroscopic objects such as primordial black holes may cause gravitational lensing of distant objects. The magnification associated with lensing will lead to additional scatter in the received flux from standard candles, and too small an observed scatter could rule out compact dark matter entirely. In this letter, we show how the scatter in fluxes of distant Type 1a supernovae could be used to distinguish between models with and without lensing by macroscopic dark matter. The proposed SNAP project, with $\sim 2400$ supernovae in the range $0.1\ls z\ls 1.7$, should be able to identify models at 99.9% confidence, if systematic errors are controlled. Note that this test is independent of any evolution of the mean supernova luminosity with redshift. The variances of the current Supernova Cosmology Project sample do not rule out compact lenses as dark matter: formally they favour such a population, but the significance is low, and removal of a single faint supernova from the sample reverses the conclusion.

In the standard model for structure formation, bound objects originate from the gravitational collapse of small perturbations arising from quantum fluctuations with random phases. In other scenarios, based on defects, structures are seeded by localized energy density. In principle, it is possible to differentiate between these models on the basis of their statistical properties; only in the former case is the initial density field an almost-perfect random gaussian field. In this paper, we investigate the use of the trispectrum of the galaxy density field, which is the connected four-point function in Fourier space, as a discriminant between gaussian and non-gaussian models. It has the advantage of having only weak non-linear growth. We define a related statistic $\tau$ which, as a test of the gaussian hypothesis, is independent of cosmology, the power spectrum and biasing, in real space, and which is, in principle, a measure of the departure from gaussian statistics. For galaxy redshift surveys, the statistic depends on cosmology and bias only through the potentially observable parameter $\beta$. We compute the expected errors on the estimate of $\tau$, and demonstrate with numerical simulations that it can be a useful discriminant of models, with the important proviso that any bias is linear on large scales. Whether it is the most effective method is uncertain and depends on the nature of the departure from gaussianity.

We derive physical parameters of galaxies from their observed spectrum, using MOPED, the optimized data compression algorithm of Heavens, Jimenez & Lahav 2000. Here we concentrate on parametrising galaxy properties, and apply the method to the NGC galaxies in Kennicutt's spectral atlas. We focus on deriving the star formation history, metallicity and dust content of galaxies. The method is very fast, taking a few seconds of CPU time to estimate 17 parameters, and so specially suited to study of large data sets, such as the Anglo-Australian 2 degree field galaxy survey and the Sloan Digital Sky Survey. Without the power of MOPED, the recovery of star formation histories in these surveys would be impractical. In the Kennicutt atlas, we find that for the spheroidals a small recent burst of star formation is required to provide the best fit to the spectrum. There is clearly a need for theoretical stellar atmospheric models with spectral resolution better than 1\AA if we are to extract all the rich information that large redshift surveys contain in their galaxy spectra.

We investigate to what extent future microwave background experiments might be able to detect a suppression of fluctuation power on large scales in flat and open universe models. Such suppression would arise if fluctuations are generated by causal processes, and a measurement of a small suppression scale would be problematic for inflation models, but consistent with many defect models. More speculatively, a measurement of a suppression scale of the order of the present Hubble radius could provide independent evidence for a fine-tuned inflation model leading to a low-density universe. We find that, depending on the primordial power spectrum, a suppression scale modestly larger than the visible Horizon can be detected, but that the detectability drops very rapidly with increasing scale. For models with two periods of inflation, there is essentially no possibility of detecting a causal suppression scale.

We compute precise predictions for the two-point correlation function of local maxima (or minima) in the temperature of the microwave background, under the assumption that it is a random gaussian field. For a given power spectrum and peak threshold there are no adjustable parameters, and since this analysis does not make the small-angle approximation of Heavens & Sheth (1999), it is essentially complete. We find oscillatory features which are absent in the temperature autocorrelation function, and we also find that the small-angle approximation to the peak-peak correlation function is accurate to better than 0.01 on all scales. These high-precision predictions can form the basis of a sensitive test of the gaussian hypothesis with upcoming all-sky microwave background experiments MAP and Planck, affording a thorough test of the inflationary theory of the early Universe. To illustrate the effectiveness of the technique, we apply it to simulated maps of the microwave sky arising from the cosmic string model of structure formation, and compare with the bispectrum as a non-gaussian discriminant. We also show how peak statistics can be a valuable tool in assessing and statistically removing contamination of the map by foreground point sources.

Weak gravitational lensing is now established as a powerful method to measure mass fluctuations in the universe. It relies on the measurement of small coherent distortions of the images of background galaxies. Even low-level correlations in the intrinsic shapes of galaxies could however produce a significant spurious lensing signal. These correlations are also interesting in their own right, since their detection would constrain models of galaxy formation. Using $3\times 10^{4} - 10^5$ halos found in N-body simulations, we compute the correlation functions of the intrinsic ellipticity of spiral galaxies assuming that the disk is perpendicular to the angular momentum of the dark matter halo. We also consider a simple model for elliptical galaxies, in which the shape of the dark matter halo is assumed to be the same as that of the light. For deep lensing surveys with median redshifts $\sim 1$, we find that intrinsic correlations of $\sim 10^{-4}$ on angular scales $\theta \sim 0.1-10'$ are generally below the expected lensing signal, and contribute only a small fraction of the excess signals reported on these scales. On larger scales we find limits to the intrinsic correlation function at a level $\sim 10^{-5}$, which gives a (model-dependent) range of separations for which the intrinsic signal is about an order of magnitude below the ellipticity correlation function expected from weak lensing. Intrinsic correlations are thus negligible on these scales for dedicated weak lensing surveys. For wider but shallower surveys such as SuperCOSMOS, APM and SDSS, we cannot exclude the possibility that intrinsic correlations could dominate the lensing signal. We discuss how such surveys could be used to calibrate the importance of this effect, as well as study spin-spin correlations of spiral galaxies.

We apply a spherical harmonic analysis to the Point Source Redshift Survey (PSCz), to compute the real-space galaxy power spectrum and the degree of redshift distortion caused by peculiar velocities. We employ new parameter eigenvector and hierarchical data compression techniques, allowing a much larger number of harmonic modes to be included, and correspondingly smaller error bars. Using 4644 harmonic modes, compressed to 2278, we find that the IRAS redshift-space distortion parameter is $\beta = 0.39 \pm 0.12$ and the amplitude of galaxy clustering on a scale of $k=0.1 \Mpch$ is $\Delta_{\rm gal}(0.1)=0.42 \pm 0.02$. Combining these we find the amplitude of mass perturbations is $\Delta_m(0.1)=(0.16\pm0.04) \Omega_m^{-0.6}$. A preliminary model fitting analysis combining the PSCz amplitudes with the CMB and abundance of clusters yields the cosmological matter density parameter $\Omega_m=0.16\pm 0.03$, the amplitude of primordial perturbations $Q=(8.4\pm 3.8) \times 10^{-5}$, and the IRAS bias parameter $b=0.84\pm 0.28$.

We present a theoretical and exact analysis of the bispectrum of projected galaxy catalogues. The result can be generalized to evaluate the projection in spherical harmonics of any 3D bispectrum and therefore has applications to cosmic microwave background and gravitational lensing studies. By expanding the 2D distribution of galaxies on the sky in spherical harmonics, we show how the 3-point function of the coefficients can be used in principle to determine the bias parameter of the galaxy sample. If this can be achieved, it would allow a lifting of the degeneracy between the bias and the matter density parameter of the Universe which occurs in linear analysis of 3D galaxy catalogues. In previous papers we have shown how a similar analysis can be done in three dimensions, and we show here, through an error analysis and by implementing the method on a simulated projected catalogue, that ongoing three-dimensional galaxy redshift surveys (even with all the additional uncertainties introduced by partial sky coverage, redshift-space distortions and smaller numbers) will do far better than all-sky projected catalogues with similar selection function.

We present a method for radical linear compression of datasets where the data are dependent on some number $M$ of parameters. We show that, if the noise in the data is independent of the parameters, we can form $M$ linear combinations of the data which contain as much information about all the parameters as the entire dataset, in the sense that the Fisher information matrices are identical; i.e. the method is lossless. We explore how these compressed numbers fare when the noise is dependent on the parameters, and show that the method, although not precisely lossless, increases errors by a very modest factor. The method is general, but we illustrate it with a problem for which it is well-suited: galaxy spectra, whose data typically consist of $\sim 10^3$ fluxes, and whose properties are set by a handful of parameters such as age, brightness and a parametrised star formation history. The spectra are reduced to a small number of data, which are connected to the physical processes entering the problem. This data compression offers the possibility of a large increase in the speed of determining physical parameters. This is an important consideration as datasets of galaxy spectra reach $10^6$ in size, and the complexity of model spectra increases. In addition to this practical advantage, the compressed data may offer a classification scheme for galaxy spectra which is based rather directly on physical processes.

We have constrained the redshift-distortion parameter $\beta \equiv \Omega^{0.6}/b$ and the real-space power spectrum of the IRAS PSCz survey using a spherical-harmonic redshift-distortion analysis combined with a data compression method which is designed to deal with correlated parameters. Our latest result, $\beta=0.4 \pm 0.1$, strongly rules out $\beta=1$.

We present the results for the predicted density and peculiar velocity fields and the dipole from the PSCz survey of 15,000 IRAS galaxies over 84% of the sky. We find a significant component to the dipole arising between 6000 and 15,000 km/s, but no significant component from greater distances. The misalignment with the CMB is 20 degrees. The most remarkable feature of the PSCz model velocity field is a coherent large-scale flow along the baseline connecting Perseus-Pisces, the Local Supercluster, Great Attractor and the Shapley Concentration. We have measured the parameter beta using the amplitude of the dipole, bulk flow and point by point comparisons between the individual velocities of galaxies in the MarkIII and SFI datasets, and the large-scale clustering distortion in redshift space.All our results are consistent with beta = 0.6 +- 0.1.

Since cosmic-microwave-background (CMB) and large-scale-structure (LSS) data will shortly improve dramatically with the Microwave Anisotropy Probe (MAP) and Planck Surveyor, and the Anglo-Australian 2-Degree Field (2dF) and Sloan Digital Sky Survey (SDSS), respectively, it is timely to ask which of the CMB or LSS will provide a better probe of primordial non-gaussianity. In this paper we consider this question, using the bispectrum as a discriminating statistic. We consider several non-gaussian models and find that in each case the CMB will provide a better probe of primordial non-gaussianity. Since the bispectrum is the lowest-order statistic expected to arise in a generic non-gaussian model, our results suggest that if CMB maps appear gaussian, then apparent deviations from gaussian initial conditions in galaxy surveys can be attributed with confidence to the effects of biasing. We demonstrate this precisely for the spatial bispectrum induced by local non-linear biasing.

We present accurate predictions of the correlation function of hotspots in the microwave background radiation for gaussian theories such as those predicted in most inflation models. The correlation function of peaks above a certain threshold depends only on the threshold and the power spectrum of temperature fluctuations. Since there are both potentially observable quantities in a microwave background map, there are no adjustable parameters in the predictions. These correlations should therefore provide a powerful test of the gaussian hypothesis, and provide a useful discriminant between inflation and topological defect models such as the cosmic string model. The correlations have a number of oscillatory features, which should be detectable at high signal-to-noise with future satellite experiments such as MAP and Planck.

We investigate the influence of the star formation rate on the Tully-Fisher relation. We find that a simple model which combines the empirically-determined star-formation rate with the expected properties of galaxy halos provides a remarkably good fit to the absolute magnitude-rotation speed correlation. We find that the power-law nature, its slope, normalisation and scatter, are all readily accounted for if the Universe has a low density parameter, with or without a cosmological constant and disks are assembled at $z \sim 1 - 1.5$. Moreover, this agreement is found simultaneously in 4 wavebands. An Einstein-de Sitter Universe produces disks which are too faint unless the disks are assembled at $z\sim 0.5$. The scatter in the relation is due to a combination of the expected range of spin parameters of the halos and the range of formation redshifts. The source of the scatter opens up possibilities of a better galaxy distance indicator, if spectroscopic observations of globular clusters can be used to determine the halo rotation.

We apply the formalism of spherical harmonic decomposition to the galaxy density field of the IRAS PSCz redshift survey. The PSCz redshift survey has almost all-sky coverage and includes IRAS galaxies to a flux limit of 0.6 Jy. Using maximum likelihood methods to examine (to first order) the distortion of the galaxy pattern due to redshift coordinates, we have measured the parameter \beta= \Omega^{0.6}/b. We also simultaneously measure (a) the undistorted amplitude of perturbations in the galaxy distribution when a parameterised power spectrum is assumed, or (b) the shape and amplitude of the real-space power spectrum if the band-power in a set of passbands is measured in a step- wise fashion. These methods are extensively tested on a series of CDM, \Lambda CDM and MDM simulations and are found to be unbiased. We obtain consistent results for the subset of the PSCz catalogue with flux above 0.75 Jy but inclusion of galaxies to the formal flux limit of the catalogue gives variations which are larger than our internal errors. For the 0.75 Jy catalogue we find, in the case of a parameterised power spectrum, \beta=0.58\pm0.26 and the amplitude of the real space power measured at wavenumber k=0.1 h/Mpc is \Delta_{0.1}=0.42 \pm 0.03. Freeing the shape of the power spectrum we find that \beta=0.47\pm 0.16 (conditional error), and \Delta_{0.1}=0.47 \pm 0.03. The shape of the real-space power spectrum is consistent with a \Gamma= 0.2 CDM-like shape parameter, but does not strongly rule out a number of other models. Finally by combining our estimate of the amplitude of galaxy clustering and the distortion parameter we find the amplitude of mass fluctuations on a scale k=0.1 \hmpc is \Delta_\rho = 0.24 \Omega_0^{-0.6}$, with an uncertainty of 50%.

We study the power spectrum of galaxies in redshift space, with third order perturbation theory to include corrections that are absent in linear theory. We assume a local bias for the galaxies: i.e. the galaxy density is sampled from some local function of the underlying mass distribution. We find that the effect of the nonlinear bias in real space is to introduce two new features: first, there is a contribution to the power which is constant with wavenumber, whose nature we reveal as essentially a shot-noise term. In principle this contribution can mask the primordial power spectrum, and could limit the accuracy with which the latter might be measured on very large scales. Secondly, the effect of second- and third-order bias is to modify the effective bias (defined as the square root of the ratio of galaxy power spectrum to matter power spectrum). The effective bias is almost scale-independent over a wide range of scales. These general conclusions also hold in redshift space. In addition, we have investigated the distortion of the power spectrum by peculiar velocities, which may be used to constrain the density of the Universe. We look at the quadrupole-to-monopole ratio, and find that higher-order terms can mimic linear theory bias, but the bias implied is neither the linear bias, nor the effective bias referred to above. We test the theory with biased N-body simulations, and find excellent agreement in both real and redshift space, providing the local biasing is applied on a scale whose fractional r.m.s. density fluctuations are $< 0.5$.

The determination of the density parameter $\Omega_0$ from the large-scale distribution of galaxies is one of the major goals of modern cosmology. However, if galaxies are biased tracers of the underlying mass distribution, linear perturbation theory leads to a degeneracy between $\Omega_0$ and the linear bias parameter $b$, and the density parameter cannot be estimated. In Matarrese, Verde & Heavens (1997) we developed a method based on second-order perturbation theory to use the bispectrum to lift this degeneracy by measuring the bias parameter in an $\Omega_0$-independent way. The formalism was developed assuming that one has perfect information on the positions of galaxies in three dimensions. In galaxy redshift surveys, the three-dimensional information is imperfect, because of the contaminating effects of peculiar velocities, and the resulting clustering pattern in redshift space is distorted. In this paper, we combine second-order perturbation theory with a model for collapsed, virialised structures, to extend the method to redshift space, and demonstrate that the method should be successful in determining with reasonable accuracy the bias parameter from state-of-the-art surveys such as the Anglo-Australian 2 degree field survey and the Sloan digital sky survey.

The bispectrum of the microwave background sky is a possible discriminator between inflationary and defect models of structure formation in the Universe. The bispectrum, which is the analogue of the temperature 3-point correlation function in harmonic space, is zero for most inflationary models, but non-zero for non-gaussian models. The expected departures from zero are small, and easily masked by noise, so it is important to be able to estimate the bispectrum coefficients as accurately as possible, and to know the errors and correlations between the estimates so they may be used in combination as a diagnostic to rule out non-gaussian models. This paper presents a method for estimating in an unbiased way the bispectrum from a microwave background map in the near-gaussian limit. The method is optimal, in the sense that no other method can have smaller error bars, and in addition, the covariances between the bispectrum estimates are calculated explicitly. The method deals automatically with partial sky coverage and arbitrary noise correlations without modification. A preliminary application to the Cosmic Background Explorer 4-year dataset shows no evidence for non-gaussian behaviour.

We investigate in detail the hypothesis that low surface brightness galaxies (LSB) differ from ordinary galaxies simply because they form in halos with large spin parameters. We compute star formation rates using the Schmidt law, assuming the same gas infall dependence on surface density as used in models of the Milky Way. We build stellar population models, predicting colours, spectra, and chemical abundances. We compare our predictions with observed values of metallicity and colours for LSB galaxies and find excellent agreement with all observables. In particular, integrated colours, colour gradients, surface brightness and metallicity match very well to the observed values of LSBs for models with ages larger than 7 Gyr and high values ($\lambda > 0.05$) for the spin parameter of the halos. We also compute the global star formation rate (SFR) in the Universe due to LSBs and show that it has a flatter evolution with redshift than the corresponding SFR for normal discs. We furthermore compare the evolution in redshift of $[Zn/H]$ for our models to those observed in Damped Lyman $\alpha$ systems by \scite{Pettini+97} and show that Damped Lyman $\alpha$ systems abundances are consistent with the predicted abundances at different radii for LSBs. Finally, we show how the required late redshift of collapse of the halo may constrain the power spectrum of fluctuations.

Gravitational lensing studies suggest that the Universe may contain a population of dark galaxies; we investigate this intriguing possibility and propose a mechanism to explain their nature. In this mechanism a dark galaxy is formed with a low density disk in a dark halo of high spin parameter; such galaxies can have surface densities below the critical Toomre value for instabilities to develop, and following Kennicutt's work we expect these galaxies to have low star formation rates. The only stellar component of the galaxies is a halo system, formed during the collapse of the proto-galactic cloud. We compute synthetic stellar population models and show that, at a redshift $z=0.5$, such galaxies have apparent magnitudes $B \simeq 28, R \simeq 26$ and $I \simeq 25$, and could be unveiled by deep searches with the Hubble Space Telescope. Dark galaxies have an initial short blue phase and then become essentially invisible, therefore they may account for the blue population of galaxies at high redshift. We find a strong mass-dependence in the fraction of dark galaxies, and predict that spiral galaxies will not be found in halos with masses less than about $10^9 M_\odot$, if $\Omega=1$. Above about $10^{12}M_\odot$, all halos can produce luminous disks. The mass-dependence of the galaxy-formation efficiency introduces the possibility of `spin bias' -- luminous galaxies being associated preferentially with strongly-clustered high-mass halos. A further prediction is that the slope of the faint-end luminosity function for galaxies will be flatter than the associated halo mass function.

We report the discovery of a new gravitational lens candidate Q2138-431AB, comprising two quasar images at a redshift of 1.641 separated by 4.5 arcsecs. The spectra of the two images are very similar, and the redshifts agree to better than 115 km.sec$^{-1}$. The two images have magnitudes $B_J = 19.8$ and $B_J = 21.0$, and in spite of a deep search and image subtraction procedure, no lensing galaxy has been found with $R < 23.8$. Modelling of the system configuration implies that the mass-to-light ratio of any lensing galaxy is likely to be around $1000 M_{\odot}/L_{\odot}$, with an absolute lower limit of $200 M_{\odot}/L_{\odot}$ for an Einstein-de Sitter universe. We conclude that the most likely explanation of the observations is gravitational lensing by a dark galaxy, although it is possible we are seeing a binary quasar.

We investigate the effects on cosmological clustering statistics of empirical biasing, where the galaxy distribution is a local transformation of the present-day Eulerian density field. The effects of the suppression of galaxy numbers in voids, and their enhancement in regions of high density, are considered, independently and in combination. We compare results from numerical simulations with the predictions of simple analytic models. We find that the bias is generally scale-dependent, so that the shape of the galaxy power spectrum differs from that of the underlying mass distribution. The degree of bias is always a monotonic function of scale, tending to an asymptotic value on scales where the density fluctuations are linear. The scale dependence is often rather weak, with many reasonable prescriptions giving a bias which is nearly independent of scale. We have investigated whether such an Eulerian bias can reconcile a range of theoretical power spectra with the twin requirements of fitting the galaxy power spectrum and reproducing the observed mass-to-light ratios in clusters. It is not possible to satisfy these constraints for any member of the family of CDM-like power spectra in an Einstein - de Sitter universe when normalised to match COBE on large scales and galaxy cluster abundances on intermediate scales. We discuss what modifications of the mass power spectrum might produce agreement with the observational data.

We discuss the problems of applying Maximum Likelihood methods to the CMB and how one can make it both efficient and optimal. The solution is a generalised eigenvalue problem that allows virtually no loss of information about the parameter being estimated, but can allow a substantial compression of the data set. We discuss the more difficult question of simultaneous estimation of many parameters, and propose solutions. A much fuller account of most of this work is available (Tegmark, Taylor & Heavens 1997)

Evidence that the Universe may be close to the critical density, required for its expansion eventually to be halted, comes principally from dynamical studies of large-scale structure. These studies either use the observed peculiar velocity field of galaxies directly, or indirectly by quantifying its anisotropic effect on galaxy clustering in redshift surveys. A potential difficulty with both such approaches is that the density parameter $\Omega_0$ is obtained only in the combination $\beta = \Omega_0^{0.6}/b$, if linear perturbation theory is used. The determination of the density parameter $\Omega_0$ is therefore compromised by the lack of a good measurement of the bias parameter $b$, which relates the clustering of sample galaxies to the clustering of mass. In this paper, we develop an idea of Fry (1994), using second-order perturbation theory to investigate how to measure the bias parameter on large scales. The use of higher-order statistics allows the degeneracy between $b$ and $\Omega_0$ to be lifted, and an unambiguous determination of $\Omega_0$ then becomes possible. We apply a likelihood approach to the bispectrum, the three-point function in Fourier space. This paper is the first step in turning the idea into a practical proposition for redshift surveys, and is principally concerned with noise properties of the bispectrum, which are non-trivial. The calculation of the required bispectrum covariances involves the six-point function, including many noise terms, for which we have developed a generating functional approach which will be of value in calculating high-order statistics in general.

In this paper we consider methods of analysis and optimal design of redshift surveys. In the first part, we develop a formalism for analysing galaxy redshift surveys which are essentially two-dimensional, such as thin declination slices. The formalism is a power spectrum method, using spherical coordinates, allowing the distorting effects of galaxy peculiar velocities to be calculated to linear order on the assumption of statistical isotropy but without further approximation. In this paper, we calculate the measured two-dimensional power for a constant declination strip, widely used in redshift surveys. We present a likelihood method for estimating the three-dimensional real-space power spectrum and the redshift distortion simultaneously, and show that for thin surveys of reasonable depth, the large-scale 3D power cannot be measured with high accuracy. The redshift distortion may be estimated successfully, and with higher accuracy if the 3D power spectrum can be measured independently, for example from a large-scale sky-projected catalogue. In the second part, we show how a 3D survey design can be optimized to measure the power spectrum, considering whether areal coverage is more important than depth, and whether the survey should be sampled sparsely or not. We show quite generally that width is better than depth, and show how the optimal sparse-sampling fraction, f, depends on the power, P, to be measured. For a Schechter luminosity function, a simple optimization fP \simeq 500 h^{-3} Mpc^3 is found.

Since cosmology is no longer "the data-starved science", the problem of how to best analyze large data sets has recently received considerable attention, and Karhunen-Loeve eigenvalue methods have been applied to both galaxy redshift surveys and Cosmic Microwave Background (CMB) maps. We present a comprehensive discussion of methods for estimating cosmological parameters from large data sets, which includes the previously published techniques as special cases. We show that both the problem of estimating several parameters jointly and the problem of not knowing the parameters a priori can be readily solved by adding an extra singular value decomposition step. It has recently been argued that the information content in a sky map from a next generation CMB satellite is sufficient to measure key cosmological parameters (h, Omega, Lambda, etc) to an accuracy of a few percent or better - in principle. In practice, the data set is so large that both a brute force likelihood analysis and a direct expansion in signal-to-noise eigenmodes will be computationally unfeasible. We argue that it is likely that a Karhunen-Loeve approach can nonetheless measure the parameters with close to maximal accuracy, if preceded by an appropriate form of quadratic "pre-compression". We also discuss practical issues regarding parameter estimation from present and future galaxy redshift surveys, and illustrate this with a generalized eigenmode analysis of the IRAS 1.2 Jy survey optimized for measuring beta=Omega^{0.6}/b using redshift space distortions.

It has been proposed that the cosmological constant $\Lambda$ might be measured from geometric effects on large-scale structure. A positive vacuum density leads to correlation-function contours which are squashed in the radial direction when calculated assuming a matter-dominated model. We show that this effect will be somewhat harder to detect than previous calculations have suggested: the squashing factor is likely to be $<1.3$, given realistic constraints on the matter contribution to $\Omega$. Moreover, the geometrical distortion risks being confused with the redshift-space distortions caused by the peculiar velocities associated with the growth of galaxy clustering. These depend on the density and bias parameters via the combination $\beta\equiv \Omega^{0.6}/b$, and we show that the main practical effect of a geometrical flattening factor $F$ is to simulate gravitational instability with $\beta_{\rm eff}\simeq 0.5(F-1)$. Nevertheless, with datasets of sufficient size it is possible to distinguish the two effects; we discuss in detail how this should be done. New-generation redshift surveys of galaxies and quasars are potentially capable of detecting a non-zero vacuum density, if it exists at a cosmologically interesting level.

We re-examine the effects of redshift space distortion in all-sky galaxy redshift surveys in the formalism of spherical harmonics. Within this framework we show how one can treat both the large-scale linear effects, and the small-scale nonlinear clustering, exactly to first order. The method also allows in principle a determination of the power spectrum of perturbations, requiring no assumptions beyond that of linear theory. The method therefore offers significant advantages over Fourier techniques when dealing with all-sky surveys. We apply our likelihood analysis to both simulated data, and real data, using the IRAS 1.2-Jy galaxy catalogue, for which we find a maximum likelihood $\beta \simeq 1.1\pm 0.3$, and a real-space fluctuation amplitude corresponding to $\sigma_{8,{\rm IRAS}} = 0.68\pm 0.05$.

This paper analyses the effects of random noise in determining errors and confidence levels for galaxy redshifts obtained by cross-correlation techniques. The main finding is that confidence levels have previously been overestimated, and errors inaccurately calculated in certain applications. New formul\ae\ are presented.