Flagging of Radio Frequency Interference (RFI) is an increasingly important challenge in radio astronomy. We present R-Net, a deep convolutional ResNet architecture that significantly outperforms existing algorithms -- including the default MeerKAT RFI flagger, and deep U-Net architectures -- across all metrics including AUC, F1-score and MCC. We demonstrate the robustness of this improvement on both single dish and interferometric simulations and, using transfer learning, on real data. Our R-Net model's precision is approximately $90\%$ better than the current MeerKAT flagger at $80\%$ recall and has a 35\% higher F1-score with no additional performance cost. We further highlight the effectiveness of transfer learning from a model initially trained on simulated MeerKAT data and fine-tuned on real, human-flagged, KAT-7 data. Despite the wide differences in the nature of the two telescope arrays, the model achieves an AUC of 0.91, while the best model without transfer learning only reaches an AUC of 0.67. We consider the use of phase information in our models but find that without calibration the phase adds almost no extra information relative to amplitude data only. Our results strongly suggest that deep learning on simulations, boosted by transfer learning on real data, will likely play a key role in the future of RFI flagging of radio astronomy data.

We present an analysis of DES17X1boj and DES16E2bjy, two peculiar transients discovered by the Dark Energy Survey (DES). They exhibit nearly identical double-peaked light curves which reach very different maximum luminosities (M$_\mathrm{r}$ = -15.4 and M$_\mathrm{r}$ = -17.9, respectively). The light curve evolution of these events is highly atypical and has not been reported before. The transients are found in different host environments: DES17X1boj was found near the nucleus of a spiral galaxy, while DES16E2bjy is located in the outskirts of a passive red galaxy. Early photometric data is well fitted with a blackbody and the resulting moderate photospheric expansion velocities (1800 km/s for DES17X1boj and 4800 km/s for DES16E2bjy) suggest an explosive or eruptive origin. Additionally, a feature identified as high-velocity CaII absorption (v $\approx$ 9400km/s) in the near-peak spectrum of DES17X1boj may imply that it is a supernova. While similar light curve evolution suggests a similar physical origin for these two transients, we are not able to identify or characterise the progenitors.

Anomaly detection is challenging, especially for large datasets in high dimensions. Here we explore a general anomaly detection framework based on dimensionality reduction and unsupervised clustering. We release DRAMA, a general python package that implements the general framework with a wide range of built-in options. We test DRAMA on a wide variety of simulated and real datasets, in up to 3000 dimensions, and find it robust and highly competitive with commonly-used anomaly detection algorithms, especially in high dimensions. The flexibility of the DRAMA framework allows for significant optimization once some examples of anomalies are available, making it ideal for online anomaly detection, active learning and highly unbalanced datasets.

We present griz light curves of 251 Type Ia Supernovae (SNe Ia) from the first 3 years of the Dark Energy Survey Supernova Program's (DES-SN) spectroscopically classified sample. The photometric pipeline described in this paper produces the calibrated fluxes and associated uncertainties used in the cosmological parameter analysis (Brout et al. 2018-SYS, DES Collaboration et al. 2018) by employing a scene modeling approach that simultaneously forward models a variable transient flux and temporally constant host galaxy. We inject artificial point sources onto DECam images to test the accuracy of our photometric method. Upon comparison of input and measured artificial supernova fluxes, we find flux biases peak at 3 mmag. We require corrections to our photometric uncertainties as a function of host galaxy surface brightness at the transient location, similar to that seen by the DES Difference Imaging Pipeline used to discover transients. The public release of the light curves can be found at https://des.ncsa.illinois.edu/releases/sn.

We present the analysis underpinning the measurement of cosmological parameters from 207 spectroscopically classified type Ia supernovae (SNe Ia) from the first three years of the Dark Energy Survey Supernova Program (DES-SN), spanning a redshift range of 0.017<$z$<0.849. We combine the DES-SN sample with an external sample of 122 low-redshift ($z$<0.1) SNe Ia, resulting in a "DES-SN3YR" sample of 329 SNe Ia. Our cosmological analyses are blinded: after combining our DES-SN3YR distances with constraints from the Cosmic Microwave Background (CMB; Planck Collaboration 2016), our uncertainties in the measurement of the dark energy equation-of-state parameter, $w$, are .042 (stat) and .059 (stat+syst) at 68% confidence. We provide a detailed systematic uncertainty budget, which has nearly equal contributions from photometric calibration, astrophysical bias corrections, and instrumental bias corrections. We also include several new sources of systematic uncertainty. While our sample is <1/3 the size of the Pantheon sample, our constraints on $w$ are only larger by 1.4$\times$, showing the impact of the DES SN Ia light curve quality. We find that the traditional stretch and color standardization parameters of the DES SNe Ia are in agreement with earlier SN Ia samples such as Pan-STARRS1 and the Supernova Legacy Survey. However, we find smaller intrinsic scatter about the Hubble diagram (0.077 mag). Interestingly, we find no evidence for a Hubble residual step ( 0.007 $\pm$ 0.018 mag) as a function of host galaxy mass for the DES subset, in 2.4$\sigma$ tension with previous measurements. We also present novel validation methods of our sample using simulated SNe Ia inserted in DECam images and using large catalog-level simulations to test for biases in our analysis pipelines.

We present an improved measurement of the Hubble constant (H_0) using the 'inverse distance ladder' method, which adds the information from 207 Type Ia supernovae (SNe Ia) from the Dark Energy Survey (DES) at redshift 0.018 < z < 0.85 to existing distance measurements of 122 low redshift (z < 0.07) SNe Ia (Low-z) and measurements of Baryon Acoustic Oscillations (BAOs). Whereas traditional measurements of H_0 with SNe Ia use a distance ladder of parallax and Cepheid variable stars, the inverse distance ladder relies on absolute distance measurements from the BAOs to calibrate the intrinsic magnitude of the SNe Ia. We find H_0 = 67.8 +/- 1.3 km s-1 Mpc-1 (statistical and systematic uncertainties, 68% confidence). Our measurement makes minimal assumptions about the underlying cosmological model, and our analysis was blinded to reduce confirmation bias. We examine possible systematic uncertainties and all are below the statistical uncertainties. Our H_0 value is consistent with estimates derived from the Cosmic Microwave Background assuming a LCDM universe (Planck Collaboration et al. 2018).

We present the first cosmological parameter constraints using measurements of type Ia supernovae (SNe Ia) from the Dark Energy Survey Supernova Program (DES-SN). The analysis uses a subsample of 207 spectroscopically confirmed SNe Ia from the first three years of DES-SN, combined with a low-redshift sample of 122 SNe from the literature. Our "DES-SN3YR" result from these 329 SNe Ia is based on a series of companion analyses and improvements covering SN Ia discovery, spectroscopic selection, photometry, calibration, distance bias corrections, and evaluation of systematic uncertainties. For a flat LCDM model we find a matter density Omega_m = 0.331 +_ 0.038. For a flat wCDM model, and combining our SN Ia constraints with those from the cosmic microwave background (CMB), we find a dark energy equation of state w = -0.978 +_ 0.059, and Omega_m = 0.321 +_ 0.018. For a flat w0waCDM model, and combining probes from SN Ia, CMB and baryon acoustic oscillations, we find w0 = -0.885 +_ 0.114 and wa = -0.387 +_ 0.430. These results are in agreement with a cosmological constant and with previous constraints using SNe Ia (Pantheon, JLA).

The combination of multiple observational probes has long been advocated as a powerful technique to constrain cosmological parameters, in particular dark energy. The Dark Energy Survey has measured 207 spectroscopically--confirmed Type Ia supernova lightcurves; the baryon acoustic oscillation feature; weak gravitational lensing; and galaxy clustering. Here we present combined results from these probes, deriving constraints on the equation of state, $w$, of dark energy and its energy density in the Universe. Independently of other experiments, such as those that measure the cosmic microwave background, the probes from this single photometric survey rule out a Universe with no dark energy, finding $w=-0.80^{+0.09}_{-0.11}$. The geometry is shown to be consistent with a spatially flat Universe, and we obtain a constraint on the baryon density of $\Omega_b=0.069^{+0.009}_{-0.012}$ that is independent of early Universe measurements. These results demonstrate the potential power of large multi-probe photometric surveys and pave the way for order of magnitude advances in our constraints on properties of dark energy and cosmology over the next decade.

We present spectroscopy from the first three seasons of the Dark Energy Survey Supernova Program (DES-SN). We describe the supernova spectroscopic program in full: strategy, observations, data reduction, and classification. We have spectroscopically confirmed 307 supernovae, including 251 type Ia supernovae (SNe Ia) over a redshift range of $0.017 < z < 0.85$. We determine the effective spectroscopic selection function for our sample, and use it to investigate the redshift-dependent bias on the distance moduli of SNe Ia we have classified. We also provide a full overview of the strategy, observations, and data products of DES-SN, which has discovered 12,015 likely supernovae during these first three seasons. The data presented here are used for the first cosmology analysis by DES-SN ('DES-SN3YR'), the results of which are given in DES Collaboration (2018a).

With the advent of powerful telescopes such as the SKA and LSST, we are entering a golden era of multiwavelength transient astronomy. In order to cope with the dramatic increase in data volume as well as successfully prioritise spectroscopic follow-up resources, we propose a new machine learning approach for the classification of radio and multiwavelength transients. The algorithm consists of three steps: (1) augmentation and interpolation of the data using Gaussian processes; (2) feature extraction using a wavelet decomposition; (3) classification with the robust machine learning algorithm known as random forests. We apply this algorithm to existing radio transient data, illustrating its ability to accurately classify most of the eleven classes of radio variables and transients after just eight hours of observations, achieving an overall accuracy of $73.5\%$. We show how performance is expected to increase as more training data are acquired, by training the classifier on a simulated representative training set, achieving an overall accuracy of $97.4\%$. Finally, we outline a general approach for including multiwavelength data for general transient classification, and demonstrate its effectiveness by incorporating a single optical data point into the analysis, which improves the overall accuracy by $\approx 22\%$.

Calibration uncertainties have been the leading systematic uncertainty in recent analyses using type Ia Supernovae (SNe Ia) to measure cosmological parameters. To improve the calibration, we present the application of Spectral Energy Distribution (SED)-dependent "chromatic corrections" to the supernova light-curve photometry from the Dark Energy Survey (DES). These corrections depend on the combined atmospheric and instrumental transmission function for each exposure, and they affect photometry at the 0.01 mag (1%) level, comparable to systematic uncertainties in calibration and photometry. Fitting our combined DES and low-z SN Ia sample with Baryon Acoustic Oscillation (BAO) and Cosmic Microwave Background (CMB) priors for the cosmological parameters $\Omega_{\rm m}$ (the fraction of the critical density of the universe comprised of matter) and w (the dark energy equation of state parameter), we compare those parameters before and after applying the corrections. We find the change in w and $\Omega_{\rm m}$ due to not including chromatic corrections are -0.002 and 0.000, respectively, for the DES-SN3YR sample with BAO and CMB priors, consistent with a larger DES-SN3YR-like simulation, which has a w-change of 0.0005 with an uncertainty of 0.008 and an $\Omega_{\rm m}$ change of 0.000 with an uncertainty of 0.002 . However, when considering samples on individual CCDs we find large redshift-dependent biases (approximately 0.02 in distance modulus) for supernova distances.

Point source detection at low signal-to-noise is challenging for astronomical surveys, particularly in radio interferometry images where the noise is correlated. Machine learning is a promising solution, allowing the development of algorithms tailored to specific telescope arrays and science cases. We present DeepSource - a deep learning solution - that uses convolutional neural networks to achieve these goals. DeepSource enhances the Signal-to-Noise Ratio (SNR) of the original map and then uses dynamic blob detection to detect sources. Trained and tested on two sets of 500 simulated 1 deg x 1 deg MeerKAT images with a total of 300,000 sources, DeepSource is essentially perfect in both purity and completeness down to SNR = 4 and outperforms PyBDSF in all metrics. For uniformly-weighted images it achieves a Purity x Completeness (PC) score at SNR = 3 of 0.73, compared to 0.31 for the best PyBDSF model. For natural-weighting we find a smaller improvement of ~40% in the PC score at SNR = 3. If instead we ask where either of the purity or completeness first drop to 90%, we find that DeepSource reaches this value at SNR = 3.6 compared to the 4.3 of PyBDSF (natural-weighting). A key advantage of DeepSource is that it can learn to optimally trade off purity and completeness for any science case under consideration. Our results show that deep learning is a promising approach to point source detection in astronomical images.

We develop a machine learning (ML) framework to populate large dark matter-only simulations with baryonic galaxies. Our ML framework takes input halo properties including halo mass, environment, spin, and recent growth history, and outputs central galaxy and halo baryonic properties including stellar mass ($M_*$), star formation rate (SFR), metallicity ($Z$), neutral ($\rm HI$) and molecular ($\rm H_2$) hydrogen mass. We apply this to the MUFASA cosmological hydrodynamic simulation, and show that it recovers the mean trends of output quantities with halo mass highly accurately, including following the sharp drop in SFR and gas in quenched massive galaxies. However, the scatter around the mean relations is under-predicted. Examining galaxies individually, at $z=0$ the stellar mass and metallicity are accurately recovered ($\sigma\lesssim 0.2$~dex), but SFR and $\rm HI$ show larger scatter ($\sigma\gtrsim 0.3$~dex); these values improve somewhat at $z=1,2$. Remarkably, ML quantitatively recovers second parameter trends in galaxy properties, e.g. that galaxies with higher gas content and lower metallicity have higher SFR at a given $M_*$. Testing various ML algorithms, we find that none perform significantly better than the others, nor does ensembling improve performance, likely because none of the algorithms reproduce the large observed scatter around the mean properties. For the random forest algorithm, we find that halo mass and nearby ($\sim 200$~kpc) environment are the most important predictive variables followed by growth history, while halo spin and $\sim$Mpc scale environment are not important. Finally we study the impact of additionally inputting key baryonic properties $M_*$, SFR and $Z$, as would be available e.g. from an equilibrium model, and show that particularly providing the SFR enables $\rm HI$ to be recovered substantially more accurately.

We main goal of this paper is to test whether the NIR peak magnitudes of SNe Ia could be accurately estimated with only a single observation obtained close to maximum light, provided the time of B band maximum and the optical stretch parameter are known. We obtained multi-epoch UBVRI and single-epoch J and H photometric observations of 16 SNe Ia in the redshift range z=0.037-0.183, doubling the leverage of the current SN Ia NIR Hubble diagram and the number of SNe beyond redshift 0.04. This sample was analyzed together with 102 NIR and 458 optical light curves (LCs) of normal SNe Ia from the literature. The analysis of 45 well-sampled NIR LCs shows that a single template accurately describes them if its time axis is stretched with the optical stretch parameter. This allows us to estimate the NIR peak magnitudes even with one observation obtained within 10 days from B-band maximum. We find that the NIR Hubble residuals show weak correlation with DM_15 and E(B-V), and for the first time we report a possible dependence on the J_max-H_max color. The intrinsic NIR luminosity scatter of SNe Ia is estimated to be around 0.10 mag, which is smaller than what can be derived for a similarly heterogeneous sample at optical wavelengths. In conclusion, we find that SNe Ia are at least as good standard candles in the NIR as in the optical. We showed that it is feasible to extended the NIR SN Ia Hubble diagram to z=0.2 with very modest sampling of the NIR LCs, if complemented by well-sampled optical LCs. Our results suggest that the most efficient way to extend the NIR Hubble diagram to high redshift would be to obtain a single observation close to the NIR maximum. (abridged)

We explore the use of random forest and gradient boosting, two powerful tree-based machine learning algorithms, for the detection of cosmic strings in maps of the cosmic microwave background (CMB), through their unique Gott-Kaiser-Stebbins effect on the temperature anisotropies.The information in the maps is compressed into feature vectors before being passed to the learning units. The feature vectors contain various statistical measures of processed CMB maps that boost the cosmic string detectability. Our proposed classifiers, after training, give results improved over or similar to the claimed detectability levels of the existing methods for string tension, $G\mu$. They can make $3\sigma$ detection of strings with $G\mu \gtrsim 2.1\times 10^{-10}$ for noise-free, $0.9'$-resolution CMB observations. The minimum detectable tension increases to $G\mu \gtrsim 3.0\times 10^{-8}$ for a more realistic, CMB S4-like (II) strategy, still a significant improvement over the previous results.

We measure a value for the cosmic expansion of $H(z) = 89 \pm 23$(stat) $\pm$ 44(syst) km s$^{-1}$ Mpc$^{-1}$ at a redshift of $z \simeq 0.47$ based on the differential age technique. This technique, also known as cosmic chronometers, uses the age difference between two redshifts for a passively evolving population of galaxies to calculate the expansion rate of the Universe. Our measurement is based on analysis of high quality spectra of Luminous Red Galaxies (LRGs) obtained with the Southern African Large Telescope (SALT) in two narrow redshift ranges of $z \simeq 0.40$ and $z \simeq 0.55$ as part of an initial pilot study. Ages were estimated by fitting single stellar population models to the observed spectra. This measurement presents one of the best estimates of $H(z)$ via this method at $z\sim0.5$ to date.

Supernova cosmology without spectra will be an important component of future surveys such as LSST. This lack of supernova spectra results in uncertainty in the redshifts which, if ignored, leads to significantly biased estimates of cosmological parameters. Here we present a hierarchical Bayesian formalism -- zBEAMS -- that addresses this problem by marginalising over the unknown or uncertain supernova redshifts to produce unbiased cosmological estimates that are competitive with supernova data with spectroscopically confirmed redshifts. zBEAMS provides a unified treatment of both photometric redshifts and host galaxy misidentification (occurring due to chance galaxy alignments or faint hosts), effectively correcting the inevitable contamination in the Hubble diagram. Like its predecessor BEAMS, our formalism also takes care of non-Ia supernova contamination by marginalising over the unknown supernova type. We illustrate this technique with simulations of supernovae with photometric redshifts and host galaxy misidentification. A novel feature of the photometric redshift case is the important role played by the redshift distribution of the supernovae.

We discuss the ground-breaking science that will be possible with a wide area survey, using the MeerKAT telescope, known as MeerKLASS (MeerKAT Large Area Synoptic Survey). The current specifications of MeerKAT make it a great fit for science applications that require large survey speeds but not necessarily high angular resolutions. In particular, for cosmology, a large survey over $\sim 4,000 \, {\rm deg}^2$ for $\sim 4,000$ hours will potentially provide the first ever measurements of the baryon acoustic oscillations using the 21cm intensity mapping technique, with enough accuracy to impose constraints on the nature of dark energy. The combination with multi-wavelength data will give unique additional information, such as exquisite constraints on primordial non-Gaussianity using the multi-tracer technique, as well as a better handle on foregrounds and systematics. Such a wide survey with MeerKAT is also a great match for HI galaxy studies, providing unrivalled statistics in the pre-SKA era for galaxies resolved in the HI emission line beyond local structures at z > 0.01. It will also produce a large continuum galaxy sample down to a depth of about 5\,$\mu$Jy in L-band, which is quite unique over such large areas and will allow studies of the large-scale structure of the Universe out to high redshifts, complementing the galaxy HI survey to form a transformational multi-wavelength approach to study galaxy dynamics and evolution. Finally, the same survey will supply unique information for a range of other science applications, including a large statistical investigation of galaxy clusters as well as produce a rotation measure map across a huge swathe of the sky. The MeerKLASS survey will be a crucial step on the road to using SKA1-MID for cosmological applications and other commensal surveys, as described in the top priority SKA key science projects (abridged).

The MIGHTEE large survey project will survey four of the most well-studied extragalactic deep fields, totalling 20 square degrees to $\mu$Jy sensitivity at Giga-Hertz frequencies, as well as an ultra-deep image of a single ~1 square degree MeerKAT pointing. The observations will provide radio continuum, spectral line and polarisation information. As such, MIGHTEE, along with the excellent multi-wavelength data already available in these deep fields, will allow a range of science to be achieved. Specifically, MIGHTEE is designed to significantly enhance our understanding of, (i) the evolution of AGN and star-formation activity over cosmic time, as a function of stellar mass and environment, free of dust obscuration; (ii) the evolution of neutral hydrogen in the Universe and how this neutral gas eventually turns into stars after moving through the molecular phase, and how efficiently this can fuel AGN activity; (iii) the properties of cosmic magnetic fields and how they evolve in clusters, filaments and galaxies. MIGHTEE will reach similar depth to the planned SKA all-sky survey, and thus will provide a pilot to the cosmology experiments that will be carried out by the SKA over a much larger survey volume.

We present photometry and time-series spectroscopy of the nearby type Ia supernova (SN Ia) SN 2015F over $-16$ days to $+80$ days relative to maximum light, obtained as part of the Public ESO Spectroscopic Survey of Transient Objects (PESSTO). SN 2015F is a slightly sub-luminous SN Ia with a decline rate of $\Delta m15(B)=1.35 \pm 0.03$ mag, placing it in the region between normal and SN 1991bg-like events. Our densely-sampled photometric data place tight constraints on the epoch of first light and form of the early-time light curve. The spectra exhibit photospheric C II $\lambda 6580$ absorption until $-4$ days, and high-velocity Ca II is particularly strong at $<-10$ days at expansion velocities of $\simeq$23000\kms. At early times, our spectral modelling with syn++ shows strong evidence for iron-peak elements (Fe II, Cr II, Ti II, and V II) expanding at velocities $>14000$ km s$^{-1}$, suggesting mixing in the outermost layers of the SN ejecta. Although unusual in SN Ia spectra, including V II in the modelling significantly improves the spectral fits. Intriguingly, we detect an absorption feature at $\sim$6800 \AA\ that persists until maximum light. Our favoured explanation for this line is photospheric Al II, which has never been claimed before in SNe Ia, although detached high-velocity C II material could also be responsible. In both cases the absorbing material seems to be confined to a relatively narrow region in velocity space. The nucleosynthesis of detectable amounts of Al II would argue against a low-metallicity white dwarf progenitor. We also show that this 6800 \AA\ feature is weakly present in other normal SN Ia events, and common in the SN 1991bg-like sub-class.

Bayesian graphical models are an efficient tool for modelling complex data and derive self-consistent expressions of the posterior distribution of model parameters. We apply Bayesian graphs to perform statistical analyses of Type Ia supernova (SN Ia) luminosity distance measurements from the joint light-curve analysis (JLA) data set. In contrast to the $\chi^2$ approach used in previous studies, the Bayesian inference allows us to fully account for the standard-candle parameter dependence of the data covariance matrix. Comparing with $\chi^2$ analysis results, we find a systematic offset of the marginal model parameter bounds. We demonstrate that the bias is statistically significant in the case of the SN Ia standardization parameters with a maximal 6 $\sigma$ shift of the SN light-curve colour correction. In addition, we find that the evidence for a host galaxy correction is now only 2.4 $\sigma$. Systematic offsets on the cosmological parameters remain small, but may increase by combining constraints from complementary cosmological probes. The bias of the $\chi^2$ analysis is due to neglecting the parameter-dependent log-determinant of the data covariance, which gives more statistical weight to larger values of the standardization parameters. We find a similar effect on compressed distance modulus data. To this end, we implement a fully consistent compression method of the JLA data set that uses a Gaussian approximation of the posterior distribution for fast generation of compressed data. Overall, the results of our analysis emphasize the need for a fully consistent Bayesian statistical approach in the analysis of future large SN Ia data sets.

We outline a new method to compute the Bayes Factor for model selection which bypasses the Bayesian Evidence. Our method combines multiple models into a single, nested, Supermodel using one or more hyperparameters. Since the models are now nested the Bayes Factors between the models can be efficiently computed using the Savage-Dickey Density Ratio (SDDR). In this way model selection becomes a problem of parameter estimation. We consider two ways of constructing the supermodel in detail: one based on combined models, and a second based on combined likelihoods. We report on these two approaches for a Gaussian linear model for which the Bayesian evidence can be calculated analytically and a toy nonlinear problem. Unlike the combined model approach, where a standard Monte Carlo Markov Chain (MCMC) struggles, the combined-likelihood approach fares much better in providing a reliable estimate of the log-Bayes Factor. This scheme potentially opens the way to computationally efficient ways to compute Bayes Factors in high dimensions that exploit the good scaling properties of MCMC, as compared to methods such as nested sampling that fail for high dimensions.

We show that multiple machine learning algorithms can match human performance in classifying transient imaging data from the Sloan Digital Sky Survey (SDSS) supernova survey into real objects and artefacts. This is a first step in any transient science pipeline and is currently still done by humans, but future surveys such as the Large Synoptic Survey Telescope (LSST) will necessitate fully machine-enabled solutions. Using features trained from eigenimage analysis (principal component analysis, PCA) of single-epoch g, r and i-difference images, we can reach a completeness (recall) of 96 per cent, while only incorrectly classifying at most 18 per cent of artefacts as real objects, corresponding to a precision (purity) of 84 per cent. In general, random forests performed best, followed by the k-nearest neighbour and the SkyNet artificial neural net algorithms, compared to other methods such as na\"ive Bayes and kernel support vector machine. Our results show that PCA-based machine learning can match human success levels and can naturally be extended by including multiple epochs of data, transient colours and host galaxy information which should allow for significant further improvements, especially at low signal-to-noise.

Classifying transients based on multi band light curves is a challenging but crucial problem in the era of GAIA and LSST since the sheer volume of transients will make spectroscopic classification unfeasible. Here we present a nonparametric classifier that uses the transient's light curve measurements to predict its class given training data. It implements two novel components: the first is the use of the BAGIDIS wavelet methodology - a characterization of functional data using hierarchical wavelet coefficients. The second novelty is the introduction of a ranked probability classifier on the wavelet coefficients that handles both the heteroscedasticity of the data in addition to the potential non-representativity of the training set. The ranked classifier is simple and quick to implement while a major advantage of the BAGIDIS wavelets is that they are translation invariant, hence they do not need the light curves to be aligned to extract features. Further, BAGIDIS is nonparametric so it can be used for blind searches for new objects. We demonstrate the effectiveness of our ranked wavelet classifier against the well-tested Supernova Photometric Classification Challenge dataset in which the challenge is to correctly classify light curves as Type Ia or non-Ia supernovae. We train our ranked probability classifier on the spectroscopically-confirmed subsample (which is not representative) and show that it gives good results for all supernova with observed light curve timespans greater than 100 days (roughly 55% of the dataset). For such data, we obtain a Ia efficiency of 80.5% and a purity of 82.4% yielding a highly competitive score of 0.49 whilst implementing a truly "model-blind" approach to supernova classification. Consequently this approach may be particularly suitable for the classification of astronomical transients in the era of large synoptic sky surveys.

There are two redshifts in cosmology: $z_{obs}$, the observed redshift computed via spectral lines, and the model redshift, $z$, defined by the effective FLRW scale factor. In general these do not coincide. We place observational constraints on the allowed distortions of $z$ away from $z_{obs}$ - a possibility we dub redshift remapping. Remapping is degenerate with cosmic dynamics for either $d_L(z)$ or $H(z)$ observations alone: for example, the simple remapping $z = \alpha_1 z_{obs} +\alpha_2 z_{obs}^2$ allows a decelerating Einstein de Sitter universe to fit the observed supernova Hubble diagram as successfully as $\Lambda$CDM, highlighting that supernova data alone cannot prove that the universe is accelerating. We show however, that redshift remapping leads to apparent violations of cosmic distance duality that can be used to detect its presence even when neither a specific theory of gravity nor the Copernican Principle are assumed. Combining current data sets favours acceleration but does not yet rule out redshift remapping as an alternative to dark energy. Future surveys, however, will provide exquisite constraints on remapping and any models -- such as backreaction -- that predict it.

Radio interferometers suffer from the problem of missing information in their data, due to the gaps between the antennas. This results in artifacts, such as bright rings around sources, in the images obtained. Multiple deconvolution algorithms have been proposed to solve this problem and produce cleaner radio images. However, these algorithms are unable to correctly estimate uncertainties in derived scientific parameters or to always include the effects of instrumental errors. We propose an alternative technique called Bayesian Inference for Radio Observations (BIRO) which uses a Bayesian statistical framework to determine the scientific parameters and instrumental errors simultaneously directly from the raw data, without making an image. We use a simple simulation of Westerbork Synthesis Radio Telescope data including pointing errors and beam parameters as instrumental effects, to demonstrate the use of BIRO.

New telescopes like the Square Kilometre Array (SKA) will push into a new sensitivity regime and expose systematics, such as direction-dependent effects, that could previously be ignored. Current methods for handling such systematics rely on alternating best estimates of instrumental calibration and models of the underlying sky, which can lead to inadequate uncertainty estimates and biased results because any correlations between parameters are ignored. These deconvolution algorithms produce a single image that is assumed to be a true representation of the sky, when in fact it is just one realization of an infinite ensemble of images compatible with the noise in the data. In contrast, here we report a Bayesian formalism that simultaneously infers both systematics and science. Our technique, Bayesian Inference for Radio Observations (BIRO), determines all parameters directly from the raw data, bypassing image-making entirely, by sampling from the joint posterior probability distribution. This enables it to derive both correlations and accurate uncertainties, making use of the flexible software MEQTREES to model the sky and telescope simultaneously. We demonstrate BIRO with two simulated sets of Westerbork Synthesis Radio Telescope data sets. In the first, we perform joint estimates of 103 scientific (flux densities of sources) and instrumental (pointing errors, beamwidth and noise) parameters. In the second example, we perform source separation with BIRO. Using the Bayesian evidence, we can accurately select between a single point source, two point sources and an extended Gaussian source, allowing for 'super-resolution' on scales much smaller than the synthesized beam.

The Square Kilometre Array (SKA) will conduct the biggest spectroscopic galaxy survey ever, by detecting the 21cm emission line of neutral hydrogen (HI) from around a billion galaxies over 3/4 of the sky, out to a redshift of z~2. This will allow the redshift-space matter power spectrum, and corresponding dark energy observables, to be measured with unprecedented precision. In this paper, we present an improved model of the HI galaxy number counts and bias from semi-analytic simulations, and use it to calculate the expected yield of HI galaxies from surveys with a variety of Phase 1 and 2 SKA configurations. We illustrate the relative performance of the different surveys by forecasting errors on the radial and transverse scales of the baryon acoustic oscillation (BAO) feature, finding that the full "billion galaxy survey" with SKA2 will deliver the largest dark energy figure of merit of any current or future large-scale structure survey.

New supernova surveys such as the Dark Energy Survey, Pan-STARRS and the LSST will produce an unprecedented number of photometric supernova candidates, most with no spectroscopic data. Avoiding biases in cosmological parameters due to the resulting inevitable contamination from non-Ia supernovae can be achieved with the BEAMS formalism, allowing for fully photometric supernova cosmology studies. Here we extend BEAMS to deal with the case in which the supernovae are correlated by systematic uncertainties. The analytical form of the full BEAMS posterior requires evaluating 2^N terms, where N is the number of supernova candidates. This `exponential catastrophe' is computationally unfeasible even for N of order 100. We circumvent the exponential catastrophe by marginalising numerically instead of analytically over the possible supernova types: we augment the cosmological parameters with nuisance parameters describing the covariance matrix and the types of all the supernovae, \tau_i, that we include in our MCMC analysis. We show that this method deals well even with large, unknown systematic uncertainties without a major increase in computational time, whereas ignoring the correlations can lead to significant biases and incorrect credible contours. We then compare the numerical marginalisation technique with a perturbative expansion of the posterior based on the insight that future surveys will have exquisite light curves and hence the probability that a given candidate is a Type Ia will be close to unity or zero, for most objects. Although this perturbative approach changes computation of the posterior from a 2^N problem into an N^2 or N^3 one, we show that it leads to biases in general through a small number of misclassifications, implying that numerical marginalisation is superior.

For future surveys, spectroscopic follow-up for all supernovae will be extremely difficult. However, one can use light curve fitters, to obtain the probability that an object is a Type Ia. One may consider applying a probability cut to the data, but we show that the resulting non-Ia contamination can lead to biases in the estimation of cosmological parameters. A different method, which allows the use of the full dataset and results in unbiased cosmological parameter estimation, is Bayesian Estimation Applied to Multiple Species (BEAMS). BEAMS is a Bayesian approach to the problem which includes the uncertainty in the types in the evaluation of the posterior. Here we outline the theory of BEAMS and demonstrate its effectiveness using both simulated datasets and SDSS-II data. We also show that it is possible to use BEAMS if the data are correlated, by introducing a numerical marginalisation over the types of the objects. This is largely a pedagogical introduction to BEAMS with references to the main BEAMS papers.

The Fisher Information Matrix formalism is extended to cases where the data is divided into two parts (X,Y), where the expectation value of Y depends on X according to some theoretical model, and X and Y both have errors with arbitrary covariance. In the simplest case, (X,Y) represent data pairs of abscissa and ordinate, in which case the analysis deals with the case of data pairs with errors in both coordinates, but X can be any measured quantities on which Y depends. The analysis applies for arbitrary covariance, provided all errors are gaussian, and provided the errors in X are small, both in comparison with the scale over which the expected signal Y changes, and with the width of the prior distribution. This generalises the Fisher Matrix approach, which normally only considers errors in the `ordinate' Y. In this work, we include errors in X by marginalising over latent variables, effectively employing a Bayesian hierarchical model, and deriving the Fisher Matrix for this more general case. The methods here also extend to likelihood surfaces which are not gaussian in the parameter space, and so techniques such as DALI (Derivative Approximation for Likelihoods) can be generalised straightforwardly to include arbitrary gaussian data error covariances. For simple mock data and theoretical models, we compare to Markov Chain Monte Carlo experiments, illustrating the method with cosmological supernova data. We also include the new method in the Fisher4Cast software.

We present cosmological constraints from a joint analysis of type Ia supernova (SN Ia) observations obtained by the SDSS-II and SNLS collaborations. The data set includes several low-redshift samples (z<0.1), all 3 seasons from the SDSS-II (0.05 < z < 0.4), and 3 years from SNLS (0.2 <z < 1) and totals \ntotc spectroscopically confirmed type Ia supernovae with high quality light curves. We have followed the methods and assumptions of the SNLS 3-year data analysis except for the following important improvements: 1) the addition of the full SDSS-II spectroscopically-confirmed SN Ia sample in both the training of the SALT2 light curve model and in the Hubble diagram analysis (\nsdssc SNe), 2) inter-calibration of the SNLS and SDSS surveys and reduced systematic uncertainties in the photometric calibration, performed blindly with respect to the cosmology analysis, and 3) a thorough investigation of systematic errors associated with the SALT2 modeling of SN Ia light-curves. We produce recalibrated SN Ia light-curves and associated distances for the SDSS-II and SNLS samples. The large SDSS-II sample provides an effective, independent, low-z anchor for the Hubble diagram and reduces the systematic error from calibration systematics in the low-z SN sample. For a flat LCDM cosmology we find Omega_m=0.295+-0.034 (stat+sys), a value consistent with the most recent CMB measurement from the Planck and WMAP experiments. Our result is 1.8sigma (stat+sys) different than the previously published result of SNLS 3-year data. The change is due primarily to improvements in the SNLS photometric calibration. When combined with CMB constraints, we measure a constant dark-energy equation of state parameter w=-1.018+-0.057 (stat+sys) for a flat universe. Adding BAO distance measurements gives similar constraints: w=-1.027+-0.055.

This paper describes the data release of the Sloan Digital Sky Survey-II (SDSS-II) Supernova Survey conducted between 2005 and 2007. Light curves, spectra, classifications, and ancillary data are presented for 10,258 variable and transient sources discovered through repeat ugriz imaging of SDSS Stripe 82, a 300 deg2 area along the celestial equator. This data release is comprised of all transient sources brighter than r~22.5 mag with no history of variability prior to 2004. Dedicated spectroscopic observations were performed on a subset of 889 transients, as well as spectra for thousands of transient host galaxies using the SDSS-III BOSS spectrographs. Photometric classifications are provided for the candidates with good multi-color light curves that were not observed spectroscopically. From these observations, 4607 transients are either spectroscopically confirmed, or likely to be, supernovae, making this the largest sample of supernova candidates ever compiled. We present a new method for SN host-galaxy identification and derive host-galaxy properties including stellar masses, star-formation rates, and the average stellar population ages from our SDSS multi-band photometry. We derive SALT2 distance moduli for a total of 1443 SN Ia with spectroscopic redshifts as well as photometric redshifts for a further 677 purely-photometric SN Ia candidates. Using the spectroscopically confirmed subset of the three-year SDSS-II SN Ia sample and assuming a flat Lambda-CDM cosmology, we determine Omega_M = 0.315 +/- 0.093 (statistical error only) and detect a non-zero cosmological constant at 5.7 sigmas.

Using a sample of 608 Type Ia supernovae from the SDSS-II and BOSS surveys, combined with a sample of foreground galaxies from SDSS-II, we estimate the weak lensing convergence for each supernova line-of-sight. We find that the correlation between this measurement and the Hubble residuals is consistent with the prediction from lensing (at a significance of 1.7sigma. Strong correlations are also found between the residuals and supernova nuisance parameters after a linear correction is applied. When these other correlations are taken into account, the lensing signal is detected at 1.4sigma. We show for the first time that distance estimates from supernovae can be improved when lensing is incorporated by including a new parameter in the SALT2 methodology for determining distance moduli. The recovered value of the new parameter is consistent with the lensing prediction. Using CMB data from WMAP7, H0 data from HST and SDSS BAO measurements, we find the best-fit value of the new lensing parameter and show that the central values and uncertainties on Omega_m and w are unaffected. The lensing of supernovae, while only seen at marginal significance in this low redshift sample, will be of vital importance for the next generation of surveys, such as DES and LSST, which will be systematics dominated.

We present the spectroscopy from 5254 galaxies that hosted supernovae (SNe) or other transient events in the Sloan Digital Sky Survey II (SDSS-II). Obtained during SDSS-I, SDSS-II, and the Baryon Oscillation Spectroscopic Survey (BOSS), this sample represents the largest systematic, unbiased, magnitude limited spectroscopic survey of supernova (SN) host galaxies. Using the host galaxy redshifts, we test the impact of photometric SN classification based on SDSS imaging data with and without using spectroscopic redshifts of the host galaxies. Following our suggested scheme, there are a total of 1166 photometrically classified SNe Ia when using a flat redshift prior and 1126 SNe Ia when the host spectroscopic redshift is assumed. For 1024 (87.8%) candidates classified as likely SNe Ia without redshift information, we find that the classification is unchanged when adding the host galaxy redshift. Using photometry from SDSS imaging data and the host galaxy spectra, we also report host galaxy properties for use in future nalysis of SN astrophysics. Finally, we investigate the differences in the interpretation of the light curve properties with and without knowledge of the redshift. When using the SALT2 light curve fitter, we find a 21% increase in the number of fits that converge when using the spectroscopic redshift. Without host galaxy redshifts, we find that SALT2 light curve fits are systematically biased towards lower photometric redshift estimates and redder colors in the limit of low signal-to-noise data. The general improvements in performance of the light curve fitter and the increased diversity of the host galaxy sample highlights the importance of host galaxy spectroscopy for current photometric SN surveys such as the Dark Energy Survey and future surveys such as the Large Synoptic Survey Telescope.

We present the cosmological analysis of 752 photometrically-classified Type Ia Supernovae (SNe Ia) obtained from the full Sloan Digital Sky Survey II (SDSS-II) Supernova (SN) Survey, supplemented with host-galaxy spectroscopy from the SDSS-III Baryon Oscillation Spectroscopic Survey (BOSS). Our photometric-classification method is based on the SN typing technique of Sako et al. (2011), aided by host galaxy redshifts (0.05<z<0.55). SNANA simulations of our methodology estimate that we have a SN Ia typing efficiency of 70.8%, with only 3.9% contamination from core-collapse (non-Ia) SNe. We demonstrate that this level of contamination has no effect on our cosmological constraints. We quantify and correct for our selection effects (e.g., Malmquist bias) using simulations. When fitting to a flat LambdaCDM cosmological model, we find that our photometric sample alone gives omega_m=0.24+0.07-0.05 (statistical errors only). If we relax the constraint on flatness, then our sample provides competitive joint statistical constraints on omega_m and omega_lambda, comparable to those derived from the spectroscopically-confirmed three-year Supernova Legacy Survey (SNLS3). Using only our data, the statistics-only result favors an accelerating universe at 99.96% confidence. Assuming a constant wCDM cosmological model, and combining with H0, CMB and LRG data, we obtain w=-0.96+0.10-0.10, omega_m=0.29+0.02-0.02 and omega_k=0.00+0.03-0.02 (statistical errors only), which is competitive with similar spectroscopically confirmed SNe Ia analyses. Overall this comparison is re-assuring, considering the lower redshift leverage of the SDSS-II SN sample (z<0.55) and the lack of spectroscopic confirmation used herein. These results demonstrate the potential of photometrically-classified SNe Ia samples in improving cosmological constraints.

Distance measurement provide no constraints on curvature independent of assumptions about the dark energy, raising the question, how flat is our Universe if we make no such assumptions? Allowing for general evolution of the dark energy equation of state with 20 free parameters that are allowed to cross the phantom divide, w(z) = -1, we show that while it is indeed possible to match the first peak in the Cosmic Microwave Background with non-flat models and arbitrary Hubble constant, H_0, the full WMAP7 and supernova data alone imply -0.12 < \Omega_k < 0.01 (2\sigma). If we add an H_0 prior, this tightens significantly to \Omega_k = 0.002 \pm 0.009 . These constitute the most conservative and model-independent constraints on curvature available today, and illustrate that the curvature-dynamics degeneracy is broken by current data, with a key role played by the Integrated Sachs Wolfe effect rather than the distance to the surface of last scattering. If one imposes a quintessence prior on the dark energy (-1 \leq w(z) \leq 1) then just the WMAP7 and supernova data alone force the Universe to near flatness: \Omega_k = 0.013 \pm 0.012. Finally, allowing for curvature, we find that all datasets are consistent with a Harrison-Zel'dovich spectral index, n_s = 1, at 2\sigma, illustrating the interplay between early and late-universe constraints.

We introduce Bayesian Estimation Applied to Multiple Species (BEAMS), an algorithm designed to deal with parameter estimation when using contaminated data. We present the algorithm and demonstrate how it works with the help of a Gaussian simulation. We then apply it to supernova data from the Sloan Digital Sky Survey (SDSS), showing how the resulting confidence contours of the cosmological parameters shrink significantly.

Baryon Acoustic Oscillation (BAO) surveys will be a leading method for addressing the dark energy challenge in the next decade. We explore in detail the effect of allowing for small amplitude admixtures of general isocurvature perturbations in addition to the dominant adiabatic mode. We find that non-adiabatic initial conditions leave the sound speed unchanged but instead excite different harmonics. These harmonics couple differently to Silk damping, altering the form and evolution of acoustic waves in the baryon-photon fluid prior to decoupling. This modifies not only the scale on which the sound waves imprint onto the baryon distribution, which is used as the standard ruler in BAO surveys, but also the shape, width and height of the BAO peak. We discuss these effects in detail and show how more general initial conditions impact our interpretation of cosmological data in dark energy studies. We find that the inclusion of these additional isocurvature modes leads to an increase in the Dark Energy Task Force Figure of merit by 140% and 60% for the BOSS and ADEPT experiments respectively when considered in conjunction with Planck data. We also show that the incorrect assumption of adiabaticity has the potential to bias our estimates of the dark energy parameters by $3\sigma$ ($1\sigma$) for a single correlated isocurvature mode, and up to $8\sigma$ ($3\sigma$) for three correlated isocurvature modes in the case of the BOSS (ADEPT) experiment. We find that the use of the large scale structure data in conjunction with CMB data improves our ability to measure the contributions of different modes to the initial conditions by as much as 100% for certain modes in the fully correlated case.

We study SN 2006oz, a newly-recognized member of the class of H-poor, super-luminous supernovae. We present multi-color light curves from the SDSS-II SN Survey, that cover the rise time, as well as an optical spectrum that shows that the explosion occurred at z~0.376. We fitted black body functions to estimate the temperature and radius evolution of the photosphere and used the parametrized code SYNOW to model the spectrum. We constructed a bolometric light curve and compared it with explosion models. The very early light curves show a dip in the g- and r-bands and a possible initial cooling phase in the u-band before rising to maximum light. The bolometric light curve shows a precursor plateau with a duration of 6-10 days in the rest-frame. A lower limit of M_u < -21.5 can be placed on the absolute peak luminosity of the SN, while the rise time is constrained to be at least 29 days. During our observations, the emitting sphere doubled its radius to 2x10^15 cm, while the temperature remained hot at 15000 K. As for other similar SNe, the spectrum is best modeled with elements including O II and Mg II, while we tentatively suggest that Fe III might be present. We suggest that the precursor plateau might be related to a recombination wave in a circumstellar medium (CSM) and discuss whether this is a common property of all similar explosions. The subsequent rise can be equally well described by input from a magnetar or by ejecta-CSM interaction, but the models are not well constrained owing to the lack of post-maximum observations, and CSM interaction has difficulties accounting for the precursor plateau self-consistently. Radioactive decay is less likely to be the mechanism that powers the luminosity. The host galaxy, detected in deep imaging with the 10 m GTC, is a moderately young and star-forming, but not a starburst, galaxy. It has an absolute magnitude of M_g = -16.9.

The Fisher Matrix is the backbone of modern cosmological forecasting. We describe the Fisher4Cast software: a general-purpose, easy-to-use, Fisher Matrix framework. It is open source, rigorously designed and tested and includes a Graphical User Interface (GUI) with automated LATEX file creation capability and point-and-click Fisher ellipse generation. Fisher4Cast was designed for ease of extension and, although written in Matlab, is easily portable to open-source alternatives such as Octave and Scilab. Here we use Fisher4Cast to present new 3-D and 4-D visualisations of the forecasting landscape and to investigate the effects of growth and curvature on future cosmological surveys. Early releases have been available at http://www.cosmology.org.za since May 2008 with 750 downloads in the first year. Version 2.2 is made public with this paper and includes a Quick Start guide and the code used to produce the figures in this paper, in the hope that it will be useful to the cosmology and wider scientific communities.

We show that in cases of marginal detections (~ 3\sigma), such as that of Baryonic Acoustic Oscillations (BAO) in cosmology, the often-used Gaussian approximation to the full likelihood is very poor, especially beyond ~3\sigma. This can radically alter confidence intervals on parameters and implies that one cannot naively extrapolate 1\sigma-errorbars to 3\sigma, and beyond. We propose a simple fitting formula which corrects for this effect in posterior probabilities arising from marginal detections. Alternatively the full likelihood should be used for parameter estimation rather than the Gaussian approximation of a just mean and an error.

We present Keck high-quality rest-frame ultraviolet (UV) through optical spectra of 21 Type Ia supernovae (SNe Ia) in the redshift range 0.11 < z < 0.37 and a mean redshift of 0.22 that were discovered during the Sloan Digital Sky Survey-II (SDSS-II) SN Survey. Using the broad-band photometry of the SDSS survey, we are able to reconstruct the SN host-galaxy spectral energy distributions (SEDs), allowing for a correction for the host-galaxy contamination in the SN Ia spectra. Comparison of composite spectra constructed from a subsample of 17 high-quality spectra to those created from a low-redshift sample with otherwise similar properties shows that the Keck/SDSS SNe Ia have, on average, extremely similar rest-frame optical spectra but show a UV flux excess. This observation is confirmed by comparing synthesized broad-band colors of the individual spectra, showing a difference in mean colors at the 2.4 - 4.4 sigma level for various UV colors. We further see a slight difference in the UV spectral shape between SNe with low-mass and high-mass host galaxies. Additionally, we detect a relationship between the flux ratio at 2770 and 2900 A and peak luminosity that differs from that observed at low redshift. We find that changing the UV SED of an SN Ia within the observed dispersion can change the inferred distance moduli by ~0.1 mag. This effect only occurs when the data probe the rest-frame UV. We suggest that this discrepancy could be due to differences in the host-galaxy population of the two SN samples or to small-sample statistics.

Supernova cosmology without spectroscopic confirmation is an exciting new frontier which we address here with the Bayesian Estimation Applied to Multiple Species (BEAMS) algorithm and the full three years of data from the Sloan Digital Sky Survey II Supernova Survey (SDSS-II SN). BEAMS is a Bayesian framework for using data from multiple species in statistical inference when one has the probability that each data point belongs to a given species, corresponding in this context to different types of supernovae with their probabilities derived from their multi-band lightcurves. We run the BEAMS algorithm on both Gaussian and more realistic SNANA simulations with of order 10^4 supernovae, testing the algorithm against various pitfalls one might expect in the new and somewhat uncharted territory of photometric supernova cosmology. We compare the performance of BEAMS to that of both mock spectroscopic surveys and photometric samples which have been cut using typical selection criteria. The latter typically are either biased due to contamination or have significantly larger contours in the cosmological parameters due to small data-sets. We then apply BEAMS to the 792 SDSS-II photometric supernovae with host spectroscopic redshifts. In this case, BEAMS reduces the area of the (\Omega_m,\Omega_\Lambda) contours by a factor of three relative to the case where only spectroscopically confirmed data are used (297 supernovae). In the case of flatness, the constraints obtained on the matter density applying BEAMS to the photometric SDSS-II data are \Omega_m(BEAMS)=0.194\pm0.07. This illustrates the potential power of BEAMS for future large photometric supernova surveys such as LSST.

The original formulation of BEAMS - Bayesian Estimation Applied to Multiple Species - showed how to use a dataset contaminated by points of multiple underlying types to perform unbiased parameter estimation. An example is cosmological parameter estimation from a photometric supernova sample contaminated by unknown Type Ibc and II supernovae. Where other methods require data cuts to increase purity, BEAMS uses all of the data points in conjunction with their probabilities of being each type. Here we extend the BEAMS formalism to allow for correlations between the data and the type probabilities of the objects as can occur in realistic cases. We show with simple simulations that this extension can be crucial, providing a 50% reduction in parameter estimation variance when such correlations do exist. We then go on to perform tests to quantify the importance of the type probabilities, one of which illustrates the effect of biasing the probabilities in various ways. Finally, a general presentation of the selection bias problem is given, and discussed in the context of future photometric supernova surveys and BEAMS, which lead to specific recommendations for future supernova surveys.

Using data from the Sloan Digital Sky Supernova Survey-II, we measure the rate of Type Ia Supernovae (SNe Ia) as a function of galaxy properties at intermediate redshift. A sample of 342 SNe Ia with 0.05<z<0.25 is constructed. Using broad-band photometry we use the PEGASE spectral energy distributions (SEDs) to estimate host galaxy stellar masses and recent star-formation rates. We find that the rate of SNe Ia per unit stellar mass is significantly higher (by a factor of ~30) in highly star-forming galaxies compared to passive galaxies. When parameterizing the SN Ia rate (SNR_Ia) based on host galaxy properties, we find that the rate of SNe Ia in passive galaxies is not linearly proportional to the stellar mass, instead a SNR_Ia proportional to M^0.68 is favored. However, such a parameterization does not describe the observed SN Ia rate in star-forming galaxies. The SN Ia rate in star-forming galaxies is well fit by SNR_Ia = 1.05\pm0.16x10^{-10} M ^{0.68\pm0.01} + 1.01\pm0.09x10^{-3} SFR^{1.00\pm0.05} (statistical errors only), where M is the host galaxy mass and SFR is the star-formation rate. These results are insensitive to the selection criteria used, redshift limit considered and the inclusion of non-spectroscopically confirmed SNe Ia. We also show there is a dependence between the distribution of the MLCS light-curve decline rate parameter, \Delta, and host galaxy type. Passive galaxies host less luminous SNe Ia than seen in moderately and highly star-forming galaxies, although a population of luminous SNe is observed in passive galaxies, contradicting previous assertions that these SNe Ia are only observed in younger stellar systems. The MLCS extinction parameter, A_V, is similar in passive and moderately star-forming galaxies, but we find indications that it is smaller, on average, in highly star-forming galaxies. We confirm these results using the SALT2 light-curve fitter.

We present an analysis of peculiar velocities and their effect on supernova cosmology. In particular, we study (a) the corrections due to our own motion, (b) the effects of correlations in peculiar velocities induced by large-scale structure, and (c) uncertainties arising from a possible local under- or over-density. For all of these effects we present a case study of their impact on the cosmology derived by the Sloan Digital Sky Survey-II Supernova Survey (SDSS-II SN Survey). Correcting supernova redshifts for the CMB dipole slightly over-corrects nearby supernovae that share some of our local motion. We show that while neglecting the CMB dipole would cause a shift in the derived equation of state of Delta w ~ 0.04 (at fixed matter density) the additional local-motion correction is currently negligible (Delta w<0.01). We use a covariance-matrix approach to statistically account for correlated peculiar velocities. This down-weights nearby supernovae and effectively acts as a graduated version of the usual sharp low-redshift cut. Neglecting coherent velocities in the current sample causes a systematic shift of ~2% in the preferred value of w and will therefore have to be considered carefully when future surveys aim for percent-level accuracy. Finally, we perform n-body simulations to estimate the likely magnitude of any local density fluctuation (monopole) and estimate the impact as a function of the low-redshift cutoff. We see that for this aspect the low-z cutoff of z=0.02 is well-justified theoretically, but that living in a putative local density fluctuation leaves an indelible imprint on the magnitude-redshift relation.

We improve estimates of stellar mass and mass-weighted average age of Type Ia supernova (SN Ia) host galaxies by combining UV and near-IR photometry with optical photometry in our analysis. Using 206 SNe Ia drawn from the full three-year SDSS-II Supernova Survey (median redshift of z {\approx} 0.2) and multi-wavelength host-galaxy photometry from SDSS, GALEX, and UKIDSS, we present evidence of a correlation (1.9{\sigma} confidence level) between the residuals of SNe Ia about the best-fit Hubble relation and the mass-weighted average age of their host galaxies. The trend is such that older galaxies host SNe Ia that are brighter than average after standard light-curve corrections are made. We also confirm, at the 3.0{\sigma} level, the trend seen by previous studies that more massive galaxies often host brighter SNe Ia after light-curve correction.

We analyze the three-year SDSS-II Superernova (SN) Survey data and identify a sample of 1070 photometric SN Ia candidates based on their multi-band light curve data. This sample consists of SN candidates with no spectroscopic confirmation, with a subset of 210 candidates having spectroscopic redshifts of their host galaxies measured, while the remaining 860 candidates are purely photometric in their identification. We describe a method for estimating the efficiency and purity of photometric SN Ia classification when spectroscopic confirmation of only a limited sample is available, and demonstrate that SN Ia candidates from SDSS-II can be identified photometrically with ~91% efficiency and with a contamination of ~6%. Although this is the largest uniform sample of SN candidates to date for studying photometric identification, we find that a larger spectroscopic sample of contaminating sources is required to obtain a better characterization of the background events. A Hubble diagram using SN candidates with no spectroscopic confirmation, but with host galaxy spectroscopic redshifts, yields a distance modulus dispersion that is only ~20 - 40% larger than that of the spectroscopically-confirmed SN Ia sample alone with no significant bias. A Hubble diagram with purely photometric classification and redshift-distance measurements, however, exhibit biases that require further investigation for precision cosmology.

Aims: Spectroscopic observations of Type Ia supernovae obtained at the New Technology Telescope (NTT) and the Nordic Optical Telescope (NOT), in conjunction with the SDSS-II Supernova Survey, are analysed. We use spectral indicators measured up to a month after the lightcurve peak luminosity to characterise the supernova properties, and examine these for potential correlations with host galaxy type, lightcurve shape, colour excess, and redshift. Methods: Our analysis is based on 89 Type Ia supernovae at a redshift interval z = 0.05 - 0.3, for which multiband SDSS photometry is available. A lower-z spectroscopy reference sample was used for comparisons over cosmic time. We present measurements of time series of pseudo equivalent widths and line velocities of the main spectral features in Type Ia supernovae. Results: Supernovae with shallower features are found predominantly among the intrinsically brighter slow declining supernovae. We detect the strongest correlation between lightcurve stretch and the Si ii 4000 absorption feature, which also correlates with the estimated mass and star formation rate of the host galaxy. We also report a tentative correlation between colour excess and spectral properties. If confirmed, this would suggest that moderate reddening of Type Ia supernovae is dominated by effects in the explosion or its immediate environment, as opposed to extinction by interstellar dust.

Context. The SDSS-II Supernova Survey, conducted between 2005 and 2007, was designed to detect a large number of Type Ia supernovae (SNe Ia) around z~0.2, the redshift "gap" between low-z and high-z SN searches. The survey has provided multi-band photometric lightcurves for variable targets, and SN candidates were scheduled for spectroscopic observations, primarily to provide SN classification and accurate redshifts. We present SN spectra obtained in 2006 and 2007 using the NTT and the NOT. Aims. We provide an atlas of SN spectra in the range z =0.03-0.32 that complements the well-sampled lightcurves from SDSS-II in the forthcoming three-year SDSS SN cosmology analysis. The sample can, for example, be used for spectral studies of SNe Ia, which are critical for understanding potential systematic effects when SNe are used to determine cosmological distances. Methods. The spectra were reduced in a uniform manner, and special care was taken in estimating the uncertainties for the different processing steps. Host-galaxy light was subtracted when possible and the SN type fitted using the SuperNova IDentification code (SNID). We also present comparisons between spectral and photometric dating using SALT lightcurve fits to the photometry from SDSS-II, as well as the global distribution of our sample in terms of the lightcurve parameters: stretch and colour. Results. We report new spectroscopic data from 141 SNe Ia, mainly between -9 and +15 days from lightcurve maximum, including a few cases of multi-epoch observations. This homogeneous, host-galaxy subtracted, SN Ia spectroscopic sample is among the largest such data sets and unique in its redshift interval. The sample includes two potential SN 1991T-like SNe (SN 2006on and SN 2007ni) and one potential SN 2002cx-like SN (SN 2007ie). In addition, the new compilation includes spectra from 23 confirmed Type II and 8 Type Ib/c SNe.

We report results from the Supernova Photometric Classification Challenge (SNPCC), a publicly released mix of simulated supernovae (SNe), with types (Ia, Ibc, and II) selected in proportion to their expected rate. The simulation was realized in the griz filters of the Dark Energy Survey (DES) with realistic observing conditions (sky noise, point-spread function and atmospheric transparency) based on years of recorded conditions at the DES site. Simulations of non-Ia type SNe are based on spectroscopically confirmed light curves that include unpublished non-Ia samples donated from the Carnegie Supernova Project (CSP), the Supernova Legacy Survey (SNLS), and the Sloan Digital Sky Survey-II (SDSS-II). A spectroscopically confirmed subset was provided for training. We challenged scientists to run their classification algorithms and report a type and photo-z for each SN. Participants from 10 groups contributed 13 entries for the sample that included a host-galaxy photo-z for each SN, and 9 entries for the sample that had no redshift information. Several different classification strategies resulted in similar performance, and for all entries the performance was significantly better for the training subset than for the unconfirmed sample. For the spectroscopically unconfirmed subset, the entry with the highest average figure of merit for classifying SNe~Ia has an efficiency of 0.96 and an SN~Ia purity of 0.79. As a public resource for the future development of photometric SN classification and photo-z estimators, we have released updated simulations with improvements based on our experience from the SNPCC, added samples corresponding to the Large Synoptic Survey Telescope (LSST) and the SDSS, and provided the answer keys so that developers can evaluate their own analysis.

Future photometric supernova surveys will produce vastly more candidates than can be followed up spectroscopically, highlighting the need for effective classification methods based on lightcurves alone. Here we introduce boosting and kernel density estimation techniques which have minimal astrophysical input, and compare their performance on 20,000 simulated Dark Energy Survey lightcurves. We demonstrate that these methods are comparable to the best template fitting methods currently used, and in particular do not require the redshift of the host galaxy or candidate. However both methods require a training sample that is representative of the full population, so typical spectroscopic supernova subsamples will lead to poor performance. To enable the full potential of such blind methods, we recommend that representative training samples should be used and so specific attention should be given to their creation in the design phase of future photometric surveys.

We present an analysis of the host galaxy dependencies of Type Ia Supernovae (SNe Ia) from the full three year sample of the SDSS-II Supernova Survey. We rediscover, to high significance, the strong correlation between host galaxy typeand the width of the observed SN light curve, i.e., fainter, quickly declining SNe Ia favor passive host galaxies, while brighter, slowly declining Ia's favor star-forming galaxies. We also find evidence (at between 2 to 3 sigma) that SNe Ia are ~0.1 magnitudes brighter in passive host galaxies, than in star-forming hosts, after the SN Ia light curves have been standardized using the light curve shape and color variations: This difference in brightness is present in both the SALT2 and MCLS2k2 light curve fitting methodologies. We see evidence for differences in the SN Ia color relationship between passive and star-forming host galaxies, e.g., for the MLCS2k2 technique, we see that SNe Ia in passive hosts favor a dust law of R_V ~1, while SNe Ia in star-forming hosts require R_V ~2. The significance of these trends depends on the range of SN colors considered. We demonstrate that these effects can be parameterized using the stellar mass of the host galaxy (with a confidence of >4 sigma) and including this extra parameter provides a better statistical fit to our data. Our results suggest that future cosmological analyses of SN Ia samples should include host galaxy information.

We present multi-band photometry and multi-epoch spectroscopy of the peculiar Type Ia supernova (SN Ia) 2007qd, discovered by the SDSS-II Supernova Survey. It possesses physical properties intermediate to those of the peculiar SN 2002cx and the extremely low-luminosity SN 2008ha. Optical photometry indicates that it had an extraordinarily fast rise time of <= 10 days and a peak absolute B magnitude of -15.4 +/- 0.2 at most, making it one of the most subluminous SN Ia ever observed. Follow-up spectroscopy of SN 2007qd near maximum brightness unambiguously shows the presence of intermediate-mass elements which are likely caused by carbon/oxygen nuclear burning. Near maximum brightness, SN 2007qd had a photospheric velocity of only 2800 km/s, similar to that of SN 2008ha but about 4000 and 7000 km/s less than that of SN 2002cx and normal SN Ia, respectively. We show that the peak luminosities of SN 2002cx-like objects are highly correlated with both their light-curve stretch and photospheric velocities. Its strong apparent connection to other SN 2002cx-like events suggests that SN 2007qd is also a pure deflagration of a white dwarf, although other mechanisms cannot be ruled out. It may be a critical link between SN 2008ha and the other members of the SN 2002cx-like class of objects.

Small fractions of isocurvature perturbations correlated with the dominant adiabatic mode are shown to be a significant primordial systematic for future Baryon Acoustic Oscillation (BAO) surveys, distorting the standard ruler distance by broadening and shifting the peak in the galaxy correlation function. Untreated this systematic leads to biases that can exceed $10\sigma$ in the dark energy parameters even for Planck-level isocurvature constraints. Accounting for the isocurvature modes corrects for this bias but degrades the dark energy figure of merit by at least 50\%. The BAO data in turn provides extremely powerful new constraints on the nature of the primordial perturbations. Future large galaxy surveys will thus be powerful probes of the earliest phase of the universe in addition to helping pin down the nature of dark energy.

Large planned photometric surveys will discover hundreds of thousands of supernovae (SNe), outstripping the resources available for spectroscopic follow-up and necessitating the development of purely photometric methods to exploit these events for cosmological study. We present a light-curve fitting technique for SN Ia photometric redshift (photo-z) estimation in which the redshift is determined simultaneously with the other fit parameters. We implement this "LCFIT+Z" technique within the frameworks of the MLCS2k2 and SALT-II light-curve fit methods and determine the precision on the redshift and distance modulus. This method is applied to a spectroscopically confirmed sample of 296 SNe Ia from the SDSS-II Supernova Survey and 37 publicly available SNe Ia from the Supernova Legacy Survey (SNLS). We have also applied the method to a large suite of realistic simulated light curves for existing and planned surveys, including SDSS, SNLS, and LSST. When intrinsic SN color fluctuations are included, the photo-z precision for the simulation is consistent with that in the data. Finally, we compare the LCFIT+Z photo-z precision with previous results using color-based SN photo-z estimates.

We present a measurement of the volumetric Type Ia supernova (SN Ia) rate based on data from the Sloan Digital Sky Survey II (SDSS-II) Supernova Survey. The adopted sample of supernovae (SNe) includes 516 SNe Ia at redshift z \lesssim 0.3, of which 270 (52%) are spectroscopically identified as SNe Ia. The remaining 246 SNe Ia were identified through their light curves; 113 of these objects have spectroscopic redshifts from spectra of their host galaxy, and 133 have photometric redshifts estimated from the SN light curves. Based on consideration of 87 spectroscopically confirmed non-Ia SNe discovered by the SDSS-II SN Survey, we estimate that 2.04+1.61-0.95 % of the photometric SNe Ia may be misidentified. The sample of SNe Ia used in this measurement represents an order of magnitude increase in the statistics for SN Ia rate measurements in the redshift range covered by the SDSS-II Supernova Survey. If we assume a SN Ia rate that is constant at low redshift (z < 0.15), then the SN observations can be used to infer a value of the SN rate of rV = (2.69+0.34+0.21-0.30-0.01) x10^{-5} SNe yr^{-1} Mpc-3 (H0 /(70 km s^{-1} Mpc^{-1}))^{3} at a mean redshift of ~ 0.12, based on 79 SNe Ia of which 72 are spectroscopically confirmed. However, the large sample of SNe Ia included in this study allows us to place constraints on the redshift dependence of the SN Ia rate based on the SDSS-II Supernova Survey data alone. Fitting a power-law model of the SN rate evolution, r_V(z) = A_p x ((1 + z)/(1 + z0))^{\nu}, over the redshift range 0.0 < z < 0.3 with z0 = 0.21, results in A_p = (3.43+0.15-0.15) x 10^{-5} SNe yr^{-1} Mpc-3 (H0 /(70 km s^{-1} Mpc^{-1}))^{3} and \nu = 2.04+0.90-0.89.

ABRIDGED We present measurements of the Type Ia supernova (SN) rate in galaxy clusters based on data from the Sloan Digital Sky Survey-II (SDSS-II) Supernova Survey. The cluster SN Ia rate is determined from 9 SN events in a set of 71 C4 clusters at z <0.17 and 27 SN events in 492 maxBCG clusters at 0.1 < z < 0.3$. We find values for the cluster SN Ia rate of $({0.37}^{+0.17+0.01}_{-0.12-0.01}) \mathrm{SNu}r h^{2}$ and $({0.55}^{+0.13+0.02}_{-0.11-0.01}) \mathrm{SNu}r h^{2}$ ($\mathrm{SNu}x = 10^{-12} L_{x\sun}^{-1} \mathrm{yr}^{-1}$) in C4 and maxBCG clusters, respectively, where the quoted errors are statistical and systematic, respectively. The SN rate for early-type galaxies is found to be $({0.31}^{+0.18+0.01}_{-0.12-0.01}) \mathrm{SNu}r h^{2}$ and $({0.49}^{+0.15+0.02}_{-0.11-0.01})$ $\mathrm{SNu}r h^{2}$ in C4 and maxBCG clusters, respectively. The SN rate for the brightest cluster galaxies (BCG) is found to be $({2.04}^{+1.99+0.07}_{-1.11-0.04}) \mathrm{SNu}r h^{2}$ and $({0.36}^{+0.84+0.01}_{-0.30-0.01}) \mathrm{SNu}r h^{2}$ in C4 and maxBCG clusters. The ratio of the SN Ia rate in cluster early-type galaxies to that of the SN Ia rate in field early-type galaxies is ${1.94}^{+1.31+0.043}_{-0.91-0.015}$ and ${3.02}^{+1.31+0.062}_{-1.03-0.048}$, for C4 and maxBCG clusters. The SN rate in galaxy clusters as a function of redshift...shows only weak dependence on redshift. Combining our current measurements with previous measurements, we fit the cluster SN Ia rate data to a linear function of redshift, and find $r_{L} = $ $[(0.49^{+0.15}_{-0.14}) +$ $(0.91^{+0.85}_{-0.81}) \times z]$ $\mathrm{SNu}B$ $h^{2}$. A comparison of the radial distribution of SNe in cluster to field early-type galaxies shows possible evidence for an enhancement of the SN rate in the cores of cluster early-type galaxies... we estimate the fraction of cluster SNe that are hostless to be $(9.4^+8._3-5.1)%$.

The paper has been deeply reviewed and compeltely re-written.

We analyze the rise and fall times of type Ia supernova (SN Ia) light curves discovered by the SDSS-II Supernova Survey. From a set of 391 light curves k-corrected to the rest frame B and V bands, we find a smaller dispersion in the rising portion of the light curve compared to the decline. This is in qualitative agreement with computer models which predict that variations in radioactive nickel yield have less impact on the rise than on the spread of the decline rates. The differences we find in the rise and fall properties suggest that a single 'stretch' correction to the light curve phase does not properly model the range of SN Ia light curve shapes. We select a subset of 105 light curves well-observed in both rise and fall portions of the light curves and develop a '2-stretch' fit algorithm which estimates the rise and fall times independently. We find the average time from explosion to B-band peak brightness is 17.38 +/- 0.17 days. Our average rise time is shorter than the 19.5 days found in previous studies; this reflects both the different light curve template used and the application of the 2-stretch algorithm. We find that slow declining events tend to have fast rise times, but that the distribution of rise minus fall time is broad and single-peaked. This distribution is in contrast to the bimodality in this parameter that was first suggested by Strovink (2007) from an analysis of a small set of local SNe Ia. We divide the SDSS-II sample in half based on the rise minus fall value, tr-tf <= 2 days and tr-tf>2 days, to search for differences in their host galaxy properties and Hubble residuals; we find no difference in host galaxy properties or Hubble residuals in our sample.

Baryon Acoustic Oscillations (BAO) are frozen relics left over from the pre-decoupling universe. They are the standard rulers of choice for 21st century cosmology, providing distance estimates that are, for the first time, firmly rooted in well-understood, linear physics. This review synthesises current understanding regarding all aspects of BAO cosmology, from the theoretical and statistical to the observational, and includes a map of the future landscape of BAO surveys, both spectroscopic and photometric.

We present an analysis of the luminosity distances of Type Ia Supernovae from the Sloan Digital Sky Survey-II (SDSS-II) Supernova Survey in conjunction with other intermediate redshift (z<0.4) cosmological measurements including redshift-space distortions from the Two-degree Field Galaxy Redshift Survey (2dFGRS), the Integrated Sachs-Wolfe (ISW) effect seen by the SDSS, and the latest Baryon Acoustic Oscillation (BAO) distance scale from both the SDSS and 2dFGRS. We have analysed the SDSS-II SN data alone using a variety of "model-independent" methods and find evidence for an accelerating universe at >97% level from this single dataset. We find good agreement between the supernova and BAO distance measurements, both consistent with a Lambda-dominated CDM cosmology, as demonstrated through an analysis of the distance duality relationship between the luminosity (d_L) and angular diameter (d_A) distance measures. We then use these data to estimate w within this restricted redshift range (z<0.4). Our most stringent result comes from the combination of all our intermediate-redshift data (SDSS-II SNe, BAO, ISW and redshift-space distortions), giving w = -0.81 +0.16 -0.18(stat) +/- 0.15(sys) and Omega_M=0.22 +0.09 -0.08 assuming a flat universe. This value of w, and associated errors, only change slightly if curvature is allowed to vary, consistent with constraints from the Cosmic Microwave Background. We also consider more limited combinations of the geometrical (SN, BAO) and dynamical (ISW, redshift-space distortions) probes.

We extend our study of the optimization of large baryon acoustic oscillation (BAO) surveys to return the best constraints on the dark energy, building on Paper I of this series (Parkinson et al. 2007). The survey galaxies are assumed to be pre-selected active, star-forming galaxies observed by their line emission with a constant number density across the redshift bin. Star-forming galaxies have a redshift desert in the region 1.6 < z < 2, and so this redshift range was excluded from the analysis. We use the Seo & Eisenstein (2007) fitting formula for the accuracies of the BAO measurements, using only the information for the oscillatory part of the power spectrum as distance and expansion rate rulers. We go beyond our earlier analysis by examining the effect of including curvature on the optimal survey configuration and updating the expected `prior' constraints from Planck and SDSS. We once again find that the optimal survey strategy involves minimizing the exposure time and maximizing the survey area (within the instrumental constraints), and that all time should be spent observing in the low-redshift range (z<1.6) rather than beyond the redshift desert, z>2. We find that when assuming a flat universe the optimal survey makes measurements in the redshift range 0.1 < z <0.7, but that including curvature as a nuisance parameter requires us to push the maximum redshift to 1.35, to remove the degeneracy between curvature and evolving dark energy. The inclusion of expected other data sets (such as WiggleZ, BOSS and a stage III SN-Ia survey) removes the necessity of measurements below redshift 0.9, and pushes the maximum redshift up to 1.5. We discuss considerations in determining the best survey strategy in light of uncertainty in the true underlying cosmological model.

We use the new SNe Ia discovered by the SDSS-II Supernova Survey together with additional supernova datasets as well as observations of the cosmic microwave background and baryon acoustic oscillations to constrain cosmological models. This complements the analysis presented by Kessler et al. in that we discuss and rank a number of the most popular non-standard cosmology scenarios. When this combined data-set is analyzed using the MLCS2k2 light-curve fitter, we find that more exotic models for cosmic acceleration provide a better fit to the data than the Lambda-CDM model. For example, the flat DGP model is ranked higher by our information criteria tests than the standard model. When the dataset is instead analyzed using the SALT-II light-curve fitter, the standard cosmological constant model fares best. Our investigation also includes inhomogeneous Lemaitre-Tolman-Bondi (LTB) models. While our LTB models can be made to fit the supernova data as well as any other model, the extra parameters they require are not supported by our information criteria analysis.

We present measurements of the Hubble diagram for 103 Type Ia supernovae (SNe) with redshifts 0.04 < z < 0.42, discovered during the first season (Fall 2005) of the Sloan Digital Sky Survey-II (SDSS-II) Supernova Survey. These data fill in the redshift "desert" between low- and high-redshift SN Ia surveys. We combine the SDSS-II measurements with new distance estimates for published SN data from the ESSENCE survey, the Supernova Legacy Survey, the Hubble Space Telescope, and a compilation of nearby SN Ia measurements. Combining the SN Hubble diagram with measurements of Baryon Acoustic Oscillations from the SDSS Luminous Red Galaxy sample and with CMB temperature anisotropy measurements from WMAP, we estimate the cosmological parameters w and Omega_M, assuming a spatially flat cosmological model (FwCDM) with constant dark energy equation of state parameter, w. For the FwCDM model and the combined sample of 288 SNe Ia, we find w = -0.76 +- 0.07(stat) +- 0.11(syst), Omega_M = 0.306 +- 0.019(stat) +- 0.023(syst) using MLCS2k2 and w = -0.96 +- 0.06(stat) +- 0.12(syst), Omega_M = 0.265 +- 0.016(stat) +- 0.025(syst) using the SALT-II fitter. We trace the discrepancy between these results to a difference in the rest-frame UV model combined with a different luminosity correction from color variations; these differences mostly affect the distance estimates for the SNLS and HST supernovae. We present detailed discussions of systematic errors for both light-curve methods and find that they both show data-model discrepancies in rest-frame $U$-band. For the SALT-II approach, we also see strong evidence for redshift-dependence of the color-luminosity parameter (beta). Restricting the analysis to the 136 SNe Ia in the Nearby+SDSS-II samples, we find much better agreement between the two analysis methods but with larger uncertainties.

We present ugriz light curves for 146 spectroscopically confirmed or spectroscopically probable Type Ia supernovae from the 2005 season of the SDSS-II Supernova survey. The light curves have been constructed using a photometric technique that we call scene modelling, which is described in detail here; the major feature is that supernova brightnesses are extracted from a stack of images without spatial resampling or convolution of the image data. This procedure produces accurate photometry along with accurate estimates of the statistical uncertainty, and can be used to derive photometry taken with multiple telescopes. We discuss various tests of this technique that demonstrate its capabilities. We also describe the methodology used for the calibration of the photometry, and present calibrated magnitudes and fluxes for all of the spectroscopic SNe Ia from the 2005 season.

The Carnegie Supernova Project (CSP) is designed to measure the luminosity distance for Type Ia supernovae (SNe Ia) as a function of redshift, and to set observational constraints on the dark energy contribution to the total energy content of the Universe. The CSP differs from other projects to date in its goal of providing an I-band {rest-frame} Hubble diagram. Here we present the first results from near-infrared (NIR) observations obtained using the Magellan Baade telescope for SNe Ia with 0.1 < z < 0.7. We combine these results with those from the low-redshift CSP at z <0.1 (Folatelli et al. 2009). We present light curves and an I-band Hubble diagram for this first sample of 35 SNe Ia and we compare these data to 21 new SNe Ia at low redshift. These data support the conclusion that the expansion of the Universe is accelerating. When combined with independent results from baryon acoustic oscillations (Eisenstein et al. 2005), these data yield Omega_m = 0.27 +/- 0.0 (statistical), and Omega_DE = 0.76 +/- 0.13 (statistical) +/- 0.09 (systematic), for the matter and dark energy densities, respectively. If we parameterize the data in terms of an equation of state, w, assume a flat geometry, and combine with baryon acoustic oscillations, we find that w = -1.05 +/- 0.13 (statistical) +/- 0.09 (systematic). The largest source of systematic uncertainty on w arises from uncertainties in the photometric calibration, signaling the importance of securing more accurate photometric calibrations for future supernova cosmology programs. Finally, we conclude that either the dust affecting the luminosities of SNe Ia has a different extinction law (R_V = 1.8) than that in the Milky Way (where R_V = 3.1), or that there is an additional intrinsic color term with luminosity for SNe Ia independent of the decline rate.

This is the Users' Manual for the Fisher Matrix software Fisher4Cast and covers installation, GUI help, command line basics, code flow and data structure, as well as cosmological applications and extensions. Finally we discuss the extensive tests performed on the software.

We highlight the unexpected impact of nucleosynthesis and other early universe constraints on the detectability of tracking quintessence dynamics at late times, showing that such dynamics may well be invisible until the unveiling of the Stage-IV dark energy experiments (DUNE, JDEM, LSST, SKA). Nucleosynthesis forces |w'(0)| < 0.2 for the models we consider and strongly limits potential deviations from LCDM. Surprisingly, the standard CPL parametrisation, w(z) = w_0 + w_a z/(1+z), cannot match the nucleosynthesis bound for minimally coupled tracking scalar fields. Given that such models are arguably the best-motivated alternatives to a cosmological constant these results may significantly impact future cosmological survey design and imply that dark energy may well be dynamical even if we do not detect any dynamics in the next decade.

We present a measurement of the rate of type Ia supernovae (SNe Ia) from the first of three seasons of data from the SDSS-II Supernova Survey. For this measurement, we include 17 SNe Ia at redshift $z\le0.12$. Assuming a flat cosmology with $\Omega_m = 0.3=1-\Omega_\Lambda$, we find a volumetric SN Ia rate of $[2.93^{+0.17}_{-0.04}({\rm systematic})^{+0.90}_{-0.71}({\rm statistical})] \times 10^{-5} {\rm SNe} {\rm Mpc}^{-3} h_{70}^3 {\rm year}^{-1}$, at a volume-weighted mean redshift of 0.09. This result is consistent with previous measurements of the SN Ia rate in a similar redshift range. The systematic errors are well controlled, resulting in the most precise measurement of the SN Ia rate in this redshift range. We use a maximum likelihood method to fit SN rate models to the SDSS-II Supernova Survey data in combination with other rate measurements, thereby constraining models for the redshift-evolution of the SN Ia rate. Fitting the combined data to a simple power-law evolution of the volumetric SN Ia rate, $r_V \propto (1+z)^{\beta}$, we obtain a value of $\beta = 1.5 \pm 0.6$, i.e. the SN Ia rate is determined to be an increasing function of redshift at the $\sim 2.5 \sigma$ level. Fitting the results to a model in which the volumetric SN rate, $r_V=A\rho(t)+B\dot \rho(t)$, where $\rho(t)$ is the stellar mass density and $\dot \rho(t)$ is the star formation rate, we find $A = (2.8 \pm 1.2) \times 10^{-14} \mathrm{SNe} \mathrm{M}_{\sun}^{-1} \mathrm{year}^{-1}$, $B = (9.3^{+3.4}_{-3.1})\times 10^{-4} \mathrm{SNe} \mathrm{M}_{\sun}^{-1}$.

The recent discovery of apparent cosmic acceleration has highlighted the depth of our ignorance of the fundamental properties of nature. It is commonly assumed that the explanation for acceleration must come from a new form of energy dominating the cosmos - dark energy - or a modification of Einstein's theory of Relativity. It is often overlooked, however, that a currently viable alternative explanation of the data is radial inhomogeneity which alters the Hubble diagram without any acceleration. This explanation is often ignored for two reasons: radial inhomogeneity significantly complicates analysis and predictions, and so the full details have not been investigated; and it is a philosophically highly controversial idea, revoking as it does the long-held Copernican Principle. To date, there has not been a general way of determining the validity if the Copernican Principle -- that we live at a typical position in the universe -- significantly weakening the foundations of cosmology as a scientific endeavour. Here we present an observational test for the Copernican assumption which can be automatically implemented while we search for dark energy in the coming decade. Our test is entirely independent of any model for dark energy or theory of gravity and thereby represents a model-independent test of the Copernican Principle.

This paper presents spectroscopy of supernovae discovered in the first season of the Sloan Digital Sky Survey-II Supernova Survey. This program searches for and measures multi-band light curves of supernovae in the redshift range z = 0.05 - 0.4, complementing existing surveys at lower and higher redshifts. Our goal is to better characterize the supernova population, with a particular focus on SNe Ia, improving their utility as cosmological distance indicators and as probes of dark energy. Our supernova spectroscopy program features rapid-response observations using telescopes of a range of apertures, and provides confirmation of the supernova and host-galaxy types as well as precise redshifts. We describe here the target identification and prioritization, data reduction, redshift measurement, and classification of 129 SNe Ia, 16 spectroscopically probable SNe Ia, 7 SNe Ib/c, and 11 SNe II from the first season. We also describe our efforts to measure and remove the substantial host galaxy contamination existing in the majority of our SN spectra.

We study the degeneracies between dark energy dynamics, dark matter and curvature using a non-parametric and non-perturbative approach. This allows us to examine the knock-on bias induced in the reconstructed dark energy equation of state, w(z), when there is a bias in the cosmic curvature or dark matter content, without relying on any specific parameterisation of w. Even assuming perfect Hubble, distance and volume measurements, we show that for z > 1, the bias in w(z) is up to two orders of magnitude larger than the corresponding errors in Omega_k or Omega_m. This highlights the importance of obtaining unbiased estimators of all cosmic parameters in the hunt for dark energy dynamics.

The Sloan Digital Sky Survey-II Supernova Survey has identified a large number of new transient sources in a 300 sq. deg. region along the celestial equator during its first two seasons of a three-season campaign. Multi-band (ugriz) light curves were measured for most of the sources, which include solar system objects, Galactic variable stars, active galactic nuclei, supernovae (SNe), and other astronomical transients. The imaging survey is augmented by an extensive spectroscopic follow-up program to identify SNe, measure their redshifts, and study the physical conditions of the explosions and their environment through spectroscopic diagnostics. During the survey, light curves are rapidly evaluated to provide an initial photometric type of the SNe, and a selected sample of sources are targeted for spectroscopic observations. In the first two seasons, 476 sources were selected for spectroscopic observations, of which 403 were identified as SNe. For the Type Ia SNe, the main driver for the Survey, our photometric typing and targeting efficiency is 90%. Only 6% of the photometric SN Ia candidates were spectroscopically classified as non-SN Ia instead, and the remaining 4% resulted in low signal-to-noise, unclassified spectra. This paper describes the search algorithm and the software, and the real-time processing of the SDSS imaging data. We also present the details of the supernova candidate selection procedures and strategies for follow-up spectroscopic and imaging observations of the discovered sources.

The Sloan Digital Sky Survey-II (SDSS-II) has embarked on a multi-year project to identify and measure light curves for intermediate-redshift (0.05 < z < 0.35) Type Ia supernovae (SNe Ia) using repeated five-band (ugriz) imaging over an area of 300 sq. deg. The survey region is a stripe 2.5 degrees wide centered on the celestial equator in the Southern Galactic Cap that has been imaged numerous times in earlier years, enabling construction of a deep reference image for discovery of new objects. Supernova imaging observations are being acquired between 1 September and 30 November of 2005-7. During the first two seasons, each region was imaged on average every five nights. Spectroscopic follow-up observations to determine supernova type and redshift are carried out on a large number of telescopes. In its first two three-month seasons, the survey has discovered and measured light curves for 327 spectroscopically confirmed SNe Ia, 30 probable SNe Ia, 14 confirmed SNe Ib/c, 32 confirmed SNe II, plus a large number of photometrically identified SNe Ia, 94 of which have host-galaxy spectra taken so far. This paper provides an overview of the project and briefly describes the observations completed during the first two seasons of operation.

We show that the assumption of a flat universe induces critically large errors in reconstructing the dark energy equation of state at z>~0.9 even if the true cosmic curvature is very small, O(1%) or less. The spuriously reconstructed w(z) shows a range of unusual behaviour, including crossing of the phantom divide and mimicking of standard tracking quintessence models. For 1% curvature and LCDM, the error in w grows rapidly above z~0.9 reaching (50%,100%) by redshifts of (2.5,2.9) respectively, due to the long cosmological lever arm. Interestingly, the w(z) reconstructed from distance data and Hubble rate measurements have opposite trends due to the asymmetric influence of the curved geodesics. These results show that including curvature as a free parameter is imperative in any future analyses attempting to pin down the dynamics of dark energy, especially at moderate or high redshifts.

Observed data is often contaminated by undiscovered interlopers, leading to biased parameter estimation. Here we present BEAMS (Bayesian Estimation Applied to Multiple Species) which significantly improves on the standard maximum likelihood approach in the case where the probability for each data point being `pure' is known. We discuss the application of BEAMS to future Type Ia supernovae (SNIa) surveys, such as LSST, which are projected to deliver over a million supernovae lightcurves without spectra. The multi-band lightcurves for each candidate will provide a probability of being Ia (pure) but the full sample will be significantly contaminated with other types of supernovae and transients. Given a sample of N supernovae with mean probability, P, of being Ia, BEAMS delivers parameter constraints equal to NP spectroscopically-confirmed SNIa. In addition BEAMS can be simultaneously used to tease apart different families of data and to recover properties of the underlying distributions of those families (e.g. the Type Ibc and II distributions). Hence BEAMS provides a unified classification and parameter estimation methodology which may be useful in a diverse range of problems such as photometric redshift estimation or, indeed, any parameter estimation problem where contamination is an issue.

We present extensive ugrizYHJK photometry and optical spectroscopy of SN 2005gj obtained by the SDSS-II and CSP Supernova Projects, which give excellent coverage during the first 150 days after the time of explosion. These data show that SN 2005gj is the second clear case, after SN 2002ic, of a thermonuclear explosion in a dense circumstellar environment. Both the presence of singly and doubly ionized iron-peak elements (FeIII and weak SII, SiII) near maximum light as well as the spectral evolution show that SN 2002ic-like events are Type Ia explosions. Independent evidence comes from the exponential decay in luminosity of SN 2005gj, pointing to an exponential density distribution of the ejecta. The interaction of the supernova ejecta with the dense circumstellar medium is stronger than in SN 2002ic: (1) the supernova lines are weaker; (2) the Balmer emission lines are more luminous; and (3) the bolometric luminosity is higher close to maximum light. The velocity evolution of the Halpha components suggest that the CSM around SN 2005gj is clumpy and it has a flatter density distribution compared with the steady wind solution, in agreement with SN 2002ic. An early X-ray observation with Chandra gives an upper-limit on the mass loss rate from the companion of < 2x10^{-4} Msun/yr.

We optimize the design of future spectroscopic redshift surveys for constraining the dark energy via precision measurements of the baryon acoustic oscillations (BAO), with particular emphasis on the design of the Wide-Field Multi-Object Spectrograph (WFMOS). We develop a model that predicts the number density of possible target galaxies as a function of exposure time and redshift. We use this number counts model together with fitting formulae for the accuracy of the BAO measurements to determine the effectiveness of different surveys and instrument designs. We search through the available survey parameter space to find the optimal survey with respect to the dark energy equation-of-state parameters according to the Dark Energy Task Force Figure-of-Merit, including predictions of future measurements from the Planck satellite. We optimize the survey to test the LambdaCDM model, assuming that galaxies are pre-selected using photometric redshifts to have a constant number density with redshift, and using a non-linear cut-off for the matter power spectrum that evolves with redshift. We find that line-emission galaxies are strongly preferred as targets over continuum emission galaxies. The optimal survey covers a redshift range 0.8 < z < 1.4, over the widest possible area (6000 sq. degs from 1500 hours observing time). The most efficient number of fibres for the spectrograph is 2,000, and the survey performance continues to improve with the addition of extra fibres until a plateau is reached at 10,000 fibres. The optimal point in the survey parameter space is not highly peaked and is not significantly affected by including constraints from upcoming supernovae surveys and other BAO experiments.

We present extensive u'g'r'i'BVRIYJHKs photometry and optical spectroscopy of SN 2005hk. These data reveal that SN 2005hk was nearly identical in its observed properties to SN 2002cx, which has been called ``the most peculiar known type Ia supernova.'' Both supernovae exhibited high ionization SN 1991T-like pre-maximum spectra, yet low peak luminosities like SN 1991bg. The spectra reveal that SN 2005hk, like SN 2002cx, exhibited expansion velocities that were roughly half those of typical type Ia supernovae. The R and I light curves of both supernovae were also peculiar in not displaying the secondary maximum observed for normal type Ia supernovae. Our YJH photometry of SN 2005hk reveals the same peculiarity in the near-infrared. By combining our optical and near-infrared photometry of SN 2005hk with published ultraviolet light curves obtained with the Swift satellite, we are able to construct a bolometric light curve from ~10 days before to ~60 days after B maximum. The shape and unusually low peak luminosity of this light curve, plus the low expansion velocities and absence of a secondary maximum at red and near-infrared wavelengths, are all in reasonable agreement with model calculations of a 3D deflagration which produces ~0.25 M_sun of 56Ni.

Optimally selecting a subset of targets from a larger catalog is a common problem in astronomy and cosmology. A specific example is the selection of targets from an imaging survey for multi-object spectrographic follow-up. We present a new heuristic algorithm, HYBRID, for this purpose and undertake detailed studies of its performance. HYBRID combines elements of the simulated annealing, MCMC and particle-swarm methods and is particularly successful in cases where the survey landscape has multiple curvature or clustering scales. HYBRID consistently outperforms the other methods, especially in high-dimensionality spaces with many extrema. This means many fewer simulations must be run to reach a given performance confidence level and implies very significant advantages in solving complex or computationally expensive optimisation problems.

Standard two-parameter compressions of the infinite dimensional dark energy model space show crippling limitations even with current SN-Ia data. Firstly they cannot cope with rapid evolution - the best-fit to the latest SN-Ia data shows late and very rapid evolution to w_0 = -2.85. However all of the standard parametrisations (incorrectly) claim that this best-fit is ruled out at more than 2-sigma, primarily because they track it well only at very low redshifts, z < 0.2. Further they incorrectly rule out the observationally acceptable region w << -1 for z > 1. Secondly the parametrisations give wildly different estimates for the redshift of acceleration, which vary from z_{acc}=0.14 to z_{acc}=0.59. Although these failings are largely cured by including higher-order terms (3 or 4 parameters) this results in new degeneracies which open up large regions of previously ruled-out parameter space. Finally we test the parametrisations against a suite of theoretical quintessence models. The widely used linear expansion in z is generally the worst, with errors of up to 10% at z=1 and 20% at z > 2. All of this casts serious doubt on the usefulness of the standard two-parameter compressions in the coming era of high-precision dark energy cosmology and emphasises the need for decorrelated compressions with at least three parameters.

We measure the large-scale real-space power spectrum P(k) using luminous red galaxies (LRGs) in the Sloan Digital Sky Survey (SDSS) and use this measurement to sharpen constraints on cosmological parameters from the Wilkinson Microwave Anisotropy Probe (WMAP). We employ a matrix-based power spectrum estimation method using Pseudo-Karhunen-Loeve eigenmodes, producing uncorrelated minimum-variance measurements in 20 k-bands of both the clustering power and its anisotropy due to redshift-space distortions, with narrow and well-behaved window functions in the range 0.01h/Mpc < k < 0.2h/Mpc. Results from the LRG and main galaxy samples are consistent, with the former providing higher signal-to-noise. Our results are robust to omitting angular and radial density fluctuations and are consistent between different parts of the sky. They provide a striking confirmation of the predicted large-scale LCDM power spectrum. Combining only SDSS LRG and WMAP data places robust constraints on many cosmological parameters that complement prior analyses of multiple data sets. The LRGs provide independent cross-checks on Om and the baryon fraction in good agreement with WMAP. Within the context of flat LCDM models, our LRG measurements complement WMAP by sharpening the constraints on the matter density, the neutrino density and the tensor amplitude by about a factor of two, giving Omega_m=0.24+-0.02 (1 sigma), sum m_nu < 0.9 eV (95%) and r<0.3 (95%). Baryon oscillations are clearly detected and provide a robust measurement of the comoving distance to the median survey redshift z=0.35 independent of curvature and dark energy properties. Within the LCDM framework, our power spectrum measurement improves the evidence for spatial flatness, sharpening the curvature constraint Omega_tot=1.05+-0.05 from WMAP alone to Omega_tot=1.003+-0.010. Assuming Omega_tot=1, the equation of state parameter is constrained to w=-0.94+-0.09, indicating the potential for more ambitious future LRG measurements to provide precision tests of the nature of dark energy. All these constraints are essentially independent of scales k>0.1h/Mpc and associated nonlinear complications, yet agree well with more aggressive published analyses where nonlinear modeling is crucial.

We discuss how the baryon acoustic oscillation (BAO) signatures in the galaxy power spectrum can distinguish between modified gravity and the cosmological constant as the source of cosmic acceleration. To this end we consider a model characterized by a parameter $n$, which corresponds to the Dvali-Gabadadze-Porrati (DGP) model if $n=2$ and reduces to the standard spatially flat cosmological constant concordance model for $n$ equal to infinity. We find that the different expansion histories of the modified gravity models systematically shifts the peak positions of BAO. A preliminary analysis using the current SDSS LRG sample indicates that the original DGP model is disfavored unless the matter density parameter exceeds 0.3. The constraints will be strongly tightened with future spectroscopic samples of galaxies at high redshifts. We demonstrate that WFMOS, in collaboration with other surveys such as Planck, will powerfully constrain modified gravity alternatives to dark energy as the explanation of cosmic acceleration.

We review the theory of inflation with single and multiple fields paying particular attention to the dynamics of adiabatic and entropy/isocurvature perturbations which provide the primary means of testing inflationary models. We review the theory and phenomenology of reheating and preheating after inflation providing a unified discussion of both the gravitational and nongravitational features of multi-field inflation. In addition we cover inflation in theories with extra dimensions and models such as the curvaton scenario and modulated reheating which provide alternative ways of generating large-scale density perturbations. Finally we discuss the interesting observational implications that can result from adiabatic-isocurvature correlations and non-Gaussianity.

We present fully non-linear simulations of a self-interacting scalar field in the early universe undergoing tachyonic preheating. We find that density perturbations on sub-horizon scales which are amplified by tachyonic instability maintain long range correlations even during the succeeding parametric resonance, in contrast to the standard models of preheating dominated by parametric resonance. As a result the final spectrum exhibits memory and is not universal in shape. We find that throughout the subsequent era of parametric resonance the equation of state of the universe is almost dust-like, hence the Jeans wavelength is much smaller than the horizon scale. If our 2D simulations are accurate reflections of the situation in 3D, then there are wide regions of parameter space ruled out by over-production of black holes. It is likely however that realistic parameter values, consistent with COBE/WMAP normalisation, are safetly outside this black hole over-production region.

We report a measurement of the quadrupole power spectrum in the two degree field (2dF) QSO redshift (2QZ) survey. The analysis uses an algorithm parallel to that for the estimation of the standard monopole power spectrum without first requiring computation of the correlation function or the anisotropic power spectrum. The error on the quadrupole spectrum is rather large but the best fit value of the bias parameter from the quadrupole spectrum is consistent with that from previous investigations of the 2dF data.

Vast sound waves traveling through the relativistic plasma during the first million years of the universe imprint a preferred scale in the density of matter. We now have the ability to detect this characteristic fingerprint in the clustering of galaxies at various redshifts and use it to measure the acceleration of the expansion of the Universe. The Wide-Field Multi-Object Spectrograph (WFMOS) would use this test to shed significant light on the true nature of dark energy, the mysterious source of this cosmic acceleration. WFMOS would also revolutionise studies of the kinematics of the Milky Way and provide deep insights into the clustering of galaxies at redshifts up to z~4. In this article we discuss the recent progress in large galaxy redshift surveys and detail how WFMOS will help unravel the mystery of dark energy.

The next generation of galaxy surveys will attempt to measure the baryon oscillations in the clustering power spectrum with high accuracy. These oscillations encode a preferred scale which may be used as a standard ruler to constrain cosmological parameters and dark energy models. In this paper we present simple analytical fitting formulae for the accuracy with which the preferred scale may be determined in the tangential and radial directions by future spectroscopic and photometric galaxy redshift surveys. We express these accuracies as a function of survey parameters such as the central redshift, volume, galaxy number density and (where applicable) photometric redshift error. These fitting formulae should greatly increase the efficiency of optimizing future surveys, which requires analysis of a potentially vast number of survey configurations and cosmological models. The formulae are calibrated using a grid of Monte Carlo simulations, which are analyzed by dividing out the overall shape of the power spectrum before fitting a simple decaying sinusoid to the oscillations. The fitting formulae reproduce the simulation results with a fractional scatter of 7% (10%) in the tangential (radial) directions over a wide range of input parameters. We also indicate how sparse-sampling strategies may enhance the effective survey area if the sampling scale is much smaller than the projected baryon oscillation scale.

We highlight the flexibility of the IPSO experiment-design framework by contrasting its application to CMB, weak lensing and redshift surveys. We illustrate the latter with a 10 parameter MCMC D-optimisation of a dark energy redshift survey. When averaged over a standard dark energy model space the resulting optimal survey typically has only one or two redshift bins, located at z<2. By exploiting optimisation we show how the statistical power of such surveys is significantly enhanced. Experiment design is aided by the richness of the figure of merit landscape which means one can impose secondary optimisation criteria at little cost. For example, one may choose either to maximally test a single model (such as \Lambda CDM) or to get the most general model-independent constraints possible (e.g. on a whole space of dark energy models). Such freedom points to a future where cosmological experiments become increasingly specialised and optimisation increasingly important.

Optimising the major next-generation cosmological surveys (such as {\em SNAP, KAOS etc...}) is a key problem given our ignorance of the physics underlying cosmic acceleration and the plethora of surveys planned. We propose a Bayesian design framework which (1) maximises the discrimination power of a survey without assuming any underlying dark energy model, (2) finds the best niche survey geometry given current data and future competing experiments, (3) maximises the cross-section for serendipitous discoveries and (4) can be adapted to answer specific questions (such as `is dark energy dynamical?'). Integrated Parameter Space Optimisation (IPSO) is a design framework that integrates projected parameter errors over an entire dark energy parameter space and then extremises a figure of merit (such as Shannon entropy gain which we show is stable to off-diagonal covariance matrix perturbations) as a function of survey parameters using analytical, grid or MCMC techniques. We discuss examples where the optimisation can be performed analytically. IPSO is thus a general, model-independent and scalable framework that allows us to appropriately use prior information to design the best possible surveys.

We provide a simple but robust argument that primordial black hole (PBH) production generically does {\em not} exceed astrophysical bounds during the resonant preheating phase after inflation. This conclusion is supported by fully nonlinear lattice simulations of various models in two and three dimensions which include rescattering but neglect metric perturbations. We examine the degree to which preheating amplifies density perturbations at the Hubble scale and show that at the end of the parametric resonance, power spectra are universal, with no memory of the power spectrum at the end of inflation. In addition we show how the probability distribution of density perturbations changes from exponential on very small scales to Gaussian when smoothed over the Hubble scale -- the crucial length for studies of primordial black hole formation -- hence justifying the standard assumption of Gaussianity.

Detecting dark energy dynamics is the main quest of current dark energy research. Addressing the issue demands a fully consistent analysis of CMB, large scale structure and SN-Ia data with multi-parameter freedom valid for all redshifts. Here we undertake a ten parameter analysis of general dark energy confronted with the first year WMAP, 2dF galaxy survey and latest SN-Ia data. Despite the huge freedom in dark energy dynamics there are no new degeneracies with standard cosmic parameters apart from a mild degeneracy between reionisation and the redshift of acceleration, both of which effectively suppress small scale power. Breaking this degeneracy will help significantly in detecting dynamics, if it exists. Our best-fit model to the data has significant late-time evolution at z<1.5. Phantom models are also considered and we find that the best-fit crosses w=-1 which, if confirmed, would be a clear signal for radically new physics. Treatment of such rapidly varying models requires careful integration of the dark energy density usually not implemented in standard codes, leading to crucial errors of up to 5%. Nevertheless cosmic variance means that standard LCDM models are still a very good fit to the data and evidence for dynamics is currently very weak. Independent tests of reionisation or the epoch of acceleration (e.g. ISW-LSS correlations) or reduction of cosmic variance at large scales (e.g. cluster polarisation at high redshift) may prove key in the hunt for dynamics.

We show that the redshift-space quadrupole will be a powerful tool for constraining dark energy even if the baryon oscillations are missing from the usual monopole power spectrum and even if bias is scale- and time-dependent. We calculate the accuracy with which a next-generation galaxy survey based on KAOS will measure the quadrupole power spectrum, which gives the leading anisotropies in the power spectrum in redshift space due to the linear velocity, Finger of God and Alcock-Paczynski effects. Combining the monopole and quadrupole power spectra breaks the degeneracies between the multiple bias parameters and dark energy both in the linear and nonlinear regimes and, in the complete absence of baryon oscillations ($\Omega_b=0$), leads to a roughly 500% improvement in constraints on dark energy compared with those from the monopole spectrum alone. As a result the worst case - with no baryon oscillations - has dark energy errors only mildly degraded relative to the ideal case, providing insurance on the robustness of next-generation galaxy survey constraints on dark energy.

With the WMAP data we can now begin to test realistic models of inflation involving multiple scalar fields. These naturally lead to correlated adiabatic and isocurvature (entropy) perturbations with a running spectral index. We present the first full (9 parameter) likelihood analysis of double inflation with WMAP data and find that despite the extra freedom, supersymmetric hybrid potentials are strongly constrained with less than 7% correlated isocurvature component allowed when standard priors are imposed on the cosomological parameters. As a result we also find that Akaike & Bayesian model selection criteria rather strongly prefer single-field inflation, just as equivalent analysis prefers a cosmological constant over dynamical dark energy in the late universe. It appears that simplicity is the best guide to our universe.

Dark energy affects the CMB through its perturbations and affects both CMB and Sn-Ia through its background evolution. Using recent CMB and Sn-Ia data sets, together with the most general parameterization of the dark energy equation of state available, we find that today w < -0.8 (2 sigma). We also find that the value of the normalization of the power spectrum on cluster scales, sigma8, can be used to discriminate between dynamical models of dark energy (Quintessence models) and a cosmological constant model (Lambda CDM).

There are two basic ways to measure physical distances in cosmology: One based on standard candles and one based on standard rulers. Comparing current data for each method allows us to rule out axion-photon mixing and dust-extinction as the sources of supernova dimming and generally protects the case for cosmic acceleration from attacks based on loss of photons. The combined data constrains the energy densities in a LCDM model to 0.19 < Omega_m < 0.32 and 0.47 < Omega_Lambda < 0.82 (at 2\sigma) without recourse to any further data sets. Future data will improve on these limits and allow us to place constraints on more exotic physics.

In cosmology, distances based on standard candles (e.g. supernovae) and standard rulers (e.g. baryon oscillations) agree as long as three conditions are met: (1) photon number is conserved, (2) gravity is described by a metric theory with (3) photons travelling on unique null geodesics. This is the content of distance-duality (the reciprocity relation) which can be violated by exotic physics. Here we analyse the implications of the latest cosmological data sets for distance-duality. While broadly in agreement and confirming acceleration we find a 2-sigma violation caused by excess brightening of SN-Ia at z > 0.5, perhaps due to lensing magnification bias. This brightening has been interpreted as evidence for a late-time transition in the dark energy but because it is not seen in the d_A data we argue against such an interpretation. Our results do, however, rule out significant SN-Ia evolution and extinction: the "replenishing" grey-dust model with no cosmic acceleration is excluded at more than 4-sigma despite this being the best-fit to SN-Ia data alone, thereby illustrating the power of distance-duality even with current data sets.

We investigate the constraints on quintessence arising from both renormalisable and non-renormalisable couplings where the 5d Planck mass is around the TeV scale. The quintessence field vacuum expectation value is typically of order the 4d Planck mass while non-renormalisable operators are expected to be suppressed by the 5d Planck mass. Non-renormalisable operators are therefore important in computing the 4d effective quintessence potential. We then study the quantum corrections to the quintessence potential due to fermion and graviton loops. The tower of Kaluza-Klein modes competes with the TeV-scale cut-off, altering the graviton contribution to the vacuum polarization of quintessence. Nevertheless we show that, as in four dimensions, the classical potential is stable to such radiative corrections.

Axion-photon mixing has been proposed as an alternative to acceleration as the explanation for supernovae dimming. We point out that the loss of photons due to this mixing will induce a strong asymmetry between the luminosity, d_L(z), and angular-diameter distance, d_A(z), since the latter is unaffected by mixing. In a first search for such an asymmetry we introduce a dimensionless mixing amplitude lambda so that lambda=0 if no photons are lost and lambda = 1 if axion-photon mixing occurs. The best-fit to SNIa and radio galaxy data is lambda = -0.3^{+0.6}_{-0.4} (95% CL), corresponding to an unphysical, negative, mixing length. This same argument limits the attenuation of light from supernovae due to dust. We show that future d_L and d_A data from SNAP and galaxy surveys such as DEEP2 and KAOS will detect or rule out mixing at more than 5-sigma, almost independently of the dark energy dynamics. Finally we discuss the constraints from the near maximal polarisation of the gamma-ray burst (GRB) GRB021206. Since mixing reduces the polarisation of distant sources, future observations of high redshift GRBs may provide orthogonal constraints on axion-photon mixing and related scenarios.

Cosmological dark energy is a natural source of variation of the fine structure constant. Using a model-independent approach we show that once general assumptions about the alpha-varying interactions are made, astronomical probes of its variation constrain the dark energy equation of state today to satisfy -1 < w_f < -0.96 at 3-sigma and significantly disfavour late-time changes in the equation of state. We show how dark-energy-induced spatial perturbations of alpha are linked to violations of the Equivalence Principle and are thus negligible at low-redshift, in stark contrast to the BSBM theories. This provides a new test of dark energy as the source of alpha variation.

Imagine a scenario in which the dark energy forms via the condensation of dark matter at some low redshift. The Compton wavelength therefore changes from small to very large at the transition, unlike quintessence or metamorphosis. We study CMB, large scale structure, supernova and radio galaxy constraints on condensation by performing a 4 parameter likelihood analysis over the Hubble constant and the three parameters associated with Q, the condensate field: Omega_Q, w_f and z_t (energy density and equation of state today, and redshift of transition). Condensation roughly interpolates between Lambda CDM (for large z_t) and sCDM (low z_t) and provides a slightly better fit to the data than Lambda CDM. We confirm that there is no degeneracy in the CMB between H and z_t and discuss the implications of late-time transitions for the Lyman-alpha forest. Finally we discuss the nonlinear phase of both condensation and metamorphosis, which is much more interesting than in standard quintessence models.

The dynamics of preheating after inflation has not been clearly understood yet.In particular, the issue of the generation of metric perturbations during preheating on super-horizon scale is still unsettled. Large scale perturbations may leave an imprint on the cosmic microwave background, or may become seeds for generation of primordial black holes. Hence, in order to make a connection between the particle physics models and cosmological observations, understanding the evolution of super-Hubble scale perturbations during preheating is important. Here, we propose an alternative treatment to handle this issue based on the so-called separate universe approach, which suggests less efficient amplification of super-Hubble modes during preheating than was expected before. We also point out an important issue which may have been overlooked in previous treatments.

We study the integrated Sachs-Wolfe (ISW) effect using a model-independent parameterization of the dark energy equation of state. Cosmic variance severely restricts the class of models distinguishable from Lambda-CDM. In particular if the present value of the equation of state satisfies w_Q^o <-0.8, the only distinguishable models are those with a rapidly varying equation of state, characterized by a late transition occurring at redshifts z<3. Due to the degeneracy with other cosmological parameters, models with a slowly varying behavior cannot be differentiated from each other or from a cosmological constant using even cosmic variance-limited CMB measurements. This may place a fundamental limit on our understanding of the origin of the presently observed acceleration.

We study string-gas cosmology in dilaton gravity, inspired by the fact that it naturally arises in a string theory context. Our main interest is the thermodynamical treatment of the string-gas and the resulting implications for the cosmology. Within an adiabatic approximation, thermodynamical equilibrium and a small, toroidal universe as initial conditions, we numerically solve the corresponding equations of motions in two different regimes describing the string-gas thermodynamics: (i) the Hagedorn regime, with a single scale factor, and (ii) an almost-radiation dominated regime, which includes the leading corrections due to the lightest Kaluza Klein and winding modes, with two scale factors. The scale factor in the Hagedorn regime exhibits very slow time evolution with nearly constant energy and negligible pressure. By contrast, in case (ii) we find interesting cosmological solutions where the large dimensions continue to expand and the small ones are kept undetectably small.

We show explicitly some exciting features of double-inflation: (i) it can often lead to strongly correlated adiabatic and entropy (isocurvature) power spectra. (ii) The two-field slow-roll consistency relations can be violated when the correlation is large at Hubble crossing. (iii) The spectra of adiabatic and entropy perturbations can be strongly scale-dependent and tilted toward either the red or blue. These effects are typically due to a light or time-dependent entropy mass and a non-negligible angular velocity in field space during inflation. They are illustrated via a multi-parameter numerical search for correlations in two concrete models. The correlation is found to be particularly strong in a supersymmetric scenario due to rapid growth of entropy perturbations in the tachyonic region separating the two inflationary stages. Our analysis suggests that realistic double-inflation models will provide a rich and fruitful arena for the application of future cosmic data sets and new approximation schemes which go beyond slow-roll.

Until now all well-studied dark energy models proposed have very large Compton wavelengths, $\lambda_c > 100 Mpc$ at all time and hence show no clustering on small scales. Evidence against this would disfavour both $\Lambda$CDM and quintessence. We show however, that large $\lambda_c$ is actually favoured by current CMB, LSS and SN1a data by comparing quintessence with a condensate scenario with the same background dynamics. The condensate scenario has a very small $\lambda_c$ before making a transition to negative pressure and large $\lambda_c$ and therefore exhibits a rich CMB phenomenology which roughly interpolates between $\Lambda$CDM and flat CDM models.

We study a model of the dark energy which exhibits a rapid change in its equation of state w(z), such as occurs in vacuum metamorphosis. We compare the model predictions with CMB, large scale structure and supernova data and show that a late-time transition is marginally preferred over standard Lambda-CDM.

We discuss the principles governing the selection of inflationary models for which preheating can affect the CMB. This is a (fairly small) subset of those models which have non-negligible entropy/isocurvature perturbations on large scales during inflation. We study new models which belong to this class - two-field inflation with negative nonminimal coupling and hybrid/double/supernatural inflation models where the tachyonic growth of entropy perturbations can lead to the variation of the curvature perturbation, ${\cal R}$, on super-Hubble scales. Finally we present evidence against recent claims for the variation of ${\cal R}$ in the absence of substantial super-Hubble entropy perturbations.

We study constraints from the latest CMB, large scale structure (2dF, Abell/ACO, PSCz) and SN1a data on dark energy models with a sharp transition in their equation of state, w(z). Such a transition is motivated by models like vacuum metamorphosis where non-perturbative quantum effects are important at late times. We allow the transition to occur at a specific redshift, z_t, to a final negative pressure -1 < w_f < -1/3. We find that the CMB and supernovae data, in particular, prefer a late-time transition due to the associated delay in cosmic acceleration. The best fits (with 1 sigma errors) to all the data are z_t = 2.0^{+2.2}_{-0.76}, \Omega_Q = 0.73^{+0.02}_{-0.04} and w_f = -1^{+0.2}.

We study gravitational creation of light fermions in the presence of classical scalar metric perturbations about a flat Friedmann-Lemaitre- Robertson-Walker (FLRW) background. These perturbations can be large during preheating, breaking the conformal flatness of the background spacetime. We compute numerically the total number of particles generated by the modes of the metric perturbations which have grown sufficiently to become classical. In the absence of inhomogeneities massless fermions are not gravitationally produced, and then this effect may be relevant for abundance estimates of light gravitational relics.

We study the amplification of large-scale magnetic fields during preheating and inflation in several different models. Preheating can resonantly amplify seed fields on cosmological scales. In the presence of conductivity, however, the effect of resonance is typically weakened while the amplitude of produced magnetic fields depends sensitively on the evolution of the conductivity during the preheating and thermalisation phases. In addition we discuss geometric magnetisation, where amplification of magnetic fields occurs through coupling to curvature invariants. This can be efficient during inflation due to a negative coupling instability. Finally we discuss the breaking of the conformal flatness of the background metric whereby magnetic fields can be stimulated through the growth of scalar metric perturbations during metric preheating.

Preheating after inflation may over-produce primordial black holes (PBH's) in many regions of parameter space. As an example we study two-field models with a massless self-interacting inflaton, taking into account second order field and metric backreaction effects as spatial averages. We find that a complex quilt of parameter regions above the Gaussian PBH over-production threshold emerges due to the enhancement of curvature perturbations on all scales. It should be possible to constrain realistic models of inflation through PBH over-production although many issues, such as rescattering and non-Gaussianity, remain unsolved or unexplored.

We study adiabatic (curvature) and entropy (isocurvature) perturbations produced during a period of cosmological inflation that is driven by multiple scalar fields with an arbitrary interaction potential. A local rotation in field space is performed to separate out the adiabatic and entropy modes. The resulting field equations show explicitly how on large scales entropy perturbations can source adiabatic perturbations if the background solution follows a curved trajectory in field space, and how adiabatic perturbations cannot source entropy perturbations in the long-wavelength limit. It is the effective mass of the entropy field that determines the amplitude of entropy perturbations during inflation. We present two applications of the equations. First, we show why one in general expects the adiabatic and entropy perturbations to be correlated at the end of inflation, and calculate the cross-correlation in the context of a double inflation model with two non-interacting fields. Second, we consider two-field preheating after inflation, examining conditions under which entropy perturbations can alter the large-scale curvature perturbation and showing how our new formalism has advantages in numerical stability when the background solution follows a non-trivial trajectory in field space.

This paper is dedicated to the memory of Dennis Sciama. Variable-Speed-of-Light (VSL) cosmologies are currently attracting interest as an alternative to inflation. We investigate the fundamental geometrodynamic aspects of VSL cosmologies and provide several implementations which do not explicitly break Lorentz invariance (no "hard" breaking). These "soft" implementations of Lorentz symmetry breaking provide particularly clean answers to the question "VSL with respect to what?". The class of VSL cosmologies we consider are compatible with both classical Einstein gravity and low-energy particle physics. These models solve the "kinematic" puzzles of cosmology as well as inflation does, but cannot by themselves solve the flatness problem, since in their purest form no violation of the strong energy condition occurs. We also consider a heterotic model (VSL plus inflation) which provides a number of observational implications for the low-redshift universe if chi contributes to the "dark energy" either as CDM or quintessence. These implications include modified gravitational lensing, birefringence, variation of fundamental constants and rotation of the plane of polarization of light from distant sources.

Fermion creation during preheating in the presence of multiple scalar fields exhibits a range of interesting behaviour relevant to estimating post-inflation gravitino abundances. We present non-perturbative analysis of this phenomenon over a 6-dimensional parameter space in an expanding background paying particular attention to the interplay between instant and direct fermion preheating. In the broad resonance regime we find that instant fermion production is sensitive to suppression of the long wavelength scalar field modes during inflation. Further, the standard scenario of resonant fermionic preheating through inflaton decay can be significantly modified by instant preheating, and may even lead to a decrease in the number of fermions produced. We explicitly include the effects of metric perturbations and demonstrate that they are important at small coupling but not at strong coupling, due to the rapid saturation of the Pauli bound.

We propose a new technique for detecting gravitational waves using Quantum Entangled STate (QUEST) technology. Gravitational waves reduce the non-locality of correlated quanta controlled by Bell's inequalities, distorting quantum encryption key statistics away from a pure white noise. Gravitational waves therefore act as shadow eavesdroppers. The resulting colour distortions can, at least in principle, be separated from noise and can differentiate both deterministic and stochastic sources.

We consider slow-roll inflation in the context of recently proposed four-dimensional effective gravity induced on the world-volume of a three-brane in five-dimensional Einstein gravity. We find significant modifications of the simplest chaotic inflationary scenario when the five-dimensional Planck scale is below about 10^{17} GeV. We use the comoving curvature perturbation, which remains constant on super-Hubble scales, in order to calculate the spectrum of adiabatic density perturbations generated. Modifications to the Friedmann constraint equation lead to a faster Hubble expansion at high energies and a more strongly damped evolution of the scalar field. This assists slow-roll, enhances the amount of inflation obtained in any given model, and drives the perturbations towards an exactly scale-invariant Harrison-Zel'dovich spectrum. In chaotic inflation driven by a massive scalar field we show that inflation can occur at field values far below the four-dimensional Planck scale, though above the five-dimensional fundamental scale.

Metric perturbations typically strengthen field resonances during preheating. In contrast we present a model in which the super-Hubble field resonances are completely {\em suppressed} when metric perturbations are included. The model is the nonminimal Fakir-Unruh scenario which is exactly solvable in the long-wavelength limit when metric perturbations are included, but exhibits exponential growth of super-Hubble modes in their absence. This gravitationally enhanced integrability is exceptional, both for its rarity and for the power with which it illustrates the importance of including metric perturbations in consistent studies of preheating. We conjecture a no-go result - there exists no {\em single-field} model with growth of cosmologically-relevant metric perturbations during preheating.

Can super-Hubble metric perturbations be amplified exponentially during preheating ? Yes. An analytical existence proof is provided by exploiting the conformal properties of massless inflationary models. The traditional conserved quantity \zeta is non-conserved in many regions of parameter space. We include backreaction through the homogeneous parts of the inflaton and preheating fields and discuss the role of initial conditions on the post-preheating power-spectrum. Maximum field variances are strongly underestimated if metric perturbations are ignored. We illustrate this in the case of strong self-interaction of the decay products. Without metric perturbations, preheating in this case is very inefficient. However, metric perturbations increase the maximum field variances and give alternative channels for the resonance to proceed. This implies that metric perturbations can have a large impact on calculations of relic abundances of particles produced during preheating.

The relative growth of field and metric perturbations during preheating is sensitive to initial conditions set in the preceding inflationary phase. Recent work suggests this may protect super-Hubble metric perturbations from resonant amplification during preheating. We show that this possibility is fragile and sensitive to the specific form of the interactions between the inflaton and other fields. The suppression is naturally absent in two classes of preheating in which either (1) the vacua of the non-inflaton fields during inflation are deformed away from the origin, or (2) the effective masses of non-inflaton fields during inflation are small but during preheating are large. Unlike the simple toy model of a $g^2 \phi^2 \chi^2$ coupling, most realistic particle physics models contain these other features. Moreover, they generically lead to both adiabatic and isocurvature modes and non-Gaussian scars on super-Hubble scales. Large-scale coherent magnetic fields may also appear naturally.

Variable-Speed-of-Light (VSL) cosmologies are currently attracting much interest as a possible alternative to cosmological inflation. We discuss the fundamental geometrodynamic aspects of VSL cosmologies, and provide several alternative implementations. These implementations provide a large class of VSL cosmologies that pass the zeroth-order consistency tests of being compatible with both classical Einstein gravity and low-energy particle physics. While they solve the ``kinematic'' puzzles as well as inflation does, VSL cosmologies typically do not solve the flatness problem since in their purest form no violation of the strong energy condition occurs. Nevertheless, these models are easy to unify with inflation.

Recently it has become clear that the resonant amplification of quantum field fluctuations at preheating must be accompanied by resonant amplification of scalar metric perturbations, since the two are united by Einstein's equations. Furthermore, this "metric preheating" enhances particle production and leads to gravitational rescattering effects even at linear order. In multi-field models with strong preheating (q \gg 1), metric perturbations are driven nonlinear, with the strongest amplification typically on super-Hubble scales (k \to 0). This amplification is causal, being due to the super- Hubble coherence of the inflaton condensate, and is accompanied by resonant growth of entropy perturbations. The amplification invalidates the use of the linearized Einstein field equations, irrespective of the amount of fine-tuning of the initial conditions. This has serious implications at all scales - from the large-angle cosmic microwave background (CMB) anisotropies to primordial black holes. We investigate the (q,k) parameter space in a two-field model, and introduce the time to nonlinearity, t_{nl}, as the timescale for the breakdown of the linearized Einstein equations. Backreaction effects are expected to shut down the linear resonances, but cannot remove the existing amplification, which threatens the viability of strong preheating when confronted with the CMB. We discuss ways to escape the above conclusions, including secondary phases of inflation and preheating solely to fermions. Finally we rank known classes of inflation from strongest (chaotic and strongly coupled hybrid inflation) to weakest (hidden sector, warm inflation) in terms of the distortion of the primordial spectrum due to these resonances in preheating.

General relativistic effects in the form of metric perturbations are usually neglected in the preheating era that follows inflation. We argue that in realistic multi-field models these effects are in fact crucial, and the fully coupled system of metric and quantum field fluctuations needs to be considered. Metric perturbations are resonantly amplified, breaking the scale-invariance of the primordial spectrum, and in turn stimulate scalar field resonances via gravitational rescattering. This non-gravitationally dominated nonlinear growth of gravitational fluctuations may have significant effects on the Doppler peaks in the cosmic background radiation, primordial black hole formation, gravitational waves and nonthermal symmetry restoration.

Grand unified theories may display multiply interacting fields with strong coupling dynamics. This poses two new problems: (1) What is the nature of chaotic reheating after inflation, and (2) How is reheating sensitive to the mass spectrum of these theories ? We answer these questions in two interesting limiting cases and demonstrate an increased efficiency of reheating which strongly enhances non-thermal topological defect formation, including monopoles and domain walls. Nevertheless, the large fluctuations may resolve this monopole problem via a modified Dvali-Liu-Vachaspati mechanism in which non-thermal destabilsation of discrete symmetries occurs at reheating.

Inflationary reheating via resonant production of non-minimally coupled scalar particles with only gravitational coupling is shown to be extremely strong, exhibiting a negative coupling instability for $\xi < 0$ and a wide resonance decay for $\xi \gg 1$. Since non-minimal fields are generic after renormalisation in curved spacetime, this offers a new paradigm in reheating - one which naturally allows for efficient production of the massive bosons needed for GUT baryogenesis. We also show that both vector and tensor fields are produced resonantly during reheating, extending the previously known correspondences between bosonic fields of different spins during preheating.

Inflationary reheating is almost completely controlled by the Floquet indices, $\mu_k$. Using spectral theory we demonstrate that the stability bands (where $\mu_k = 0$) of the Mathieu and Lam\'e equations are destroyed even in Minkowski spacetime, leaving a fractal Cantor set or a measure zero set of stable modes in the cases where the inflaton evolves in an almost-periodic or stochastic manner respectively. These two types of potential model the expected multi-field and quantum backreaction effects during reheating.

We consider the changes which occur in cosmological distances due to the combined effects of some null geodesics passing through low-density regions while others pass through lensing-induced caustics. This combination of effects increases observed areas corresponding to a given solid angle even when averaged over large angular scales, through the additive effect of increases on all scales, but particularly on micro-angular scales; however angular sizes will not be significantly effected on large angular scales (when caustics occur, area distances and angular-diameter distances no longer coincide). We compare our results with other works on lensing, which claim there is no such effect, and explain why the effect will indeed occur in the (realistic) situation where caustics due to lensing are significant. Whether or not the effect is significant for number counts depends on the associated angular scales and on the distribution of inhomogeneities in the universe. It could also possibly affect the spectrum of CBR anisotropies on small angular scales, indeed caustics can induce a non-Gaussian signature into the CMB at small scales and lead to stronger mixing of anisotropies than occurs in weak lensing.

We develop and apply a fully covariant 1+3 electromagnetic analogy for gravity. The free gravitational field is covariantly characterized by the Weyl gravito-electric and gravito-magnetic spatial tensor fields, whose dynamical equations are the Bianchi identities. Using a covariant generalization of spatial vector algebra and calculus to spatial tensor fields, we exhibit the covariant analogy between the tensor Bianchi equations and the vector Maxwell equations. We identify gravitational source terms, couplings and potentials with and without electromagnetic analogues. The nonlinear vacuum Bianchi equations are shown to be invariant under covariant spatial duality rotation of the gravito-electric and gravito-magnetic tensor fields. We construct the super-energy density and super-Poynting vector of the gravitational field as natural U(1) group invariants, and derive their super-energy conservation equation. A covariant approach to gravito-electric/magnetic monopoles is also presented.

We demonstrate the parametric amplification of the stochastic gravitational wave background during inflationary reheating and during axion/moduli oscillations. This amplification enhances the detectability of the string/inflationary gravity wave signal, leaving a finger-print on the spectrum which might be found with future gravitational wave detectors.

The gravitational wave equations form a parametric resonance system during oscillatory reheating after inflation, when cast in terms of the electric and magnetic parts of the Weyl tensor. This is in direct analogy with preheating. For chaotic inflation with a quadratic potential, this analogy is exact. The resulting amplification of the gravitational wave spectrum begins in the broad resonance regime for chaotic inflation but quickly moves towards narrow resonance. This increases the amplitude and breaks the scale invariance of the tensor spectrum generated during inflation, an effect which may be detectable. The parametric amplification is absent, however, in 1st-order phase transition models and "warm" inflationary models with continuous entropy production.

This paper and the others in the series challenge the standard model of the effects of gravitational lensing on observations at large distances. We show that due to the cumulative effect of lensing, areas corresponding to an observed solid angle can be quite different than would be estimated from the corresponding Friedmann-Lema\^{\i}tre model, even when averaged over large angular scales. This paper concentrates on the specific example of spherically symmetric but spatially inhomogeneous dust universes, the Lema\^{\i}tre-Tolman-Bondi models, and shows that radial lensing significantly distorts the area distance-redshift and density-redshift relations in these exact solutions compared with the standard ones for Friedmann-Lema\^{\i}tre models. Thus inhomogeneity may introduce significant errors into distance estimates based on the standard FL relations, even after all-sky averaging. In addition a useful new gauge choice is presented for these models, solving the problem of locating the past null cone exactly.

In this paper we examine the change in the estimated spatial power spectra at decoupling due to the effects of our clumpy universe which modify observational distances. We find that scales at decoupling can be significantly underestmated in our approximation of neglecting the shear of the ray bundle. We compare our results with other work on lensing and speculate on the implications for structure formation. In particular we examine a proposal to use the position of the first Doppler peak to determine $\Omega$, and find that shrinking will modify the esimated curvature, so that it must be included to obtain an accurate estimate of $\Omega$. Finally we consider future applications and improvements of our results.