The key challenge in the observation of the redshifted 21-cm signal from cosmic reionization is its separation from the much brighter foreground emission. Such separation relies on the different spectral properties of the two components, although, in real life, the foreground intrinsic spectrum is often corrupted by the instrumental response, inducing systematic effects that can further jeopardize the measurement of the 21-cm signal. In this paper, we use Gaussian Process Regression to model both foreground emission and instrumental systematics in $\sim 2$ hours of data from the Hydrogen Epoch of Reionization Array. We find that a simple co-variance model with three components matches the data well, giving a residual power spectrum with white noise properties. These consist of an "intrinsic" and instrumentally corrupted component with a coherence-scale of 20 MHz and 2.4 MHz respectively (dominating the line of sight power spectrum over scales $k_{\parallel} \le 0.2$ h cMpc$^{-1}$) and a baseline dependent periodic signal with a period of $\sim 1$ MHz (dominating over $k_{\parallel} \sim 0.4 - 0.8$h cMpc$^{-1}$) which should be distinguishable from the 21-cm EoR signal whose typical coherence-scales is $\sim 0.8$ MHz.

In 21 cm cosmology, precision calibration is key to the separation of the neutral hydrogen signal from bright astrophysical foregrounds. If not properly calibrated, each antenna element's complex gain can impart spectral structure on otherwise spectrally-smooth foregrounds, complicating that statistical separation. The Hydrogen Epoch of Reionization Array (HERA), an interferometer specialized for 21 cm cosmology and now under construction in South Africa, was designed to be largely calibrated using the self-consistency of repeated measurements of the same interferometric modes. This technique, known as redundant-baseline calibration resolves most of the internal degrees of freedom in the calibration problem. It assumes, however, on antenna elements with identical primary beams placed precisely on a redundant grid. In this work, we review the detailed implementation of the algorithms enabling redundant-baseline calibration and report results with HERA data. We quantify the effects of real-world non-redundancy and how they compare to the idealized scenario in which redundant measurements differ only in their noise realizations. Finally, we study how non-redundancy can produce spurious temporal structure in our calibration solutions---both in data and in simulations---and present strategies for mitigating that structure.

Foreground power dominates the measurements of interferometers that seek a statistical detection of highly-redshifted HI emission from the Epoch of Reionization (EoR). The chromaticity of the instrument creates a boundary in the Fourier transform of frequency (proportional to $k_\parallel$) between spectrally smooth emission, characteristic of the strong synchrotron foreground (the "wedge"), and the spectrally structured emission from HI in the EoR (the "EoR window"). Faraday rotation can inject spectral structure into otherwise smooth polarized foreground emission, which through instrument effects or miscalibration could possibly pollute the EoR window. Using data from the HERA 19-element commissioning array, we investigate the polarization response of this new instrument in the power spectrum domain. We perform a simple image-based calibration based on the unpolarized diffuse emission of the Global Sky Model, and show that it achieves qualitative redundancy between the nominally-redundant baselines of the array and reasonable amplitude accuracy. We construct power spectra of all fully polarized coherencies in all pseudo-Stokes parameters. We compare to simulations based on an unpolarized diffuse sky model and detailed electromagnetic simulations of the dish and feed, confirming that in Stokes I, the calibration does not add significant spectral structure beyond the expected level. Further, this calibration is stable over the 8 days of observations considered. Excess power is seen in the power spectra of the linear polarization Stokes parameters which is not easily attributable to leakage via the primary beam, and results from some combination of residual calibration errors and actual polarized emission. Stokes V is found to be highly discrepant from the expectation of zero power, strongly pointing to the need for more accurate polarized calibration.

Structure imprinted in foreground extragalactic point sources by ionospheric refraction has the potential to contaminate Epoch of Reionisation (EoR) power spectra of the 21~cm emission line of neutral hydrogen. The alteration of the spatial and spectral structure of foreground measurements due to total electron content (TEC) gradients in the ionosphere create a departure from the expected sky signal. We present a general framework for understanding the signatures of ionospheric behaviour in the two-dimensional (2D) neutral hydrogen power spectrum measured by a low-frequency radio interferometer. Two primary classes of ionospheric behaviour are considered, corresponding to dominant modes observed in Murchison Widefield Array (MWA) EoR data; namely, anisotropic structured wave behaviour, and isotropic turbulence. Analytic predictions for power spectrum bias due to this contamination are computed, and compared with simulations. We then apply the ionospheric metric described in Jordan et al. (2017) to study the impact of ionospheric structure on MWA data, by dividing MWA EoR datasets into classes with good and poor ionospheric conditions, using sets of matched 30-minute observations from 2014 September. The results are compared with the analytic and simulated predictions, demonstrating the observed bias in the power spectrum when the ionosphere is active (displays coherent structures or isotropic turbulence). The analysis demonstrates that unless ionospheric activity can be quantified and corrected, active data should not be included in EoR analysis in order to avoid systematic biases in cosmological power spectra. When data are corrected with a model formed from the calibration information, bias reduces below the expected 21~cm signal level. Data are considered `quiet' when the median measured source position offsets are less than 10-15~arcseconds.

Interferometric arrays seeking to measure the 21 cm signal from the Epoch of Reionization must contend with overwhelmingly bright emission from foreground sources. Accurate recovery of the 21 cm signal will require precise calibration of the array, and several new avenues for calibration have been pursued in recent years, including methods using redundancy in the antenna configuration. The newly upgraded Phase II of Murchison Widefield Array (MWA) is the first interferometer that has large numbers of redundant baselines while retaining good instantaneous UV-coverage. This array therefore provides a unique opportunity to compare redundant calibration with sky-model based algorithms. In this paper, we present the first results from comparing both calibration approaches with MWA Phase II observations. For redundant calibration, we use the package OMNICAL, and produce sky-based calibration solutions with the analysis package Fast Holographic Deconvolution (FHD). There are three principal results. (1) We report the success of OMNICAL on observations of ORBComm satellites, showing substantial agreement between redundant visibility measurements after calibration. (2) We directly compare OMNICAL calibration solutions with those from FHD, and demonstrate these two different calibration schemes give extremely similar results. (3) We explore improved calibration by combining OMNICAL and FHD. We evaluate these combined methods using power spectrum techniques developed for EoR analysis and find evidence for marginal improvements mitigating artifacts in the power spectrum. These results are likely limited by signal-to-noise in the six hours of data used, but suggest future directions for combining these two calibration schemes.

The experimental efforts to detect the redshifted 21 cm signal from the Epoch of Reionization (EoR) are limited predominantly by the chromatic instrumental systematic effect. The delay spectrum methodology for 21 cm power spectrum measurements brought new attention to the critical impact of an antenna's chromaticity on the viability of making this measurement. This methodology established a straightforward relationship between time-domain response of an instrument and the power spectrum modes accessible to a 21 cm EoR experiment. We examine the performance of a prototype of the Hydrogen Epoch of Reionization Array (HERA) array element that is currently observing in Karoo desert, South Africa. We present a mathematical framework to derive the beam integrated frequency response of a HERA prototype element in reception from the return loss measurements between 100-200 MHz and determined the extent of additional foreground contamination in the delay space. The measurement reveals excess spectral structures in comparison to the simulation studies of the HERA element. Combined with the HERA data analysis pipeline that incorporates inverse covariance weighting in optimal quadratic estimation of power spectrum, we find that in spite of its departure from the simulated response, HERA prototype element satisfies the necessary criteria posed by the foreground attenuation limits and potentially can measure the power spectrum at spatial modes as low as $k_{\parallel} > 0.1h$~Mpc$^{-1}$. The work highlights a straightforward method for directly measuring an instrument response and assessing its impact on 21 cm EoR power spectrum measurements for future experiments that will use reflector-type antenna.

We present new radio continuum observations of NGC253 from the Murchison Widefield Array at frequencies between 76 and 227 MHz. We model the broadband radio spectral energy distribution for the total flux density of NGC253 between 76 MHz and 11 GHz. The spectrum is best described as a sum of central starburst and extended emission. The central component, corresponding to the inner 500pc of the starburst region of the galaxy, is best modelled as an internally free-free absorbed synchrotron plasma, with a turnover frequency around 230 MHz. The extended emission component of the NGC253 spectrum is best described as a synchrotron emission flattening at low radio frequencies. We find that 34% of the extended emission (outside the central starburst region) at 1 GHz becomes partially absorbed at low radio frequencies. Most of this flattening occurs in the western region of the SE halo, and may be indicative of synchrotron self-absorption of shock re-accelerated electrons or an intrinsic low-energy cut off of the electron distribution. Furthermore, we detect the large-scale synchrotron radio halo of NGC253 in our radio images. At 154 - 231 MHz the halo displays the well known X-shaped/horn-like structure, and extends out to ~8kpc in z-direction (from major axis).

The Hydrogen Epoch of Reionization Array (HERA) is a radio interferometer aiming to detect the power spectrum of 21 cm fluctuations from neutral hydrogen from the Epoch of Reionization (EOR). Drawing on lessons from the Murchison Widefield Array (MWA) and the Precision Array for Probing the Epoch of Reionization (PAPER), HERA is a hexagonal array of large (14 m diameter) dishes with suspended dipole feeds. Not only does the dish determine overall sensitivity, it affects the observed frequency structure of foregrounds in the interferometer. This is the first of a series of four papers characterizing the frequency and angular response of the dish with simulations and measurements. We focus in this paper on the angular response (i.e., power pattern), which sets the relative weighting between sky regions of high and low delay, and thus, apparent source frequency structure. We measure the angular response at 137 MHz using the ORBCOMM beam mapping system of Neben et al. We measure a collecting area of 93 m^2 in the optimal dish/feed configuration, implying HERA-320 should detect the EOR power spectrum at z~9 with a signal-to-noise ratio of 12.7 using a foreground avoidance approach with a single season of observations, and 74.3 using a foreground subtraction approach. Lastly we study the impact of these beam measurements on the distribution of foregrounds in Fourier space.

We present a new method for interferometric imaging that is ideal for the large fields of view and compact arrays common in 21 cm cosmology. We first demonstrate the method with simulations for two very different low frequency interferometers, the Murchison Widefield Array (MWA) and the MIT Epoch of Reionization (MITEoR) Experiment. We then apply the method to the MITEoR data set collected in July 2013 to obtain the first northern sky map from 128 MHz to 175 MHz at about 2 degree resolution, and find an overall spectral index of -2.73+/-0.11. The success of this imaging method bodes well for upcoming compact redundant low-frequency arrays such as HERA. Both the MITEoR interferometric data and the 150 MHz sky map are publicly available at http://space.mit.edu/home/tegmark/omniscope.html.

The Detection of redshifted 21 cm emission from the epoch of reionization (EoR) is a challenging task owing to strong foregrounds that dominate the signal. In this paper, we propose a general method, based on the delay spectrum approach, to extract HI power spectra that is applicable to tracking observations using an imaging radio interferometer (Delay Spectrum with Imaging Arrays (DSIA)). Our method is based on modelling the HI signal taking into account the impact of wide field effects such as the $w$-term which are then used as appropriate weights in cross-correlating the measured visibilities. Our method is applicable to any radio interferometer that tracks a phase center and could be utilized for arrays such as MWA, LOFAR, GMRT, PAPER and HERA. In the literature the delay spectrum approach has been implemented for near-redundant baselines using drift scan observations. In this paper we explore the scheme for non-redundant tracking arrays, and this is the first application of delay spectrum methodology to such data to extract the HI signal. We analyze 3 hours of MWA tracking data on the EoR1 field. We present both 2-dimensional ($k_\parallel,k_\perp$) and 1-dimensional (k) power spectra from the analysis. Our results are in agreement with the findings of other pipelines developed to analyse the MWA EoR data.

We present an improved Global Sky Model (GSM) of diffuse galactic radio emission from 10 MHz to 5 THz, whose uses include foreground modeling for CMB and 21 cm cosmology. Our model improves on past work both algorithmically and by adding new data sets such as the Planck maps and the enhanced Haslam map. Our method generalizes the Principal Component Analysis approach to handle non-overlapping regions, enabling the inclusion of 29 sky maps with no region of the sky common to all. We also perform a blind separation of our GSM into physical components with a method that makes no assumptions about physical emission mechanisms (synchrotron, free-free, dust, etc). Remarkably, this blind method automatically finds five components that have previously only been found "by hand", which we identify with synchrotron, free-free, cold dust, warm dust, and the CMB anisotropy, with maps and spectra agreeing with previous work but in many cases with smaller error bars. The improved GSM is available online at github.com/jeffzhen/gsm2016.

We use time-domain electromagnetic simulations to determine the spectral characteristics of the Hydrogen Epoch of Reionization Arrays (HERA) antenna. These simulations are part of a multi-faceted campaign to determine the effectiveness of the dish's design for obtaining a detection of redshifted 21 cm emission from the epoch of reionization. Our simulations show the existence of reflections between HERA's suspended feed and its parabolic dish reflector that fall below -40 dB at 150 ns and, for reasonable impedance matches, have a negligible impact on HERA's ability to constrain EoR parameters. It follows that despite the reflections they introduce, dishes are effective for increasing the sensitivity of EoR experiments at relatively low cost. We find that electromagnetic resonances in the HERA feed's cylindrical skirt, which is intended to reduce cross coupling and beam ellipticity, introduces significant power at large delays ($-40$ dB at 200 ns) which can lead to some loss of measurable Fourier modes and a modest reduction in sensitivity. Even in the presence of this structure, we find that the spectral response of the antenna is sufficiently smooth for delay filtering to contain foreground emission at line-of-sight wave numbers below $k_\parallel \lesssim 0.2$ $h$Mpc$^{-1}$, in the region where the current PAPER experiment operates. Incorporating these results into a Fisher Matrix analysis, we find that the spectral structure observed in our simulations has only a small effect on the tight constraints HERA can achieve on parameters associated with the astrophysics of reionization.

The Hydrogen Epoch of Reionization Array (HERA) is a staged experiment to measure 21 cm emission from the primordial intergalactic medium (IGM) throughout cosmic reionization ($z=6-12$), and to explore earlier epochs of our Cosmic Dawn ($z\sim30$). During these epochs, early stars and black holes heated and ionized the IGM, introducing fluctuations in 21 cm emission. HERA is designed to characterize the evolution of the 21 cm power spectrum to constrain the timing and morphology of reionization, the properties of the first galaxies, the evolution of large-scale structure, and the early sources of heating. The full HERA instrument will be a 350-element interferometer in South Africa consisting of 14-m parabolic dishes observing from 50 to 250 MHz. Currently, 19 dishes have been deployed on site and the next 18 are under construction. HERA has been designated as an SKA Precursor instrument. In this paper, we summarize HERA's scientific context and provide forecasts for its key science results. After reviewing the current state of the art in foreground mitigation, we use the delay-spectrum technique to motivate high-level performance requirements for the HERA instrument. Next, we present the HERA instrument design, along with the subsystem specifications that ensure that HERA meets its performance requirements. Finally, we summarize the schedule and status of the project. We conclude by suggesting that, given the realities of foreground contamination, current-generation 21 cm instruments are approaching their sensitivity limits. HERA is designed to bring both the sensitivity and the precision to deliver its primary science on the basis of proven foreground filtering techniques, while developing new subtraction techniques to unlock new capabilities. The result will be a major step toward realizing the widely recognized scientific potential of 21 cm cosmology.

The Murchison Widefield Array (MWA) has collected hundreds of hours of Epoch of Reionization (EoR) data and now faces the challenge of overcoming foreground and systematic contamination to reduce the data to a cosmological measurement. We introduce several novel analysis techniques such as cable reflection calibration, hyper-resolution gridding kernels, diffuse foreground model subtraction, and quality control methods. Each change to the analysis pipeline is tested against a two dimensional power spectrum figure of merit to demonstrate improvement. We incorporate the new techniques into a deep integration of 32 hours of MWA data. This data set is used to place a systematic-limited upper limit on the cosmological power spectrum of $\Delta^2 \leq 2.7 \times 10^4$ mK$^2$ at $k=0.27$ h~Mpc$^{-1}$ and $z=7.1$, consistent with other published limits, and a modest improvement (factor of 1.4) over previous MWA results. From this deep analysis we have identified a list of improvements to be made to our EoR data analysis strategies. These improvements will be implemented in the future and detailed in upcoming publications.

We present deep polarimetric observations at 154 MHz with the Murchison Widefield Array (MWA), covering 625 deg^2 centered on RA=0 h, Dec=-27 deg. The sensitivity available in our deep observations allows an in-band, frequency-dependent analysis of polarized structure for the first time at long wavelengths. Our analysis suggests that the polarized structures are dominated by intrinsic emission but may also have a foreground Faraday screen component. At these wavelengths, the compactness of the MWA baseline distribution provides excellent snapshot sensitivity to large-scale structure. The observations are sensitive to diffuse polarized emission at ~54' resolution with a sensitivity of 5.9 mJy beam^-1 and compact polarized sources at ~2.4' resolution with a sensitivity of 2.3 mJy beam^-1 for a subset (400 deg^2) of this field. The sensitivity allows the effect of ionospheric Faraday rotation to be spatially and temporally measured directly from the diffuse polarized background. Our observations reveal large-scale structures (~1 deg - 8 deg in extent) in linear polarization clearly detectable in ~2 minute snapshots, which would remain undetectable by interferometers with minimum baseline lengths >110 m at 154 MHz. The brightness temperature of these structures is on average 4 K in polarized intensity, peaking at 11 K. Rotation measure synthesis reveals that the structures have Faraday depths ranging from -2 rad m^-2 to 10 rad m^-2 with a large fraction peaking at ~+1 rad m^-2. We estimate a distance of 51+/-20 pc to the polarized emission based on measurements of the in-field pulsar J2330-2005. We detect four extragalactic linearly polarized point sources within the field in our compact source survey. Based on the known polarized source population at 1.4 GHz and non-detections at 154 MHz, we estimate an upper limit on the depolarization ratio of 0.08 from 1.4 GHz to 154 MHz.

Detection of the Epoch of Reionization HI signal requires a precise understanding of the intervening galaxies and AGN, both for instrumental calibration and foreground removal. We present a catalogue of 7394 extragalactic sources at 182 MHz detected in the RA=0 field of the Murchison Widefield Array Epoch of Reionization observation programme. Motivated by unprecedented requirements for precision and reliability we develop new methods for source finding and selection. We apply machine learning methods to self-consistently classify the relative reliability of 9490 source candidates. A subset of 7466 are selected based on reliability class and signal-to-noise ratio criteria. These are statistically cross-matched to four other radio surveys using both position and flux density information. We find 7369 sources to have confident matches, including 90 partially resolved sources that split into a total of 192 sub-components. An additional 25 unmatched sources are included as new radio detections. The catalogue sources have a median spectral index of -0.85. Spectral flattening is seen toward lower frequencies with a median of -0.71 predicted at 182 MHz. The astrometric error is 7 arcsec. compared to a 2.3 arcmin. beam FWHM. The resulting catalogue covers approximately 1400 sq. deg. and is complete to approximately 80 mJy within half beam power. This provides the most reliable discrete source sky model available to date in the MWA EoR0 field for precision foreground subtraction.

We present the 21 cm power spectrum analysis approach of the Murchison Widefield Array Epoch of Reionization project. In this paper, we compare the outputs of multiple pipelines for the purpose of validating statistical limits cosmological hydrogen at redshifts between 6 and 12. Multiple, independent, data calibration and reduction pipelines are used to make power spectrum limits on a fiducial night of data. Comparing the outputs of imaging and power spectrum stages highlights differences in calibration, foreground subtraction and power spectrum calculation. The power spectra found using these different methods span a space defined by the various tradeoffs between speed, accuracy, and systematic control. Lessons learned from comparing the pipelines range from the algorithmic to the prosaically mundane; all demonstrate the many pitfalls of neglecting reproducibility. We briefly discuss the way these different methods attempt to handle the question of evaluating a significant detection in the presence of foregrounds.

We present first results from radio observations with the Murchison Widefield Array seeking to constrain the power spectrum of 21 cm brightness temperature fluctuations between the redshifts of 11.6 and 17.9 (113 and 75 MHz). Three hours of observations were conducted over two nights with significantly different levels of ionospheric activity. We use these data to assess the impact of systematic errors at low frequency, including the ionosphere and radio-frequency interference, on a power spectrum measurement. We find that after the 1-3 hours of integration presented here, our measurements at the Murchison Radio Observatory are not limited by RFI, even within the FM band, and that the ionosphere does not appear to affect the level of power in the modes that we expect to be sensitive to cosmology. Power spectrum detections, inconsistent with noise, due to fine spectral structure imprinted on the foregrounds by reflections in the signal-chain, occupy the spatial Fourier modes where we would otherwise be most sensitive to the cosmological signal. We are able to reduce this contamination using calibration solutions derived from autocorrelations so that we achieve an sensitivity of $10^4$ mK on comoving scales $k\lesssim 0.5 h$Mpc$^{-1}$. This represents the first upper limits on the $21$ cm power spectrum fluctuations at redshifts $12\lesssim z \lesssim 18$ but is still limited by calibration systematics. While calibration improvements may allow us to further remove this contamination, our results emphasize that future experiments should consider carefully the existence of and their ability to calibrate out any spectral structure within the EoR window.

Experiments that pursue detection of signals from the Epoch of Reionization (EoR) are relying on spectral smoothness of source spectra at low frequencies. This article empirically explores the effect of foreground spectra on EoR experiments by measuring high-resolution full-polarization spectra for the 586 brightest unresolved sources in one of the MWA EoR fields using 45 h of observation. A novel peeling scheme is used to subtract 2500 sources from the visibilities with ionospheric and beam corrections, resulting in the deepest, confusion-limited MWA image so far. The resulting spectra are found to be affected by instrumental effects, which limit the constraints that can be set on source-intrinsic spectral structure. The sensitivity and power-spectrum of the spectra are analysed, and it is found that the spectra of residuals are dominated by PSF sidelobes from nearby undeconvolved sources. We release a catalogue describing the spectral parameters for each measured source.

In this paper we present observations, simulations, and analysis demonstrating the direct connection between the location of foreground emission on the sky and its location in cosmological power spectra from interferometric redshifted 21 cm experiments. We begin with a heuristic formalism for understanding the mapping of sky coordinates into the cylindrically averaged power spectra measurements used by 21 cm experiments, with a focus on the effects of the instrument beam response and the associated sidelobes. We then demonstrate this mapping by analyzing power spectra with both simulated and observed data from the Murchison Widefield Array. We find that removing a foreground model which includes sources in both the main field-of-view and the first sidelobes reduces the contamination in high k_parallel modes by several percent relative to a model which only includes sources in the main field-of-view, with the completeness of the foreground model setting the principal limitation on the amount of power removed. While small, a percent-level amount of foreground power is in itself more than enough to prevent recovery of any EoR signal from these modes. This result demonstrates that foreground subtraction for redshifted 21 cm experiments is truly a wide-field problem, and algorithms and simulations must extend beyond the main instrument field-of-view to potentially recover the full 21 cm power spectrum.

Detection of the cosmological neutral hydrogen signal from the Epoch of Reionization, and estimation of its basic physical parameters, is the principal scientific aim of many current low-frequency radio telescopes. Here we describe the Cosmological HI Power Spectrum Estimator (CHIPS), an algorithm developed and implemented with data from the Murchison Widefield Array (MWA), to compute the two-dimensional and spherically-averaged power spectrum of brightness temperature fluctuations. The principal motivations for CHIPS are the application of realistic instrumental and foreground models to form the optimal estimator, thereby maximising the likelihood of unbiased signal estimation, and allowing a full covariant understanding of the outputs. CHIPS employs an inverse-covariance weighting of the data through the maximum likelihood estimator, thereby allowing use of the full parameter space for signal estimation ("foreground suppression"). We describe the motivation for the algorithm, implementation, application to real and simulated data, and early outputs. Upon application to a set of 3 hours of data, we set a 2$\sigma$ upper limit on the EoR dimensionless power at $k=0.05$~h.Mpc$^{-1}$ of $\Delta_k^2<7.6\times{10^4}$~mK$^2$ in the redshift range $z=[6.2-6.6]$, consistent with previous estimates.

We confirm our recent prediction of the "pitchfork" foreground signature in power spectra of high-redshift 21 cm measurements where the interferometer is sensitive to large-scale structure on all baselines. This is due to the inherent response of a wide-field instrument and is characterized by enhanced power from foreground emission in Fourier modes adjacent to those considered to be the most sensitive to the cosmological H I signal. In our recent paper, many signatures from the simulation that predicted this feature were validated against Murchison Widefield Array (MWA) data, but this key pitchfork signature was close to the noise level. In this paper, we improve the data sensitivity through the coherent averaging of 12 independent snapshots with identical instrument settings and provide the first confirmation of the prediction with a signal-to-noise ratio > 10. This wide-field effect can be mitigated by careful antenna designs that suppress sensitivity near the horizon. Simple models for antenna apertures that have been proposed for future instruments such as the Hydrogen Epoch of Reionization Array and the Square Kilometre Array indicate they should suppress foreground leakage from the pitchfork by ~40 dB relative to the MWA and significantly increase the likelihood of cosmological signal detection in these critical Fourier modes in the three-dimensional power spectrum.

The separation of the faint cosmological background signal from bright astrophysical foregrounds remains one of the most daunting challenges of mapping the high-redshift intergalactic medium with the redshifted 21 cm line of neutral hydrogen. Advances in mapping and modeling of diffuse and point source foregrounds have improved subtraction accuracy, but no subtraction scheme is perfect. Precisely quantifying the errors and error correlations due to missubtracted foregrounds allows for both the rigorous analysis of the 21 cm power spectrum and for the maximal isolation of the "EoR window" from foreground contamination. We present a method to infer the covariance of foreground residuals from the data itself in contrast to previous attempts at a priori modeling. We demonstrate our method by setting limits on the power spectrum using a 3 h integration from the 128-tile Murchison Widefield Array. Observing between 167 and 198 MHz, we find at 95% confidence a best limit of Delta^2(k) < 3.7 x 10^4 mK^2 at comoving scale k = 0.18 hMpc^-1 and at z = 6.8, consistent with existing limits.

Detection of 21~cm emission of HI from the epoch of reionization, at redshifts z>6, is limited primarily by foreground emission. We investigate the signatures of wide-field measurements and an all-sky foreground model using the delay spectrum technique that maps the measurements to foreground object locations through signal delays between antenna pairs. We demonstrate interferometric measurements are inherently sensitive to all scales, including the largest angular scales, owing to the nature of wide-field measurements. These wide-field effects are generic to all observations but antenna shapes impact their amplitudes substantially. A dish-shaped antenna yields the most desirable features from a foreground contamination viewpoint, relative to a dipole or a phased array. Comparing data from recent Murchison Widefield Array observations, we demonstrate that the foreground signatures that have the largest impact on the HI signal arise from power received far away from the primary field of view. We identify diffuse emission near the horizon as a significant contributing factor, even on wide antenna spacings that usually represent structures on small scales. For signals entering through the primary field of view, compact emission dominates the foreground contamination. These two mechanisms imprint a characteristic "pitchfork" signature on the "foreground wedge" in Fourier delay space. Based on these results, we propose that selective down-weighting of data based on antenna spacing and time can mitigate foreground contamination substantially by a factor ~100 with negligible loss of sensitivity.

The Murchison Widefield Array (MWA) is a new low-frequency interferometric radio telescope built in Western Australia at one of the locations of the future Square Kilometre Array (SKA). We describe the automated radio-frequency interference (RFI) detection strategy implemented for the MWA, which is based on the AOFlagger platform, and present 72-231-MHz RFI statistics from 10 observing nights. RFI detection removes 1.1% of the data. RFI from digital TV (DTV) is observed 3% of the time due to occasional ionospheric or atmospheric propagation. After RFI detection and excision, almost all data can be calibrated and imaged without further RFI mitigation efforts, including observations within the FM and DTV bands. The results are compared to a previously published Low-Frequency Array (LOFAR) RFI survey. The remote location of the MWA results in a substantially cleaner RFI environment compared to LOFAR's radio environment, but adequate detection of RFI is still required before data can be analysed. We include specific recommendations designed to make the SKA more robust to RFI, including: the availability of sufficient computing power for RFI detection; accounting for RFI in the receiver design; a smooth band-pass response; and the capability of RFI detection at high time and frequency resolution (second and kHz-scale respectively).

In order to study the "Cosmic Dawn" and the Epoch of Reionization with 21 cm tomography, we need to statistically separate the cosmological signal from foregrounds known to be orders of magnitude brighter. Over the last few years, we have learned much about the role our telescopes play in creating a putatively foreground-free region called the "EoR window." In this work, we examine how an interferometer's effects can be taken into account in a way that allows for the rigorous estimation of 21 cm power spectra from interferometric maps while mitigating foreground contamination and thus increasing sensitivity. This requires a precise understanding of the statistical relationship between the maps we make and the underlying true sky. While some of these calculations would be computationally infeasible if performed exactly, we explore several well-controlled approximations that make mapmaking and the calculation of map statistics much faster, especially for compact and highly redundant interferometers designed specifically for 21 cm cosmology. We demonstrate the utility of these methods and the parametrized trade-offs between accuracy and speed using one such telescope, the upcoming Hydrogen Epoch of Reionization Array, as a case study.

A number of experiments are currently working towards a measurement of the 21 cm signal from the Epoch of Reionization. Whether or not these experiments deliver a detection of cosmological emission, their limited sensitivity will prevent them from providing detailed information about the astrophysics of reionization. In this work, we consider what types of measurements will be enabled by a next-generation of larger 21 cm EoR telescopes. To calculate the type of constraints that will be possible with such arrays, we use simple models for the instrument, foreground emission, and the reionization history. We focus primarily on an instrument modeled after the $\sim 0.1~\rm{km}^2$ collecting area Hydrogen Epoch of Reionization Array (HERA) concept design, and parameterize the uncertainties with regard to foreground emission by considering different limits to the recently described "wedge" footprint in k-space. Uncertainties in the reionization history are accounted for using a series of simulations which vary the ionizing efficiency and minimum virial temperature of the galaxies responsible for reionization, as well as the mean free path of ionizing photons through the IGM. Given various combinations of models, we consider the significance of the possible power spectrum detections, the ability to trace the power spectrum evolution versus redshift, the detectability of salient power spectrum features, and the achievable level of quantitative constraints on astrophysical parameters. Ultimately, we find that $0.1~\rm{km}^2$ of collecting area is enough to ensure a very high significance ($\gtrsim30\sigma$) detection of the reionization power spectrum in even the most pessimistic scenarios. This sensitivity should allow for meaningful constraints on the reionization history and astrophysical parameters, especially if foreground subtraction techniques can be improved and successfully implemented.

We report on the MIT Epoch of Reionization (MITEoR) experiment, a pathfinder low-frequency radio interferometer whose goal is to test technologies that improve the calibration precision and reduce the cost of the high-sensitivity 3D mapping required for 21 cm cosmology. MITEoR accomplishes this by using massive baseline redundancy, which enables both automated precision calibration and correlator cost reduction. We demonstrate and quantify the power and robustness of redundancy for scalability and precision. We find that the calibration parameters precisely describe the effect of the instrument upon our measurements, allowing us to form a model that is consistent with $\chi^2$ per degree of freedom < 1.2 for as much as 80% of the observations. We use these results to develop an optimal estimator of calibration parameters using Wiener filtering, and explore the question of how often and how finely in frequency visibilities must be reliably measured to solve for calibration coefficients. The success of MITEoR with its 64 dual-polarization elements bodes well for the more ambitious Hydrogen Epoch of Reionization Array (HERA) project and other next-generation instruments, which would incorporate many identical or similar technologies.

We present techniques for bridging the gap between idealized inverse covariance weighted quadratic estimation of 21 cm power spectra and the real-world challenges presented universally by interferometric observation. By carefully evaluating various estimators and adapting our techniques for large but incomplete data sets, we develop a robust power spectrum estimation framework that preserves the so-called "EoR window" and keeps track of estimator errors and covariances. We apply our method to observations from the 32-tile prototype of the Murchinson Widefield Array to demonstrate the importance of a judicious analysis technique. Lastly, we apply our method to investigate the dependence of the clean EoR window on frequency--especially the frequency dependence of the so-called "wedge" feature--and establish upper limits on the power spectrum from z = 6.2 to z = 11.7. Our lowest limit is Delta(k) < 0.3 Kelvin at 95% confidence at a comoving scale k = 0.046 Mpc^-1 and z = 9.5.

Mapping our universe in 3D by imaging the redshifted 21 cm line from neutral hydrogen has the potential to overtake the cosmic microwave background as our most powerful cosmological probe, because it can map a much larger volume of our Universe, shedding new light on the epoch of reionization, inflation, dark matter, dark energy, and neutrino masses. We report on MITEoR, a pathfinder low-frequency radio interferometer whose goal is to test technologies that greatly reduce the cost of such 3D mapping for a given sensitivity. MITEoR accomplishes this by using massive baseline redundancy both to enable automated precision calibration and to cut the correlator cost scaling from N^2 to NlogN, where N is the number of antennas. The success of MITEoR with its 64 dual-polarization elements bodes well for the more ambitious HERA project, which would incorporate many identical or similar technologies using an order of magnitude more antennas, each with dramatically larger collecting area.

[Abridged] The spatially averaged global spectrum of the redshifted 21cm line has generated much experimental interest, for it is potentially a direct probe of the Epoch of Reionization and the Dark Ages. Since the cosmological signal here has a purely spectral signature, most proposed experiments have little angular sensitivity. This is worrisome because with only spectra, the global 21cm signal can be difficult to distinguish from foregrounds such as Galactic synchrotron radiation, as both are spectrally smooth and the latter is orders of magnitude brighter. We establish a mathematical framework for global signal data analysis in a way that removes foregrounds optimally, complementing spectra with angular information. We explore various experimental design trade-offs, and find that 1) with spectral-only methods, it is impossible to mitigate errors that arise from uncertainties in foreground modeling; 2) foreground contamination can be significantly reduced for experiments with fine angular resolution; 3) most of the statistical significance in a positive detection during the Dark Ages comes from a characteristic high-redshift trough in the 21cm brightness temperature; and 4) Measurement errors decrease more rapidly with integration time for instruments with fine angular resolution. We show that if observations and algorithms are optimized based on these findings, an instrument with a 5 degree beam can achieve highly significant detections (greater than 5-sigma) of even extended (high Delta-z) reionization scenarios after integrating for 500 hrs. This is in contrast to instruments without angular resolution, which cannot detect gradual reionization. Abrupt ionization histories can be detected at the level of 10-100's of sigma. The expected errors are also low during the Dark Ages, with a 25-sigma detection of the expected cosmological signal after only 100 hrs of integration.

We develop and demonstrate an acceleration of the Liu & Tegmark quadratic estimator formalism for inverse variance foreground subtraction and power spectrum estimation in 21 cm tomography from O(N^3) to O(N log N), where N is the number of voxels of data. This technique makes feasible the megavoxel scale analysis necessary for current and upcoming radio interferometers by making only moderately restrictive assumptions about foreground models and survey geometry. We exploit iterative and Monte Carlo techniques and the symmetries of the foreground covariance matrices to quickly estimate the 21 cm brightness temperature power spectrum, P(k_parallel, k_perpendicular), the Fisher information matrix, the error bars, the window functions, and the bias. We also extend the Liu & Tegmark foreground model to include bright point sources with known positions in a way that scales as O[(N log N)(N point sources)] < O(N^5/3). As a first application of our method, we forecast error bars and window functions for the upcoming 128-tile deployment of the Murchinson Widefield Array, showing that 1000 hours of observation should prove sufficiently sensitive to detect the power spectrum signal from the Epoch of Reionization.

We develop an approach for linking the power spectra, bispectrum, and trispectrum to the geometric and kinematical features of multifield inflationary Lagrangians. Our geometric approach can also be useful in determining when a complicated multifield model can be well approximated by a model with one, two, or a handful of fields. To arrive at these results, we focus on the mode interactions in the kinematical basis, starting with the case of no sourcing and showing that there is a series of mode conservation laws analogous to the conservation law for the adiabatic mode in single-field inflation. We then treat the special case of a quadratic potential with canonical kinetic terms, showing that it produces a series of mode sourcing relations identical in form to that for the adiabatic mode. We build on this result to show that the mode sourcing relations for general multifield inflation are extension of this special case but contain higher-order covariant derivatives of the potential and corrections from the field metric. In parallel, we show how these interactions depend on the geometry of the inflationary Lagrangian and on the kinematics of the associated field trajectory. Finally, we consider how the mode interactions and effective number of fields active during inflation are reflected in the spectra and introduce a multifield consistency relation, as well as a multifield observable that can potentially distinguish two-field scenarios from scenarios involving three or more effective fields.

We analyze cosmology assuming unitary quantum mechanics, using a tripartite partition into system, observer and environment degrees of freedom. This generalizes the second law of thermodynamics to "The system's entropy can't decrease unless it interacts with the observer, and it can't increase unless it interacts with the environment." The former follows from the quantum Bayes Theorem we derive. We show that because of the long-range entanglement created by cosmological inflation, the cosmic entropy decreases exponentially rather than linearly with the number of bits of information observed, so that a given observer can reduce entropy by much more than the amount of information her brain can store. Indeed, we argue that as long as inflation has occurred in a non-negligible fraction of the volume, almost all sentient observers will find themselves in a post-inflationary low-entropy Hubble volume, and we humans have no reason to be surprised that we do so as well, which solves the so-called inflationary entropy problem. An arguably worse problem for unitary cosmology involves gamma-ray-burst constraints on the "Big Snap", a fourth cosmic doomsday scenario alongside the "Big Crunch", "Big Chill" and "Big Rip", where an increasingly granular nature of expanding space modifies our life-supporting laws of physics. Our tripartite framework also clarifies when it is valid to make the popular quantum gravity approximation that the Einstein tensor equals the quantum expectation value of the stress-energy tensor, and how problems with recent attempts to explain dark energy as gravitational backreaction from super-horizon scale fluctuations can be understood as a failure of this approximation.

We study the quantum measurement problem in the context of an infinite, statistically uniform space, as could be generated by eternal inflation. It has recently been argued that when identical copies of a quantum measurement system exist, the standard projection operators and Born rule method for calculating probabilities must be supplemented by estimates of relative frequencies of observers. We argue that an infinite space actually renders the Born rule redundant, by physically realizing all outcomes of a quantum measurement in different regions, with relative frequencies given by the square of the wave function amplitudes. Our formal argument hinges on properties of what we term the quantum confusion operator, which projects onto the Hilbert subspace where the Born rule fails, and we comment on its relation to the oft-discussed quantum frequency operator. This analysis unifies the classical and quantum levels of parallel universes that have been discussed in the literature, and has implications for several issues in quantum measurement theory. It also shows how, even for a single measurement, probabilities may be interpreted as relative frequencies in unitary (Everettian) quantum mechanics. We also argue that after discarding a zero-norm part of the wavefunction, the remainder consists of a superposition of indistinguishable terms, so that arguably "collapse" of the wavefunction is irrelevant, and the "many worlds" of Everett's interpretation are unified into one. Finally, the analysis suggests a "cosmological interpretation" of quantum theory in which the wave function describes the actual spatial collection of identical quantum systems, and quantum uncertainty is attributable to the observer's inability to self-locate in this collection.

Before it becomes a sensitive probe of the Epoch of Reionization, the Dark Ages, and fundamental physics, 21 cm tomography must successfully contend with the issue of foreground contamination. Broadband foreground sources are expected to be roughly four orders of magnitude larger than any cosmological signals, so precise foreground models will be necessary. Such foreground models often contain a large number of parameters, reflecting the complicated physics that governs foreground sources. In this paper, we concentrate on spectral modeling (neglecting, for instance, bright point source removal from spatial maps) and show that 21 cm tomography experiments will likely not be able to measure these parameters without large degeneracies, simply because the foreground spectra are so featureless and generic. However, we show that this is also an advantage, because it means that the foregrounds can be characterized to a high degree of accuracy once a small number of parameters (likely three or four, depending on one's instrumental specifications) are measured. This provides a simple understanding for why 21 cm foreground subtraction schemes are able to remove most of the contaminants by suppressing just a small handful of simple spectral forms. In addition, this suggests that the foreground modeling process should be relatively simple and will likely not be an impediment to the foreground subtraction schemes that are necessary for a successful 21 cm tomography experiment.

21cm tomography promises to be a powerful tool for estimating cosmological parameters, constraining the epoch of reionization, and probing the so-called dark ages. However, realizing this promise will require the extraction of a cosmological power spectrum from beneath overwhelmingly large sources of foreground contamination. In this paper, we develop a unified matrix-based framework for foreground subtraction and power spectrum estimation, which allows us to quantify the errors and biases that arise in the power spectrum as a result of foreground subtraction. We find that existing line-of-sight foreground subtraction proposals can lead to substantial mode-mixing as well as residual noise and foreground biases, whereas our proposed inverse variance foreground subtraction eliminates noise and foreground biases, gives smaller error bars, and produces less correlated measurements of the power spectrum. We also numerically confirm the intuitive belief in the literature that 21cm foreground subtraction is best done using frequency rather than angular information.

We measure the luminosity and color dependence of galaxy clustering in the SDSS DR7 main galaxy sample, focusing on the projected correlation function w_p(r_p) of volume-limited samples. We interpret our measurements using halo occupation distribution (HOD) modeling assuming a Lambda-CDM cosmology. The amplitude of w_p(r_p) grows slowly with luminosity for L < L_* and increases sharply at higher luminosities, with bias factor b(>L)=1.06+0.23(L/L_*)^{1.12}. At fixed luminosity, redder galaxies have a stronger and steeper w_p(r_p), a trend that runs steadily from the bluest galaxies to the reddest galaxies. The individual luminosity trends for the red and blue galaxy populations are strikingly different. Blue galaxies show a slow but steady increase of w_p(r_p) with luminosity, at all scales. The large-scale clustering of red galaxies shows little luminosity dependence until a sharp increase at L > 4L_*, but the lowest luminosity red galaxies (0.04-0.25 L_*) show very strong clustering on scales r_p < 2 Mpc/h. Most of the observed trends can be naturally understood within the LCDM+HOD framework. The growth of w_p(r_p) with luminosity reflects an overall shift in the halo mass scale, in particular an increase in the minimum host halo mass Mmin. The mass at which a halo has, on average, one satellite galaxy brighter than L is M_1 ~ 17 Mmin(L) over most of the luminosity range. The growth and steepening of w_p(r_p) for redder galaxies reflects the increasing fraction of galaxies that are satellite systems in high mass halos instead of central systems in low mass halos, a trend that is especially marked at low luminosities. Our extensive measurements, provided in tabular form, will allow detailed tests of theoretical models of galaxy formation, a firm grounding of semi-empirical models of the galaxy population, and new cosmological tests.

We derive semi-analytic formulae for the local bispectrum and trispectrum in general two-field inflation and provide a simple geometric recipe for building observationally allowed models with observable non-Gaussianity. We use the \delta N formalism and the transfer function formalism to express the bispectrum almost entirely in terms of model-independent physical quantities. Similarly, we calculate the trispectrum and show that the trispectrum parameter \tau NL can be expressed entirely in terms of spectral observables, which provides a new consistency relation unique to two-field inflation. We show that in order to generate observably large non-Gaussianity during inflation, the sourcing of curvature modes by isocurvature modes must be extremely sensitive to the initial conditions, and that the amount of sourcing must be moderate in order to avoid excessive fine-tuning. Under some minimal assumptions, we argue that the first condition is satisfied only when neighboring trajectories through the two-dimensional field space diverge during inflation. Geometrically, this means that the inflaton must roll along a ridge in the potential V for some time during inflation and that its trajectory must turn slightly (but not too sharply) in field space. Therefore, it follows that two-field scenarios with attractor solutions necessarily produce small non-Gaussianity. This explains why it has been so difficult to achieve large non-Gaussianity in two-field inflation, and why it has only been achieved in a narrow class of models like hybrid inflation and certain product potentials where the potential and/or the initial conditions are fine-tuned. Some of our conclusions generalize qualitatively to general multi-field inflation.

We derive semi-analytic formulae for the power spectra of two-field inflation assuming an arbitrary potential and non-canonical kinetic terms, and we use them both to build phenomenological intuition and to constrain classes of two-field models using WMAP data. Using covariant formalism, we first develop a framework for understanding the background field kinematics and introduce a "slow-turn" approximation. Next, we find covariant expressions for the evolution of the adiabatic/curvature and entropy/isocurvature modes, and we discuss how the mode evolution can be inferred directly from the background kinematics and the geometry of the field manifold. From these expressions, we derive semi-analytic formulae for the curvature, isocurvature, and cross spectra, and the spectral observables, all to second-order in the slow-roll and slow-turn approximations. In tandem, we show how our covariant formalism provides useful intuition into how the characteristics of the inflationary Lagrangian translate into distinct features in the power spectra. In particular, we find that key features of the power spectra can be directly read off of the nature of the roll path, the curve the field vector rolls along with respect to the field manifold. For example, models whose roll path makes a sharp turn 60 e-folds before inflation ends tend to be ruled out because they produce strong departures from scale invariance. Finally, we apply our formalism to confront four classes of two-field models with WMAP data, including doubly quadratic and quartic potentials and non-standard kinetic terms, showing how whether a model is ruled out depends not only on certain features of the inflationary Lagrangian, but also on the initial conditions. Ultimately, models must possess the right balance of kinematical and dynamical behaviors, which we capture in a set of functions that can be reconstructed from spectral observables.

We show that the class of antenna layouts for telescope arrays allowing cheap analysis hardware (with correlator cost scaling as N log N rather than N^2 with the number of antennas N) is encouragingly large, including not only previously discussed rectangular grids but also arbitrary hierarchies of such grids, with arbitrary rotations and shears at each level. We show that all correlations for such a 2D array with an n-level hierarchy can be efficiently computed via a Fast Fourier Transform in not 2 but 2n dimensions. This can allow major correlator cost reductions for science applications requiring exquisite sensitivity at widely separated angular scales, for example 21cm tomography (where short baselines are needed to probe the cosmological signal and long baselines are needed for point source removal), helping enable future 21cm experiments with thousands or millions of cheap dipole-like antennas. Such hierarchical grids combine the angular resolution advantage of traditional array layouts with the cost advantage of a rectangular Fast Fourier Transform Telescope. We also describe an algorithm for how a subclass of hierarchical arrays can efficiently use rotation synthesis to produce global sky maps with minimal noise and a well-characterized synthesized beam.

The so-called corner turning problem is a major bottleneck for radio telescopes with large numbers of antennas. The problem is essentially that of rapidly transposing a matrix that is too large to store on one single device; in radio interferometry, it occurs because data from each antenna needs to be routed to an array of processors that will each handle a limited portion of the data (a frequency range, say) but requires input from each antenna. We present a low-cost solution allowing the correlator to transpose its data in real time, without contending for bandwidth, via a butterfly network requiring neither additional RAM memory nor expensive general-purpose switching hardware. We discuss possible implementations of this using FPGA, CMOS, analog logic and optical technology, and conclude that the corner turner cost can be small even for upcoming massive radio arrays.

Growing interest in 21 cm tomography has led to the design and construction of broadband radio interferometers with low noise, moderate angular resolution, high spectral resolution, and wide fields of view. With characteristics somewhat different from traditional radio instruments, these interferometers may require new calibration techniques in order to reach their design sensitivities. Self-calibration or redundant calibration techniques that allow an instrument to be calibrated off complicated sky emission structures are ideal. In particular, the large number of redundant baselines possessed by these new instruments makes redundant calibration an especially attractive option. In this paper, we explore the errors and biases in existing redundant calibration schemes through simulations, and show how statistical biases can be eliminated. We also develop a general calibration formalism that includes both redundant baseline methods and basic point source calibration methods as special cases, and show how slight deviations from perfect redundancy and coplanarity can be taken into account.

We survey observational constraints on the parameter space of inflation and axions and map out two allowed windows: the classic window and the inflationary anthropic window. The cosmology of the latter is particularly interesting; inflationary axion cosmology predicts the existence of isocurvature fluctuations in the CMB, with an amplitude that grows with both the energy scale of inflation and the fraction of dark matter in axions. Statistical arguments favor a substantial value for the latter, and so current bounds on isocurvature fluctuations imply tight constraints on inflation. For example, an axion Peccei-Quinn scale of 10^16 GeV excludes any inflation model with energy scale > 3.8*10^14 GeV (r > 2*10^(-9)) at 95% confidence, and so implies negligible gravitational waves from inflation, but suggests appreciable isocurvature fluctuations.

We make an estimate of the likelihood function for the Higgs vacuum expectation value by imposing anthropic constraints on the existence of atoms while allowing the other parameters of the Standard Model to also be variable. We argue that the most important extra ingredients are the Yukawa couplings, and for the intrinsic distribution of Yukawa couplings we use the scale invariant distribution which is favored phenomenologically. The result is successful phenomenologically, favoring values close to the observed vev. We also discuss modifications that could change these conclusions. Our work supports the hypothesis that the anthropic constraints could be the origin of the small value of the Higgs vev.

The spectroscopic Sloan Digital Sky Survey (SDSS) Data Release 7 (DR7) galaxy sample represents the final set of galaxies observed using the original SDSS target selection criteria. We analyse the clustering of galaxies within this sample, including both the Luminous Red Galaxy (LRG) and Main samples, and also include the 2-degree Field Galaxy Redshift Survey (2dFGRS) data. Baryon Acoustic Oscillations are observed in power spectra measured for different slices in redshift; this allows us to constrain the distance--redshift relation at multiple epochs. We achieve a distance measure at redshift z=0.275, of r_s(z_d)/D_V(0.275)=0.1390+/-0.0037 (2.7% accuracy), where r_s(z_d) is the comoving sound horizon at the baryon drag epoch, D_V(z)=[(1+z)^2D_A^2cz/H(z)]^(1/3), D_A(z) is the angular diameter distance and H(z) is the Hubble parameter. We find an almost independent constraint on the ratio of distances D_V(0.35)/D_V(0.2)=1.736+/-0.065, which is consistent at the 1.1sigma level with the best fit Lambda-CDM model obtained when combining our z=0.275 distance constraint with the WMAP 5-year data. The offset is similar to that found in previous analyses of the SDSS DR5 sample, but the discrepancy is now of lower significance, a change caused by a revised error analysis and a change in the methodology adopted, as well as the addition of more data. Using WMAP5 constraints on Omega_bh^2 and Omega_ch^2, and combining our BAO distance measurements with those from the Union Supernova sample, places a tight constraint on Omega_m=0.286+/-0.018 and H_0 = 68.2+/-2.2km/s/Mpc that is robust to allowing curvature and non-Lambda dark energy. This result is independent of the behaviour of dark energy at redshifts greater than those probed by the BAO and supernova measurements. (abridged)

The Sunyaev-Zel'dovich (SZ) effect is the inverse Compton-scattering of cosmic microwave background (CMB) photons by hot electrons in the intervening gas throughout the universe. The effect has a distinct spectral signature that allows its separation from other signals in multifrequency CMB datasets. Using CMB anisotropies measured at three frequencies by the BOOMERanG 2003 flight we constrain SZ fluctuations in the 10 arcmin to 1 deg angular range. Propagating errors and potential systematic effects through simulations, we obtain an overall upper limit of 15.3 uK (2 sigma) for rms SZ fluctuations in a broad bin between multipoles of of 250 and 1200 at the Rayleigh-Jeans (RJ) end of the spectrum. When combined with other CMB anisotropy and SZ measurements, we find that the local universe normalization of the density perturbations is sigma-8(SZ) < 0.96 at the 95% confidence level, consistent with sigma-8 determined from primordial perturbations.

We present the power spectrum of the reconstructed halo density field derived from a sample of Luminous Red Galaxies (LRGs) from the Sloan Digital Sky Survey Seventh Data Release (DR7). The halo power spectrum has a direct connection to the underlying dark matter power for k <= 0.2 h/Mpc, well into the quasi-linear regime. This enables us to use a factor of ~8 more modes in the cosmological analysis than an analysis with kmax = 0.1 h/Mpc, as was adopted in the SDSS team analysis of the DR4 LRG sample (Tegmark et al. 2006). The observed halo power spectrum for 0.02 < k < 0.2 h/Mpc is well-fit by our model: chi^2 = 39.6 for 40 degrees of freedom for the best fit LCDM model. We find \Omega_m h^2 * (n_s/0.96)^0.13 = 0.141^{+0.009}_{-0.012} for a power law primordial power spectrum with spectral index n_s and \Omega_b h^2 = 0.02265 fixed, consistent with CMB measurements. The halo power spectrum also constrains the ratio of the comoving sound horizon at the baryon-drag epoch to an effective distance to z=0.35: r_s/D_V(0.35) = 0.1097^{+0.0039}_{-0.0042}. Combining the halo power spectrum measurement with the WMAP 5 year results, for the flat LCDM model we find \Omega_m = 0.289 +/- 0.019 and H_0 = 69.4 +/- 1.6 km/s/Mpc. Allowing for massive neutrinos in LCDM, we find \sum m_{\nu} < 0.62 eV at the 95% confidence level. If we instead consider the effective number of relativistic species Neff as a free parameter, we find Neff = 4.8^{+1.8}_{-1.7}. Combining also with the Kowalski et al. (2008) supernova sample, we find \Omega_{tot} = 1.011 +/- 0.009 and w = -0.99 +/- 0.11 for an open cosmology with constant dark energy equation of state w.

21 cm tomography is expected to be difficult in part because of serious foreground contamination. Previous studies have found that line-of-sight approaches are capable of cleaning foregrounds to an acceptable level on large spatial scales, but not on small spatial scales. In this paper, we introduce a Fourier-space formalism for describing the line-of-sight methods, and use it to introduce an improved new method for 21 cm foreground cleaning. Heuristically, this method involves fitting foregrounds in Fourier space using weighted polynomial fits, with each pixel weighted according to its information content. We show that the new method reproduces the old one on large angular scales, and gives marked improvements on small scales at essentially no extra computational cost.

We use Minkowski Functionals (MF) to constrain a primordial non-Gaussian contribution to the CMB intensity field as observed in the 150 GHz and 145 GHz BOOMERanG maps from the 1998 and 2003 flights, respectively, performing for the first time a joint analysis of the two datasets. A perturbative expansion of the MF formulae in the limit of a weakly non-Gaussian field yields analytical formulae, derived by Hikage et al. (2006), which can be used to constrain the coupling parameter f_NL without the need for non-Gaussian simulations. We find -1020<f_NL<390 at 95% CL, significantly improving the previous constraints by De Troia et al. (2007) on the BOOMERanG 2003 dataset. These are the best f_NL limits to date for suborbital probes.

Was Einstein wrong? This paper provides a detailed technical review of Einstein's special and general relativity from an astrophysical perspective, including the historical development of the theories, experimental tests, modern applications to black holes, cosmology and parallel universes, and last but not least, novel ways of expressing their seven most important equations.

We propose an all-digital telescope for 21 cm tomography, which combines key advantages of both single dishes and interferometers. The electric field is digitized by antennas on a rectangular grid, after which a series of Fast Fourier Transforms recovers simultaneous multifrequency images of up to half the sky. Thanks to Moore's law, the bandwidth up to which this is feasible has now reached about 1 GHz, and will likely continue doubling every couple of years. The main advantages over a single dish telescope are cost and orders of magnitude larger field-of-view, translating into dramatically better sensitivity for large-area surveys. The key advantages over traditional interferometers are cost (the correlator computational cost for an N-element array scales as N log N rather than N^2) and a compact synthesized beam. We argue that 21 cm tomography could be an ideal first application of a very large Fast Fourier Transform Telescope, which would provide both massive sensitivity improvements per dollar and mitigate the off-beam point source foreground problem with its clean beam. Another potentially interesting application is cosmic microwave background polarization.

I use cosmology examples to illustrate that the second law of thermodynamics is not old and tired, but alive and kicking, continuing to stimulate interesting research on really big puzzles. The question "Why is the entropy so low?" (despite the second law) suggests that our observable universe is merely a small and rather uniform patch in a vastly larger space stretched out by cosmological inflation. The question "Why is the entropy so high" (compared to the complexity required to describe many candidate "theories of everything") independently suggests that physical reality is much larger than the part we can observe.

A new and powerful probe of the origin and evolution of structures in the Universe has emerged and been actively developed over the last decade. In the coming decade, non-Gaussianity, i.e., the study of non-Gaussian contributions to the correlations of cosmological fluctuations, will become an important probe of both the early and the late Universe. Specifically, it will play a leading role in furthering our understanding of two fundamental aspects of cosmology and astrophysics: (i) the physics of the very early universe that created the primordial seeds for large-scale structures, and (ii) the subsequent growth of structures via gravitational instability and gas physics at later times. To date, observations of fluctuations in the Cosmic Microwave Background (CMB) and the Large-Scale Structure of the Universe (LSS) have focused largely on the Gaussian contribution as measured by the two-point correlations (or the power spectrum) of density fluctuations. However, an even greater amount of information is contained in non-Gaussianity and a large discovery space therefore still remains to be explored. Many observational probes can be used to measure non-Gaussianity, including CMB, LSS, gravitational lensing, Lyman-alpha forest, 21-cm fluctuations, and the abundance of rare objects such as clusters of galaxies and high-redshift galaxies. Not only does the study of non-Gaussianity maximize the science return from a plethora of present and future cosmological experiments and observations, but it also carries great potential for important discoveries in the coming decade.

How did the universe evolve? The fine angular scale (l>1000) temperature and polarization anisotropies in the CMB are a Rosetta stone for understanding the evolution of the universe. Through detailed measurements one may address everything from the physics of the birth of the universe to the history of star formation and the process by which galaxies formed. One may in addition track the evolution of the dark energy and discover the net neutrino mass. We are at the dawn of a new era in which hundreds of square degrees of sky can be mapped with arcminute resolution and sensitivities measured in microKelvin. Acquiring these data requires the use of special purpose telescopes such as the Atacama Cosmology Telescope (ACT), located in Chile, and the South Pole Telescope (SPT). These new telescopes are outfitted with a new generation of custom mm-wave kilo-pixel arrays. Additional instruments are in the planning stages.

One of the next decade's most exciting prospects is to explore the cosmic "dark ages," during which the first stars in the Universe formed, with the 21 cm line of neutral hydrogen. At z>6, this light redshifts into the low-frequency radio regime and can be observed with new generations of wide-field arrays. These experiments have the potential to observe enormous volumes and to improve both cosmological and astrophysical constraints dramatically. In this white paper, we describe how the next decade will see the first steps toward unlocking this potential, allowing us to answer two key questions about the fundamental physics of our Universe and the intergalactic medium: Does the standard cosmological model describe the Universe during the "dark ages?" How did the intergalactic medium evolve during this important time, ending with the reionization of hydrogen?

The cosmic dark ages and the epoch of reionization, during which the first generations of stars and galaxies formed, are among the most compelling frontiers of extragalactic astrophysics and cosmology. Here we describe an exciting new probe of these eras: the 21 cm line of neutral hydrogen, which will allow us to map the neutral intergalactic medium throughout the era of first galaxy formation. In this white paper, we describe how these studies can answer two key questions about galaxy formation: What were the properties of high-redshift galaxies? How did they affect the Universe around them?

21 cm tomography is emerging as a promising probe of the cosmological dark ages and the epoch of reionization, as well as a tool for observational cosmology in general. However, serious sources of foreground contamination must be subtracted for experimental efforts to be viable. In this paper, we focus on the removal of unresolved extragalactic point sources with smooth spectra, and evaluate how the residual foreground contamination after cleaning depends on instrumental and algorithmic parameters. A crucial but often ignored complication is that the synthesized beam of an interferometer array shrinks towards higher frequency, causing complicated frequency structure in each sky pixel as "frizz" far from the beam center contracts across unresolved radio sources. We find that current-generation experiments should nonetheless be able to clean out this points source contamination adequately, and quantify the instrumental and algorithmic design specifications required to meet this foreground challenge.

There is growing interest in using 3-dimensional neutral hydrogen mapping with the redshifted 21 cm line as a cosmological probe, as it has been argued to have a greater long-term potential than the cosmic microwave background. However, its utility depends on many assumptions. To aid experimental planning and design, we quantify how the precision with which cosmological parameters can be measured depends on a broad range of assumptions. We cover assumptions related to modeling of the ionization power spectrum and associated nonlinearity, experimental specifications like array layout and noise, cosmological assumptions about reionization history and inter-galactic medium (IGM) evolution, and assumptions about astrophysical foregrounds. We derive simple analytic approximations for how various assumptions affect the results, and find that ionization power modeling is most important, followed by array layout (crudely, the more compact, the better). We also present an accurate yet robust method for measuring cosmological parameters in practice, separating the physics from the astrophysics by exploiting both gravitationally induced clustering anisotropy and the fact that the ionization power spectra are rather smooth functions that can be accurately fit by 7 phenomenological parameters. For example, a future square kilometer array optimized for 21 cm tomography could improve the sensitivity of the Planck CMB satellite to spatial curvature and neutrino masses by up to two orders of magnitude, to Delta-Omega_k ~ 0.0002 and Delta m_nu ~ 0.007 eV, and give a 4 sigma detection of the spectral index running predicted by the simpliest inflation models.

As galaxy surveys become larger and more complex, keeping track of the completeness, magnitude limit, and other survey parameters as a function of direction on the sky becomes an increasingly challenging computational task. For example, typical angular masks of the Sloan Digital Sky Survey contain about N=300,000 distinct spherical polygons. Managing masks with such large numbers of polygons becomes intractably slow, particularly for tasks that run in time O(N^2) with a naive algorithm, such as finding which polygons overlap each other. Here we present a "divide-and-conquer" solution to this challenge: we first split the angular mask into predefined regions called "pixels," such that each polygon is in only one pixel, and then perform further computations, such as checking for overlap, on the polygons within each pixel separately. This reduces O(N^2) tasks to O(N), and also reduces the important task of determining in which polygon(s) a point on the sky lies from O(N) to O(1), resulting in significant computational speedup. Additionally, we present a method to efficiently convert any angular mask to and from the popular HEALPix format. This method can be generically applied to convert to and from any desired spherical pixelization. We have implemented these techniques in a new version of the mangle software package, which is freely available at http://space.mit.edu/home/tegmark/mangle/, along with complete documentation and example applications. These new methods should prove quite useful to the astronomical community, and since mangle is a generic tool for managing angular masks on a sphere, it has the potential to benefit terrestrial mapmaking applications as well.

We prove that inflation is forbidden in the most well understood class of semi-realistic type IIA string compactifications: Calabi-Yau compactifications with only standard NS-NS 3-form flux, R-R fluxes, D6-branes and O6-planes at large volume and small string coupling. With these ingredients, the first slow-roll parameter satisfies epsilon >= 27/13 whenever V > 0, ruling out both inflation (including brane/anti-brane inflation) and de Sitter vacua in this limit. Our proof is based on the dependence of the 4-dimensional potential on the volume and dilaton moduli in the presence of fluxes and branes. We also describe broader classes of IIA models which may include cosmologies with inflation and/or de Sitter vacua. The inclusion of extra ingredients, such as NS 5-branes and geometric or non-geometric NS-NS fluxes, evades the assumptions used in deriving the no-go theorem. We focus on NS 5-branes and outline how such ingredients may prove fruitful for cosmology, but we do not provide an explicit model. We contrast the results of our IIA analysis with the rather different situation in IIB.

Understanding diffuse Galactic radio emission is interesting both in its own right and for minimizing foreground contamination of cosmological measurements. Cosmic Microwave Background experiments have focused on frequencies > 10 GHz, whereas 21 cm tomography of the high redshift universe will mainly focus on < 0.2 GHz, for which less is currently known about Galactic emission. Motivated by this, we present a global sky model derived from all publicly available total power large-area radio surveys, digitized with optical character recognition when necessary and compiled into a uniform format, as well as the new Villa Elisa data extending the 1.4 GHz map to the entire sky. We quantify statistical and systematic uncertainties in these surveys by comparing them with various global multi-frequency model fits. We find that a principal component based model with only three components can fit the 11 most accurate data sets (at 10, 22, 45 & 408 MHz and 1.4, 2.3, 23, 33, 41, 61, 94 GHz) to an accuracy around 1%-10% depending on frequency and sky region. Both our data compilation and our software returning a predicted all-sky map at any frequency from 10 MHz to 100 GHz are publicly available at http://space.mit.edu/home/angelica/gsm .

Differences in clustering properties between galaxy subpopulations complicate the cosmological interpretation of the galaxy power spectrum, but can also provide insights about the physics underlying galaxy formation. To study the nature of this relative clustering, we perform a counts-in-cells analysis of galaxies in the Sloan Digital Sky Survey (SDSS) in which we measure the relative bias between pairs of galaxy subsamples of different luminosities and colours. We use a generalized chi-squared test to determine if the relative bias between each pair of subsamples is consistent with the simplest deterministic linear bias model, and we also use a maximum likelihood technique to further understand the nature of the relative bias between each pair. We find that the simple, deterministic model is a good fit for the luminosity-dependent bias on scales above ~2 Mpc/h, which is good news for using magnitude-limited surveys for cosmology. However, the colour-dependent bias shows evidence for stochasticity and/or non-linearity which increases in strength toward smaller scales, in agreement with previous studies of stochastic bias. Also, confirming hints seen in earlier work, the luminosity-dependent bias for red galaxies is significantly different from that of blue galaxies: both luminous and dim red galaxies have higher bias than moderately bright red galaxies, whereas the biasing of blue galaxies is not strongly luminosity-dependent. These results can be used to constrain galaxy formation models and also to quantify how the colour and luminosity selection of a galaxy survey can impact measurements of the cosmological matter power spectrum.

Attempts to connect string theory with astrophysical observation are hampered by a jargon barrier, where an intimidating profusion of orientifolds, Kahler potentials, etc. dissuades cosmologists from attempting to work out the astrophysical observables of specific string theory solutions from the recent literature. We attempt to help bridge this gap by giving a pedagogical exposition with detailed examples, aimed at astrophysicists and high energy theorists alike, of how to compute predictions for familiar cosmological parameters when starting with a 10-dimensional string theory action. This is done by investigating inflation in string theory, since inflation is the dominant paradigm for how early universe physics determines cosmological parameters. We analyze three explicit string models from the recent literature, each containing an infinite number of "vacuum" solutions. Our numerical investigation of some natural candidate inflatons, the so-called "moduli fields", fails to find inflation. We also find in the simplest models that, after suitable field redefinitions, vast numbers of these vacua differ only in an overall constant multiplying the effective inflaton potential, a difference which affects neither the potential's shape nor its ability to support slow-roll inflation. This illustrates that even having an infinite number of vacua does not guarantee having inflating ones. This may be an artifact of the simplicity of the models that we study. Instead, more complicated string theory models appear to be required, suggesting that explicitly identifying the inflating subset of the string landscape will be challenging.

We analyze the BOOMERanG 2003 (B03) 145 GHz temperature map to constrain the amplitude of a non Gaussian, primordial contribution to CMB fluctuations. We perform a pixel space analysis restricted to a portion of the map chosen in view of high sensitivity, very low foreground contamination and tight control of systematic effects. We set up an estimator based on the three Minkowski functionals which relies on high quality simulated data, including non Gaussian CMB maps. We find good agreement with the Gaussian hypothesis and derive the first limits based on BOOMERanG data for the non linear coupling parameter f_NL as -300<f_NL<650 at 68% CL and -800<f_NL<1050 at 95% CL.

I explore physics implications of the External Reality Hypothesis (ERH) that there exists an external physical reality completely independent of us humans. I argue that with a sufficiently broad definition of mathematics, it implies the Mathematical Universe Hypothesis (MUH) that our physical world is an abstract mathematical structure. I discuss various implications of the ERH and MUH, ranging from standard physics topics like symmetries, irreducible representations, units, free parameters, randomness and initial conditions to broader issues like consciousness, parallel universes and Godel incompleteness. I hypothesize that only computable and decidable (in Godel's sense) structures exist, which alleviates the cosmological measure problem and help explain why our physical laws appear so simple. I also comment on the intimate relation between mathematical structures, computations, simulations and physical systems.

It is well-entrenched folklore that torsion gravity theories predict observationally negligible torsion in the solar system, since torsion (if it exists) couples only to the intrinsic spin of elementary particles, not to rotational angular momentum. We argue that this assumption has a logical loophole which can and should be tested experimentally. In the spirit of action=reaction, if a rotating mass like a planet can generate torsion, then a gyroscope should also feel torsion. Using symmetry arguments, we show that to lowest order, the torsion field around a uniformly rotating spherical mass is determined by seven dimensionless parameters. These parameters effectively generalize the PPN formalism and provide a concrete framework for further testing GR. We construct a parametrized Lagrangian that includes both standard torsion-free GR and Hayashi- Shirafuji maximal torsion gravity as special cases. We demonstrate that classic solar system tests rule out the latter and constrain two observable parameters. We show that Gravity Probe B (GPB) is an ideal experiment for further constraining torsion theories, and work out the most general torsion-induced precession of its gyroscope in terms of our torsion parameters

We analyze the 3-year WMAP data and look for a deviation from Gaussianity in the form of a 3-point function that has either of the two theoretically motivated shapes: local and equilateral. There is no evidence of departure from Gaussianity and the analysis gives the presently tightest bounds on the parameters f_ NL^local and f_NL^equil., which define the amplitude of respectively the local and the equilateral non-Gaussianity: -36 < f_NL^local < 100, -256 < f_NL^equil. < 332 at 95% C.L.

We search for viable f(R) theories of gravity, making use of the equivalence between such theories and scalar-tensor gravity. We find that models can be made consistent with solar system constraints either by giving the scalar a high mass or by exploiting the so-called chameleon effect. However, in both cases, it appears likely that any late-time cosmic acceleration will be observationally indistinguishable from acceleration caused by a cosmological constant. We also explore further observational constraints from, e.g., big bang nucleosynthesis and inflation.

We measure the large-scale real-space power spectrum P(k) using luminous red galaxies (LRGs) in the Sloan Digital Sky Survey (SDSS) and use this measurement to sharpen constraints on cosmological parameters from the Wilkinson Microwave Anisotropy Probe (WMAP). We employ a matrix-based power spectrum estimation method using Pseudo-Karhunen-Loeve eigenmodes, producing uncorrelated minimum-variance measurements in 20 k-bands of both the clustering power and its anisotropy due to redshift-space distortions, with narrow and well-behaved window functions in the range 0.01h/Mpc < k < 0.2h/Mpc. Results from the LRG and main galaxy samples are consistent, with the former providing higher signal-to-noise. Our results are robust to omitting angular and radial density fluctuations and are consistent between different parts of the sky. They provide a striking confirmation of the predicted large-scale LCDM power spectrum. Combining only SDSS LRG and WMAP data places robust constraints on many cosmological parameters that complement prior analyses of multiple data sets. The LRGs provide independent cross-checks on Om and the baryon fraction in good agreement with WMAP. Within the context of flat LCDM models, our LRG measurements complement WMAP by sharpening the constraints on the matter density, the neutrino density and the tensor amplitude by about a factor of two, giving Omega_m=0.24+-0.02 (1 sigma), sum m_nu < 0.9 eV (95%) and r<0.3 (95%). Baryon oscillations are clearly detected and provide a robust measurement of the comoving distance to the median survey redshift z=0.35 independent of curvature and dark energy properties. Within the LCDM framework, our power spectrum measurement improves the evidence for spatial flatness, sharpening the curvature constraint Omega_tot=1.05+-0.05 from WMAP alone to Omega_tot=1.003+-0.010. Assuming Omega_tot=1, the equation of state parameter is constrained to w=-0.94+-0.09, indicating the potential for more ambitious future LRG measurements to provide precision tests of the nature of dark energy. All these constraints are essentially independent of scales k>0.1h/Mpc and associated nonlinear complications, yet agree well with more aggressive published analyses where nonlinear modeling is crucial.

We present a Fourier analysis of the clustering of galaxies in the combined Main galaxy and Luminous Red Galaxy (LRG) Sloan Digital Sky Survey (SDSS) Data Release 5 (DR5) sample. The aim of our analysis is to consider how well we can measure the cosmological matter density using the signature of the horizon at matter-radiation equality embedded in the large-scale power spectrum. The new data constrains the power spectrum on scales 100--600h^-1Mpc with significantly higher precision than previous analyses of just the SDSS Main galaxies, due to our larger sample and the inclusion of the LRGs. This improvement means that we can now reveal a discrepancy between the shape of the measured power and linear CDM models on scales 0.01<k<0.15hMpc^-1, with linear model fits favouring a lower matter density (Omega_m=0.22+/-0.04) on scales 0.01<k<0.06hMpc^-1 and a higher matter density (Omega_m=0.32+/-0.01) when smaller scales are included, assuming a flat LCDM model with h=0.73 and n_s=0.96. This discrepancy could be explained by scale-dependent bias and, by analysing subsamples of galaxies, we find that the ratio of small-scale to large-scale power increases with galaxy luminosity, so all of the SDSS galaxies cannot trace the same power spectrum shape over 0.01<k<0.2hMpc^-1. However, the data are insufficient to clearly show a luminosity-dependent change in the largest scale at which a significant increase in clustering is observed, although they do not rule out such an effect. Significant scale-dependent galaxy bias on large-scales, which changes with the r-band luminosity of the galaxies, could potentially explain differences in our Omega_m estimates and differences previously observed between 2dFGRS and SDSS power spectra and the resulting parameter constraints.

Twenty-one centimeter tomography is emerging as a powerful tool to explore the end of the cosmic dark ages and the reionization epoch, but it will only be as good as our ability to accurately model and remove astrophysical foreground contamination. Previous treatments of this problem have focused on the angular structure of the signal and foregrounds and what can be achieved with limited spectral resolution (bandwidths in the 1 MHz range). In this paper we introduce and evaluate a ``blind'' method to extract the multifrequency 21cm signal by taking advantage of the smooth frequency structure of the Galactic and extragalactic foregrounds. We find that 21 cm tomography is typically limited by foregrounds on scales $k\ll 1h/$Mpc and limited by noise on scales $k\gg 1h/$Mpc, provided that the experimental bandwidth can be made substantially smaller than 0.1 MHz. Our results show that this approach is quite promising even for scenarios with rather extreme contamination from point sources and diffuse Galactic emission, which bodes well for upcoming experiments such as LOFAR, MWA, PAST, and SKA.

We present the 3D real space clustering power spectrum of a sample of \~600,000 luminous red galaxies (LRGs) measured by the Sloan Digital Sky Survey (SDSS), using photometric redshifts. This sample of galaxies ranges from redshift z=0.2 to 0.6 over 3,528 deg^2 of the sky, probing a volume of 1.5 (Gpc/h)^3, making it the largest volume ever used for galaxy clustering measurements. We measure the angular clustering power spectrum in eight redshift slices and combine these into a high precision 3D real space power spectrum from k=0.005 (h/Mpc) to k=1 (h/Mpc). We detect power on gigaparsec scales, beyond the turnover in the matter power spectrum, on scales significantly larger than those accessible to current spectroscopic redshift surveys. We also find evidence for baryonic oscillations, both in the power spectrum, as well as in fits to the baryon density, at a 2.5 sigma confidence level. The statistical power of these data to constrain cosmology is ~1.7 times better than previous clustering analyses. Varying the matter density and baryon fraction, we find \Omega_M = 0.30 \pm 0.03, and \Omega_b/\Omega_M = 0.18 \pm 0.04, The detection of baryonic oscillations also allows us to measure the comoving distance to z=0.5; we find a best fit distance of 1.73 \pm 0.12 Gpc, corresponding to a 6.5% error on the distance. These results demonstrate the ability to make precise clustering measurements with photometric surveys (abridged).

Most analysis of Cosmic Microwave Background spherical harmonic coefficients a_lm has focused on estimating the power spectrum C_l=<|a_lm|^2> rather than the coefficients themselves. We present a minimum-variance method for measuring a_lm given anisotropic noise, incomplete sky coverage and foreground contamination, and apply it to the WMAP data. Our method is shown to constitute lossless data compression in the sense that the widely used quadratic estimators of the power spectrum C_l can be computed directly from our a_lm-estimators. As the Galactic cut is increased, the error bars Delta-a_lm on low multipoles go from being dominated by foregrounds to being dominated by sample variance from other multipoles, with the intervening minimum defining the optimal cut. Applying our method to the WMAP quadrupole and octopole, we find that their previously reported "axis of evil" alignment appears to be rather robust to Galactic cut and foreground contamination.

One of the most spectacular scientific breakthroughs in past decades was using measurements of the fluctuations in the cosmic microwave background (CMB) to test precisely our understanding of the history and composition of the Universe. This report presents a roadmap for leading CMB research to its logical next step, using precision polarization measurements to learn about ultra-high-energy physics and the Big Bang itself.

One of the main challenges in cosmology is to quantify how small density fluctuations at the recombination epoch z around 1000 evolved into the galaxies and the large-scale structure we observe in the universe today. This thesis discusses ways of probing the intermediate epoch, focusing on the thermal history. The main emphasis is on the role played by non-linear feedback, where a small fraction of matter forming luminous objects can inject enough energy into the inter-galactic medium to radically alter subsequent events. Chapter 2, "A Cosmology Primer", provides a 60-page introduction to cosmology that may be useful for the reader with a general physics background.

We identify 31 dimensionless physical constants required by particle physics and cosmology, and emphasize that both microphysical constraints and selection effects might help elucidate their origin. Axion cosmology provides an instructive example, in which these two kinds of arguments must both be taken into account, and work well together. If a Peccei-Quinn phase transition occurred before or during inflation, then the axion dark matter density will vary from place to place with a probability distribution. By calculating the net dark matter halo formation rate as a function of all four relevant cosmological parameters and assessing other constraints, we find that this probability distribution, computed at stable solar systems, is arguably peaked near the observed dark matter density. If cosmologically relevant WIMP dark matter is discovered, then one naturally expects comparable densities of WIMPs and axions, making it important to follow up with precision measurements to determine whether WIMPs account for all of the dark matter or merely part of it.

Numerous Earth-destroying doomsday scenarios have recently been analyzed, including breakdown of a metastable vacuum state and planetary destruction triggered by a "strangelet'' or microscopic black hole. We point out that many previous bounds on their frequency give a false sense of security: one cannot infer that such events are rare from the the fact that Earth has survived for so long, because observers are by definition in places lucky enough to have avoided destruction. We derive a new upper bound of one per 10^9 years (99.9% c.l.) on the exogenous terminal catastrophe rate that is free of such selection bias, using planetary age distributions and the relatively late formation time of Earth.

We have produced a new conformal map of the universe illustrating recent discoveries, ranging from Kuiper belt objects in the Solar system, to the galaxies and quasars from the Sloan Digital Sky Survey. This map projection, based on the logarithm map of the complex plane, preserves shapes locally, and yet is able to display the entire range of astronomical scales from the Earth's neighborhood to the cosmic microwave background. The conformal nature of the projection, preserving shapes locally, may be of particular use for analyzing large scale structure. Prominent in the map is a Sloan Great Wall of galaxies 1.37 billion light years long, 80% longer than the Great Wall discovered by Geller and Huchra and therefore the largest observed structure in the universe.

We describe a method to estimate the mass distribution of a gravitational lens and the position of the sources from combined strong and weak lensing data. The algorithm combines weak and strong lensing data in a unified way producing a solution which is valid in both the weak and strong lensing regimes. We study how the result depends on the relative weighting of the weak and strong lensing data and on choice of basis to represent the mass distribution. We find that combining weak and strong lensing information has two major advantages: it eliminates the need for priors and/or regularization schemes for the intrinsic size of the background galaxies (this assumption was needed in previous strong lensing algorithms) and it corrects for biases in the recovered mass in the outer regions where the strong lensing data is less sensitive. The code is implemented into a software package called WSLAP (Weak & Strong Lensing Analysis Package) which is publicly available at http://darwin.cfa.harvard.edu/SLAP/

We develop a method to constrain the level of non-Gaussianity of density perturbations when the 3-point function is of the "equilateral" type. Departures from Gaussianity of this form are produced by single field models such as ghost or DBI inflation and in general by the presence of higher order derivative operators in the effective Lagrangian of the inflaton. We show that the induced shape of the 3-point function can be very well approximated by a factorizable form, making the analysis practical. We also show that, unless one has a full sky map with uniform noise, in order to saturate the Cramer-Rao bound for the error on the amplitude of the 3-point function, the estimator must contain a piece that is linear in the data. We apply our technique to the WMAP data obtaining a constraint on the amplitude f_NL^equil of "equilateral" non-Gaussianity: -366 < f_NL^equil < 238 at 95% C.L. We also apply our technique to constrain the so-called "local" shape, which is predicted for example by the curvaton and variable decay width models. We show that the inclusion of the linear piece in the estimator improves the constraint over those obtained by the WMAP team, to -27 < f_NL^local < 121 at 95% C.L.

We measure the projected correlation function w_p(r_p) from the Sloan Digital Sky Survey for a flux-limited sample of 118,000 galaxies and for a volume limited subset of 22,000 galaxies with absolute magnitude M_r<-21. Both correlation functions show subtle but systematic departures from the best-fit power law, in particular a change in slope at r_p~1-2 Mpc/h. These departures are stronger for the volume-limited sample, which is restricted to relatively luminous galaxies. We show that the inflection point in w_p(r_p) can be naturally explained by contemporary models of galaxy clustering, according to which it marks the transition from a large scale regime dominated by galaxy pairs in separate dark matter halos to a small scale regime dominated by galaxy pairs in the same dark matter halo. For example, given the dark halo population predicted by an inflationary cold dark matter scenario, the projected correlation function of the volume-limited sample can be well reproduced by a model in which the mean number of M_r<-21 galaxies in a halo of mass M>M_1=4.74 X 10^{13}\msun/h is <N>_M=(M/M_1)^{0.89}, with 75% of the galaxies residing in less massive, single-galaxy halos. This physically motivated model has the same number of free parameters as a power law, and it fits the w_p(r_p) data better, with a \chi^2/d.o.f.=0.93 compared to 6.12 (for 10 degrees of freedom, incorporating the covariance of the correlation function errors). Departures from a power-law correlation function encode information about the relation between galaxies and dark matter halos. Higher precision measurements of these departures for multiple classes of galaxies will constrain galaxy bias and provide new tests of the theory of galaxy formation.

We study the luminosity and color dependence of the galaxy 2-point correlation function in the Sloan Digital Sky Survey, starting from a sample of 200,000 galaxies over 2500 deg^2. We concentrate on the projected correlation function w(r_p), which is directly related to the real space \xi(r). The amplitude of w(r_p) grows continuously with luminosity, rising more steeply above the characteristic luminosity L_*. Redder galaxies exhibit a higher amplitude and steeper correlation function at all luminosities. The correlation amplitude of blue galaxies increases continuously with luminosity, but the luminosity dependence for red galaxies is less regular, with bright red galaxies more strongly clustered at large scales and faint red galaxies more strongly clustered at small scales. We interpret these results using halo occupation distribution (HOD) models assuming concordance cosmological parameters. For most samples, an HOD model with two adjustable parameters fits the w(r_p) data better than a power-law, explaining inflections at r_p ~ 1-3 Mpc/h as the transition between the 1-halo and 2-halo regimes of \xi(r). The implied minimum mass for a halo hosting a central galaxy above a luminosity threshold L grows as M_min ~ L at low luminosities and more steeply above L_*. The mass at which an average halo has one satellite galaxy brighter than L is M_1 ~ 23 M_min(L). These results imply a conditional luminosity function (at fixed halo mass) in which central galaxies lie far above a Schechter function extrapolation of the satellite population. HOD models nicely explain the joint luminosity-color dependence of w(r_p) in terms of the color fractions of central and satellite populations as a function of halo mass. The inferred HOD properties are in good qualitative agreement with theoretical predictions.

How did it all begin? Although this question has undoubtedly lingered for as long as humans have walked the Earth, the answer still eludes us. Yet since my grandparents were born, scientists have been able to refine this question to a degree I find truly remarkable. In this brief essay, I describe some of my own past and ongoing work on this topic, centering on cosmological inflation. I focus on (1) observationally testing whether this picture is correct and (2) working out implications for the nature of physical reality (e.g., the global structure of spacetime, dark energy and our cosmic future, parallel universes and fundamental versus environmental physical laws). (2) clearly requires (1) to determine whether to believe the conclusions. I argue that (1) also requires (2), since it affects the probability calculations for inflation's observational predictions.

We present the BOOMERanG-03 experiment and maps of the Stokes parameters I, Q, U of the microwave sky obtained during a 14 day balloon flight in 2003. Three regions of the southern sky were surveyed: a deep survey (~ 90 square degrees) and a shallow survey (~ 750 square degrees) at high Galactic latitudes (both centered at RA ~ 5.5 h, dec ~ -45 deg) and a survey of ~ 300 square degrees across the Galactic plane at RA ~ 9.1 h, dec ~ -47 deg. All three surveys were carried out in three wide frequency bands centered at 145, 245 and 345 GHz, with an angular resolution of ~ 10'. The 145 GHz maps of Stokes I are dominated by Cosmic Microwave Background (CMB) temperature anisotropy, which is mapped with high signal to noise ratio. The measured anisotropy pattern is consistent with the pattern measured in the same region by BOOMERanG-98 and by WMAP. The 145 GHz maps of Stokes Q and U provide a robust statistical detection of polarization of the CMB when subjected to a power spectrum analysis. This amplitude of the polarization is consistent with that of the CMB in the $\Lambda$CDM cosmological scenario. At 145 GHz, in the CMB surveys, the intensity and polarization of the astrophysical foregrounds are found to be negligible with respect to the cosmological signal. At 245 and 345 GHz we detect ISD emission correlated to the 3000 GHz IRAS/DIRBE maps, and give upper limits for any other non-CMB component. We also present intensity maps of the surveyed section of the Galactic plane. These are compared to monitors of different interstellar components, showing that a variety of emission mechanisms is present in that region.

We report on observations of the Cosmic Microwave Background (CMB) obtained during the January 2003 flight of Boomerang . These results are derived from 195 hours of observation with four 145 GHz Polarization Sensitive Bolometer (PSB) pairs, identical in design to the four 143 GHz Planck HFI polarized pixels. The data include 75 hours of observations distributed over 1.84% of the sky with an additional 120 hours concentrated on the central portion of the field, itself representing 0.22% of the full sky. From these data we derive an estimate of the angular power spectrum of temperature fluctuations of the CMB in 24 bands over the multipole range (50 < l < 1500). A series of features, consistent with those expected from acoustic oscillations in the primordial photon-baryon fluid, are clearly evident in the power spectrum, as is the exponential damping of power on scales smaller than the photon mean free path at the epoch of last scattering (l > 900). As a consistency check, the collaboration has performed two fully independent analyses of the time ordered data, which are found to be in excellent agreement.

We present a measurement of the temperature-polarization angular cross power spectrum, <TE>, of the Cosmic Microwave Background. The result is based on $\sim 200$ hours of data from 8 polarization sensitive bolometers operating at 145 GHz during the 2003 flight of BOOMERANG. We detect a significant <TE> correlation in the $\ell$-range between 50 and 950 with a statistical significance > 3.5 sigma. Contamination by polarized foreground emission and systematic effects are negligible in comparison with statistical uncertainty. The spectrum is consistent with previous detections and with the "concordance model" that assumes adiabatic initial conditions. This is the first measurement of <TE> using bolometric detectors.

We present the cosmological parameters from the CMB intensity and polarization power spectra of the 2003 Antarctic flight of the BOOMERANG telescope. The BOOMERANG data alone constrains the parameters of the $\Lambda$CDM model remarkably well and is consistent with constraints from a multi-experiment combined CMB data set. We add LSS data from the 2dF and SDSS redshift surveys to the combined CMB data set and test several extensions to the standard model including: running of the spectral index, curvature, tensor modes, the effect of massive neutrinos, and an effective equation of state for dark energy. We also include an analysis of constraints to a model which allows a CDM isocurvature admixture.

We report measurements of the CMB polarization power spectra from the January 2003 Antarctic flight of BOOMERANG. The primary results come from six days of observation of a patch covering 0.22% of the sky centered near R.A. = 82.5 deg., Dec= -45 deg. The observations were made using four pairs of polarization sensitive bolometers operating in bands centered at 145 GHz. Using two independent analysis pipelines, we measure a non-zero <EE> signal in the range 100< l <1000 with a significance 4.8-sigma, a 2-sigma upper limit of 8.6 uK^2 for any <BB> contribution, and a 2-sigma upper limit of 7.0 uK^2 for the <EB> spectrum. Estimates of foreground intensity fluctuations and the non-detection of <BB> and <EB> signals rule out any significant contribution from galactic foregrounds. The results are consistent with a Lambda-CDM cosmology seeded by adiabatic perturbations. We note that this is the first detection of CMB polarization with bolometric detectors.

JEDI will probe dark energy in three independent ways by measuring the expansion history of the universe: (1) using type Ia supernovae as cosmological standard candles over a range of distances, (2) using baryon oscillations as a cosmological standard ruler over a range of cosmic epochs, (3) mapping the weak gravitational lensing distortion by foreground galaxies of the images of background galaxies at different distances. JEDI will unravel the nature of dark energy with accuracy and precision. JEDI is a 2m-class space telescope with the unique ability of simultaneous wide-field imaging (0.8-4.2 micron in five bands) and multi-slit spectroscopy (0.8-3.2 micron) with a field of view of 1 square degree. What makes JEDI efficient is its ability to simultaneously obtain high signal-to-noise ratio, moderate resolution slit spectra for all supernovae and ~ 5000 galaxies in its wide field of view, and to combine imaging and spectroscopy so that the appropriate balance of time is devoted to each. JEDI will measure the cosmic expansion history H(z) as a free function to < 2% accuracy in redshift bins of 0.2-0.3. Assuming a flat universe and \sigma(\Omega_m)=0.01 (0.03), JEDI could measure a departure from a vanilla \Lambda CDM model (w_0=-1, w'=0) with \sigma(w_0)=0.013 (0.031) and \sigma(w')=0.046 (0.063). JEDI will obtain the well-sampled lightcurves in Z, J, H, K, L bands and spectra of ~ 14,000 type Ia supernovae with redshifts ranging from 0 to 1.7; the redshifts of ~ 10-100 million galaxies to H ~ 23 and z ~ 4 over 1000-10,000 square degrees; and measurements of the shapes of galaxies over 1000-10,000 square degrees in Z,J,H,K,L for > 10^9 galaxies to H \~ 25.

We present a method for measuring the cosmic expansion history H(z) in uncorrelated redshift bins, and apply it to current and simulated type Ia supernova data assuming spatial flatness. If the matter density parameter Omega_m can be accurately measured from other data, then the dark energy density history X(z)=rho_X(z)/rho_X(0) can trivially be derived from this expansion history H(z). In contrast to customary ``black box'' parameter fitting, our method is transparent and easy to interpret: the measurement of H(z)^{-1} in a redshift bin is simply a linear combination of the measured comoving distances for supernovae in that bin, making it obvious how systematic errors propagate from input to output. We find the Riess et al. (2004) ``gold'' sample to be consistent with the ``vanilla'' concordance model where the dark energy is a cosmological constant. We compare two mission concepts for the NASA/DOE Joint Dark Energy Mission (JDEM), the Joint Efficient Dark-energy Investigation (JEDI), and the Supernova Accelaration Probe (SNAP), using simulated data including the effect of weak lensing (based on numerical simulations) and a systematic bias from K-corrections. Estimating H(z) in seven uncorrelated redshift bins, we find that both provide dramatic improvements over current data: JEDI can measure H(z) to about 10% accuracy and SNAP to 30-40% accuracy.

It is argued that small values of the neutrino masses may be due to anthropic selection effects. If this is the case, then the combined mass of the three neutrino species is expected to be ~1eV, neutrinos causing a non-negligible suppression of galaxy formation.

I briefly review cosmological bounds on neutrino masses and the underlying gravitational physics at a level appropriate for readers outside the field of cosmology. For the case of three massive neutrinos with standard model freezeout, the current 95% upper limit on the sum of their masses is 0.42 eV. I summarize the basic physical mechanism making matter clustering such a sensitive probe of massive neutrinos. I discuss the prospects of doing still better in coming years using tools such as lensing tomography, approaching a sensitivity around 0.03 eV. Since the lower bound from atmospheric neutrino oscillations is around 0.05 eV, upcoming cosmological measurements should detect neutrino mass if the technical and fiscal challenges can be met.

We revisit the issue of non-parametric gravitational lens reconstruction and present a new method to obtain the cluster mass distribution using strong lensing data without using any prior information on the underlying mass. The method relies on the decomposition of the lens plane into individual cells. We show how the problem in this approximation can be expressed as a system of linear equations for which a solution can be found. Moreover, we propose to include information about the null space. That is, make use of the pixels where we know there are no arcs above the sky noise. The only prior information is an estimation of the physical size of the sources. No priors on the luminosity of the cluster or shape of the halos are needed thus making the results very robust. In order to test the accuracy and bias of the method we make use of simulated strong lensing data. We find that the method reproduces accurately both the lens mass and source positions and provide error estimates.

If the inflaton potential has multiple minima, as may be expected in, e.g., the string theory "landscape", inflation predicts a probability distribution for the cosmological parameters describing spatial curvature (Omega_tot), dark energy (rho_Lambda, w, etc.), the primordial density fluctuations (Omega_tot, dark energy (rho_Lambda, w, etc.). We compute this multivariate probability distribution for various classes of single-field slow-roll models, exploring its dependence on the characteristic inflationary energy scales, the shape of the potential V and and the choice of measure underlying the calculation. We find that unless the characteristic scale Delta-phi on which V varies happens to be near the Planck scale, the only aspect of V that matters observationally is the statistical distribution of its peaks and troughs. For all energy scales and plausible measures considered, we obtain the predictions Omega_tot ~ 1+-0.00001, w=-1 and rho_Lambda in the observed ballpark but uncomfortably high. The high energy limit predicts n_s ~ 0.96, dn_s/dlnk ~ -0.0006, r ~ 0.15 and n_t ~ -0.02, consistent with observational data and indistinguishable from eternal phi^2-inflation. The low-energy limit predicts 5 parameters but prefers larger Q and redder n_s than observed. We discuss the coolness problem, the smoothness problem and the pothole paradox, which severely limit the viable class of models and measures. Our findings bode well for detecting an inflationary gravitational wave signature with future CMB polarization experiments, with the arguably best-motivated single-field models favoring the detectable level r ~ 0.03. (Abridged)

Here we present the New York University Value-Added Galaxy Catalog (NYU-VAGC), a catalog of local galaxies (mostly below a redshift of about 0.3) based on a set of publicly-released surveys (including the 2dFGRS, 2MASS, PSCz, FIRST, and RC3) matched to the Sloan Digital Sky Survey (SDSS) Data Release 2. Excluding areas masked by bright stars, the photometric sample covers 3514 square degrees and the spectroscopic sample covers 2627 square degrees (with about 85% completeness). Earlier, proprietary versions of this catalog have formed the basis of many SDSS investigations of the power spectrum, correlation function, and luminosity function of galaxies. We calculate and compile derived quantities (for example, K-corrections and structural parameters for galaxies). The SDSS catalog presented here is photometrically recalibrated, reducing systematic calibration errors across the sky from about 2% to about 1%. We include an explicit description of the geometry of the catalog, including all imaging and targeting information as a function of sky position. Finally, we have performed eyeball quality checks on a large number of objects in the catalog in order to flag deblending and other errors. This catalog is complementary to the SDSS Archive Servers, in that NYU-VAGC's calibration, geometrical description, and conveniently small size are specifically designed for studying galaxy properties and large-scale structure statistics using the SDSS spectroscopic catalog.

We combine the measurements of luminosity dependence of bias with the luminosity dependent weak lensing analysis of dark matter around galaxies to derive the galaxy bias and constrain nonlinear mass and cosmological parameters. We take advantage of theoretical and simulation predictions that predict that while halo bias is rapidly increasing with mass for high masses, it is nearly constant in low mass halos. We use a new weak lensing analysis around the same SDSS galaxies to determine their halo mass probability distribution. These halo mass probability distributions are used to predict the bias for each luminosity subsample and we find an excellent agreement with observed values. The required nonlinear mass suggests slightly lower matter density than usually assumed, Omegam=0.25+/- 0.03 for the simplest models. We combine the bias constraints with those from the WMAP and the SDSS power spectrum analysis to derive new constraints on bias and sigma_8. For the most general parameter space we find sigma_8=0.88+/- 0.06 and b_*=0.99+/- 0.07. In the context of spatially flat models we improve the limit on the neutrino mass for the case of 3 degenerate families from m_nu<0.6eV without bias to m_nu<0.18eV with bias (95% c.l.), which is weakened to m_nu<0.24eV if running is allowed. The corresponding limit for 3 massless + 1 massive neutrino is 1.37eV.

We use the projected correlation function w_p(r_p) of a volume-limited subsample of the Sloan Digital Sky Survey (SDSS) main galaxy redshift catalogue to measure the halo occupation distribution (HOD) of the galaxies of the sample. Simultaneously, we allow the cosmology to vary within cosmological constraints imposed by cosmic microwave background experiments in a Lambda-CDM model. We find that combining w_p(r_p) for this sample alone with the observations by WMAP, ACBAR, CBI and VSA can provide one of the most precise techniques available to measure cosmological parameters. For a minimal flat six-parameter Lambda-CDM model with an HOD with three free parameters, we find Omega_m=0.278^{+0.027}_{-0.027}, sigma_8=0.812^{+0.028}_{-0.027}, and H_0=69.8^{+2.6}_{-2.6}km s^{-1} Mpc^{-1}; these errors are significantly smaller than from CMB alone and similar to those obtained by combining CMB with the large-scale galaxy power spectrum assuming scale-independent bias. The corresponding HOD parameters describing the minimum halo mass and the normalization and cut-off of the satellite mean occupation are M_min=(3.03^{+0.36}_{-0.36})x 10^{12} h^{-1} M_sun, M_1 = (4.58^{+0.60}_{-0.60})x 10^{13} h^{-1} M_sun, and kappa=4.44^{+0.51}_{-0.69}. When more parameters are added to the HOD model, the error bars on the HOD parameters increase because of degeneracies, but the error bars on the cosmological parameters do not increase greatly. Similar modeling for other galaxy samples could reduce the statistical errors on these results, while more thorough investigations of the cosmology dependence of nonlinear halo bias and halo mass functions are needed to eliminate remaining systematic uncertainties, which may be comparable to statistical uncertainties.

We present the large-scale correlation function measured from a spectroscopic sample of 46,748 luminous red galaxies from the Sloan Digital Sky Survey. The survey region covers 0.72 h^{-3} Gpc^3 over 3816 square degrees and 0.16<z<0.47, making it the best sample yet for the study of large-scale structure. We find a well-detected peak in the correlation function at 100h^{-1} Mpc separation that is an excellent match to the predicted shape and location of the imprint of the recombination-epoch acoustic oscillations on the low-redshift clustering of matter. This detection demonstrates the linear growth of structure by gravitational instability between z=1000 and the present and confirms a firm prediction of the standard cosmological theory. The acoustic peak provides a standard ruler by which we can measure the ratio of the distances to z=0.35 and z=1089 to 4% fractional accuracy and the absolute distance to z=0.35 to 5% accuracy. From the overall shape of the correlation function, we measure the matter density Omega_mh^2 to 8% and find agreement with the value from cosmic microwave background (CMB) anisotropies. Independent of the constraints provided by the CMB acoustic scale, we find Omega_m = 0.273 +- 0.025 + 0.123 (1+w_0) + 0.137 Omega_K. Including the CMB acoustic scale, we find that the spatial curvature is Omega_K=-0.010+-0.009 if the dark energy is a cosmological constant. More generally, our results provide a measurement of cosmological distance, and hence an argument for dark energy, based on a geometric method with the same simple physics as the microwave background anisotropies. The standard cosmological model convincingly passes these new and robust tests of its fundamental properties.

Even when completely and consistently formulated, a fundamental theory of physics and cosmological boundary conditions may not give unambiguous and unique predictions for the universe we observe; indeed inflation, string/M theory, and quantum cosmology all arguably suggest that we can observe only one member of an ensemble with diverse properties. How, then, can such theories be tested? It has been variously asserted that in a future measurement we should observe the a priori most probable set of predicted properties (the ``bottom-up'' approach), or the most probable set compatible with all current observations (the ``top-down'' approach), or the most probable set consistent with the existence of observers (the ``anthropic'' approach). These inhabit a spectrum of levels of conditionalization and can lead to qualitatively different predictions. For example, in a context in which the densities of various species of dark matter vary among members of an ensemble of otherwise similar regions, from the top-down or anthropic viewpoints -- but not the bottom-up -- it would be natural for us to observe multiple types of dark matter with similar contributions to the observed dark matter density. In the anthropic approach it is also possible in principle to strengthen this argument and the limit the number of likely dark matter sub-components. In both cases the argument may be extendible to dark energy or primordial density perturbations. This implies that the anthropic approach to cosmology, introduced in part to explain "coincidences" between unrelated constituents of our universe, predicts that more, as-yet-unobserved coincidences should come to light.

We present the mass distribution in the central area of the cluster A1689 by fitting over 100 multiply lensed images with the non-parametric Strong Lensing Analysis Package (SLAP, Diego et al. 2004). The surface mass distribution is obtained in a robust way finding a total mass of 0.25E15 M_sun/h within a 70'' circle radius from the central peak. Our reconstructed density profile fits well an NFW profile with small perturbations due to substructure and is compatible with the more model dependent analysis of Broadhurst et al. (2004a) based on the same data. Our estimated mass does not rely on any prior information about the distribution of dark matter in the cluster. The peak of the mass distribution falls very close to the central cD and there is substructure near the center suggesting that the cluster is not fully relaxed. We also examine the effect on the recovered mass when we include the uncertainties in the redshift of the sources and in the original shape of the sources. Using simulations designed to mimic the data, we identify some biases in our reconstructed mass distribution. We find that the recovered mass is biased toward lower masses beyond 1 arcmin (150 kpc) from the central cD and that in the very center we may be affected by degeneracy problems. On the other hand, we confirm that the reconstructed mass between 25'' and 70'' is a robust, unbiased estimate of the true mass distribution and is compatible with an NFW profile.

We report the intermediate-scale (0.3 to 40 Mpc/h) clustering of 35,000 luminous early-type galaxies at redshifts 0.16 to 0.44 from the Sloan Digital Sky Survey. We present the redshift-space two-point correlation function \xi(s), the projected correlation function w_p(r_p), and the deprojected real-space correlation function \xi(r), for approximately volume-limited samples. As expected, the galaxies are highly clustered, with the correlation length varying from 9.8 +/- 0.2 Mpc/h to 11.2 +/- 0.2 Mpc/h, dependent on the specific luminosity range. For the -23.2 < Mg < -21.2 sample, the inferred bias relative to that of L* galaxies is 1.84 +/- 0.11 for 1 Mpc/h < r_p < 10 Mpc/h, with yet stronger clustering on smaller scales. We detect luminosity-dependent bias within the sample but see no evidence for redshift evolution between z=0.2 and z=0.4. We find a clear indication for deviations from a power-law in the real-space correlation function, with a dip at ~ 2 Mpc/h scales and an upturn on smaller scales. The precision measurements of these clustering trends offer new avenues for the study of the formation and evolution of these massive galaxies.

The DASI discovery of CMB polarization has opened a new chapter in cosmology. Most of the useful information about inflationary gravitational waves and reionization is on large angular scales where Galactic foreground contamination is the worst, so a key challenge is to model, quantify and remove polarized foregrounds. We use the POLAR experiment, COBE/DMR and radio surveys to provide the strongest limits to date on the TE cross power spectrum of the CMB on large angular scales and to quantify the polarized synchrotron radiation, which is likely to be the most challenging polarized contaminant for the MAP satellite. We find that the synchrotron E- and B-contributions are equal to within 10% from 408-820 MHz with a hint of E-domination at higher frequencies. We quantify Faraday Rotation and Depolarization effects in the two-dimensional (l,nu)-plane and show that they cause the synchrotron polarization percentage to drop both towards lower frequencies and towards lower multipoles.

We combine the constraints from the recent Ly-alpha forest and bias analysis of the SDSS with previous constraints from SDSS galaxy clustering, the latest supernovae, and WMAP . Combining WMAP and the lya we find for the primordial slope n_s=0.98\pm 0.02. We see no evidence of running, dn/d\ln k=-0.003\pm 0.010, a factor of 3 improvement over previous constraints. We also find no evidence of tensors, r<0.36 (95% c.l.). A positive correlation between tensors and primordial slope disfavors chaotic inflation type models with steep slopes: V \propto \phi^4 is outside the 3-sigma contour. For the amplitude we find sigma_8=0.90\pm 0.03 from the lyaf and WMAP alone. We find no evidence of neutrino mass: for the case of 3 massive neutrino families with an inflationary prior, \sum m_{\nu}<0.42eV and the mass of lightest neutrino is m_1<0.13eV at 95% c.l. For the 3 massless + 1 massive neutrino case we find m_{\nu}<0.79eV for the massive neutrino, excluding at 95% c.l. all neutrino mass solutions compatible with the LSND results. We explore dark energy constraints in models with a fairly general time dependence of dark energy equation of state, finding Omega_lambda=0.72\pm 0.02, w(z=0.3)=-0.98^{+0.10}_{-0.12}, the latter changing to w(z=0.3)=-0.92^{+0.09}_{-0.10} if tensors are allowed. We find no evidence for variation of the equation of state with redshift, w(z=1)=-1.03^{+0.21}_{-0.28}. These results rely on the current understanding of the lyaf and other probes, which need to be explored further both observationally and theoretically, but extensive tests reveal no evidence of inconsistency among different data sets used here.

Great efforts are currently being channeled into ground- and balloon-based CMB experiments, mainly to explore anisotropy on small angular scales and polarization. To optimize instrumental design and assess experimental prospects, it is important to understand in detail the atmosphere-related systematic errors that limit the science achievable with new suborbital instruments. For this purpose, we spatially compare the 648 square degree ground- and balloon-based QMASK map with the atmosphere-free WMAP map, finding beautiful agreement on all angular scales where both are sensitive. This is a reassuring quantitative assessment of the power of the state-of-the-art FFT- and matrix-based mapmaking techniques that have been used for QMASK and virtually all subsequent experiments.

It is argued that the observed vacuum energy density and the small values of the neutrino masses could be due to anthropic selection effects. Until now, these two quantities have been treated separately from each other and, in particular, anthropic predictions for the vacuum energy were made under the assumption of zero neutrino masses. Here we consider two cases. In the first, we calculate predictions for the vacuum energy for a fixed (generally non-zero) value of the neutrino mass. In the second we allow both quantities to vary from one part of the universe to another. We find that the anthropic predictions for the vacuum energy density are in a better agreement with observations when one allows for non-zero neutrino masses. We also find that the individual distributions for the vacuum energy and the neutrino masses are reasonably robust and do not change drastically when one adds the other variable.

Using the spectacular new high redshift supernova observations from the HST/GOODS program and previous supernova, CMB and galaxy clustering data, we make the most accurate measurements to date of the dark energy density rho_X as a function of cosmic time, constraining it in a rather model-independent way, assuming a flat universe. We find that Einstein's vanilla scenario where rho_X(z) is constant remains consistent with these new tight constraints, and that a Big Crunch or Big Rip is more than 50 gigayears away for a broader class of models allowing such cataclysmic events. We discuss popular pitfalls and hidden priors: parametrizing the equation-of-state w_X(z) assumes positive dark energy density and no Big Crunch, and the popular parametrization w_X(z)=w_0 +w_0' z has nominally strong constraints from CMB merely because w_0' > 0 implies an unphysical exponential blow-up rho_X ~ e^{3 w_0' z}.

We present estimates of cosmological parameters from the application of the Karhunen-Loeve transform to the analysis of the 3D power spectrum of density fluctuations using Sloan Digital Sky Survey galaxy redshifts. We use Omega_m*h and f_b = Omega_b/Omega_m to describe the shape of the power spectrum, sigma8 for the (linearly extrapolated) normalization, and beta to parametrize linear theory redshift space distortions. On scales k < 0.16 h/Mpc, our maximum likelihood values are Omega_m*h = 0.264 +/-0.043, f_b = 0.286 +/- 0.065, sigma8 = 0.966 +/- 0.048, and beta = 0.45 +/- 0.12. When we take a prior on Omega_b from WMAP, we find Omega_m*h = 0.207 +/- 0.030, which is in excellent agreement with WMAP and 2dF. This indicates that we have reasonably measured the gross shape of the power spectrum but we have difficulty breaking the degeneracy between Omega_m*h and f_b because the baryon oscillations are not resolved in the current spectroscopic survey window function.

We measure cosmological parameters using the three-dimensional power spectrum P(k) from over 200,000 galaxies in the Sloan Digital Sky Survey (SDSS) in combination with WMAP and other data. Our results are consistent with a ``vanilla'' flat adiabatic Lambda-CDM model without tilt (n=1), running tilt, tensor modes or massive neutrinos. Adding SDSS information more than halves the WMAP-only error bars on some parameters, tightening 1 sigma constraints on the Hubble parameter from h~0.74+0.18-0.07 to h~0.70+0.04-0.03, on the matter density from Omega_m~0.25+/-0.10 to Omega_m~0.30+/-0.04 (1 sigma) and on neutrino masses from <11 eV to <0.6 eV (95%). SDSS helps even more when dropping prior assumptions about curvature, neutrinos, tensor modes and the equation of state. Our results are in substantial agreement with the joint analysis of WMAP and the 2dF Galaxy Redshift Survey, which is an impressive consistency check with independent redshift survey data and analysis techniques. In this paper, we place particular emphasis on clarifying the physical origin of the constraints, i.e., what we do and do not know when using different data sets and prior assumptions. For instance, dropping the assumption that space is perfectly flat, the WMAP-only constraint on the measured age of the Universe tightens from t0~16.3+2.3-1.8 Gyr to t0~14.1+1.0-0.9 Gyr by adding SDSS and SN Ia data. Including tensors, running tilt, neutrino mass and equation of state in the list of free parameters, many constraints are still quite weak, but future cosmological measurements from SDSS and other sources should allow these to be substantially tightened.

We present a method for ultra-fast confrontation of the WMAP cosmic microwave background observations with theoretical models, implemented as a publicly available software package called CMBfit, useful for anyone wishing to measure cosmological parameters by combining WMAP with other observations. The method takes advantage of the underlying physics by transforming into a set of parameters where the WMAP likelihood surface is accurately fit by the exponential of a quartic or sextic polynomial. Building on previous physics based approximations by Hu et.al., Kosowsky et.al. and Chu et.al., it combines their speed with precision cosmology grade accuracy. A Fortran code for computing the WMAP likelihood for a given set of parameters is provided, pre-calibrated against CMBfast, accurate to Delta lnL ~ 0.05 over the entire 2sigma region of the parameter space for 6 parameter ``vanilla'' Lambda CDM models. We also provide 7-parameter fits including spatial curvature, gravitational waves and a running spectral index.

The WMAP team has produced a foreground map that can account for most of the low-frequency Galactic microwave emission in the WMAP maps, tentatively interpreting it as synchrotron emission. Finkbeiner and collaborators have challenged these conclusions, arguing that the WMAP team "synchrotron" template is in fact not dominated by synchrotron radiation, but by some dust-related Galactic emission process, perhaps spinning dust grains, making dramatically different predictions for its behavior at lower frequencies. By cross-correlating this "synchrotron" template with 10 and 15 GHz CMB observations, we find that its spectrum turns over in a manner consistent with spinning dust emission, falling about an order of magnitude below what the synchrotron interpretation would predict.

In the 1998-99 flight, BOOMERanG has produced maps of $\sim 4 %$ of the sky at high Galactic latitudes, at frequencies of 90, 150, 240 and 410 GHz, with resolution $\simgt 10'$. The faint structure of the Cosmic Microwave Background at horizon and sub-horizon scales is evident in these maps. These maps compare well to the maps recently obtained at lower frequencies by the WMAP experiment. Here we compare the amplitude and morphology of the structures observed in the two sets of maps. We also outline the polarization sensitive version of BOOMERanG, which was flown early this year to measure the linear polarization of the microwave sky at 150, 240 and 350 GHz.

We measure the large-scale real-space power spectrum P(k) using a sample of 205,443 galaxies from the Sloan Digital Sky Survey, covering 2417 square degrees with mean redshift z~0.1. We employ a matrix-based method using pseudo-Karhunen-Loeve eigenmodes, producing uncorrelated minimum-variance measurements in 22 k-bands of both the clustering power and its anisotropy due to redshift-space distortions, with narrow and well-behaved window functions in the range 0.02 h/Mpc < k < 0.3h/Mpc. We pay particular attention to modeling, quantifying and correcting for potential systematic errors, nonlinear redshift distortions and the artificial red-tilt caused by luminosity-dependent bias. Our final result is a measurement of the real-space matter power spectrum P(k) up to an unknown overall multiplicative bias factor. Our calculations suggest that this bias factor is independent of scale to better than a few percent for k<0.1h/Mpc, thereby making our results useful for precision measurements of cosmological parameters in conjunction with data from other experiments such as the WMAP satellite. As a simple characterization of the data, our measurements are well fit by a flat scale-invariant adiabatic cosmological model with h Omega_m =0.201+/- 0.017 and L* galaxy sigma_8=0.89 +/- 0.02 when fixing the baryon fraction Omega_b/Omega_m=0.17 and the Hubble parameter h=0.72; cosmological interpretation is given in a companion paper.

We investigate anomalies reported in the Cosmic Microwave Background maps from the Wilkinson Microwave Anisotropy Probe (WMAP) satellite on very large angular scales and discuss possible interpretations. Three independent anomalies involve the quadrupole and octopole: 1. The cosmic quadrupole on its own is anomalous at the 1-in-20 level by being low (the cut-sky quadrupole measured by the WMAP team is more strikingly low, apparently due to a coincidence in the orientation of our Galaxy of no cosmological significance); 2. The cosmic octopole on its own is anomalous at the 1-in-20 level by being very planar; 3. The alignment between the quadrupole and octopole is anomalous at the 1-in-60 level. Although the a priori chance of all three occurring is 1 in 24000, the multitude of alternative anomalies one could have looked for dilutes the significance of such a posteriori statistics. The simplest small universe model where the universe has toroidal topology with one small dimension of order half the horizon scale, in the direction towards Virgo, could explain the three items above. However, we rule this model out using two topological tests: the S-statistic and the matched circle test.

BOOMERANG is a balloon-borne telescope designed for long duration (LDB) flights around Antarctica. The second LDB Flight of BOOMERANG took place in January 2003. The primary goal of this flight was to measure the polarization of the CMB. The receiver uses polarization sensitive bolometers at 145 GHz. Polarizing grids provide polarization sensitivity at 245 and 345 GHz. We describe the BOOMERANG telescope noting changes made for 2003 LDB flight, and discuss some of the issues involved in the measurement of polarization with bolometers. Lastly, we report on the 2003 flight and provide an estimate of the expected results.

We perform an independent foreground analysis of the WMAP maps to produce a cleaned CMB map (available online) useful for cross-correlation with, e.g., galaxy and X-ray maps. We use a variant of the Tegmark & Efstathiou (1996) technique that is completely blind, making no assumptions about the CMB power spectrum, the foregrounds, WMAP detector noise or external templates. Compared with the foreground-cleaned internal linear combination map produced by the WMAP team, our map has the advantage of containing less non-CMB power (from foregrounds and detector noise) outside the Galactic plane. The difference is most important on the the angular scale of the first acoustic peak and below, since our cleaned map is at the highest (13') rather than lowest (49') WMAP resolution. We also produce a Wiener filtered CMB map, representing our best guess as to what the CMB sky actually looks like, as well as CMB-free maps at the five WMAP frequencies useful for foreground studies. We argue that our CMB map is clean enough that the lowest multipoles can be measured without any galaxy cut, and obtain a quadrupole value that is slightly less low than that from the cut-sky WMAP team analysis. This can be understood from a map of the CMB quadrupole, which shows much of its power falling within the Galaxy cut region, seemingly coincidentally. Intriguingly, both the quadrupole and the octopole are seen to have power suppressed along a particular spatial axis, which lines up between the two, roughly towards (l,b) \~ (-110,60) in Virgo.

We present measurements of the angular cross-correlation between luminous red galaxies from the Sloan Digital Sky Survey and the cosmic microwave background temperature maps from the Wilkinson Microwave Anisotropy Probe. We find a statistically significant achromatic positive correlation between these two data sets, which is consistent with the expected signal from the late Integrated Sachs-Wolfe (ISW) effect. We do not detect any anti-correlation on small angular scales as would be produced from a large Sunyaev-Zel'dovich (SZ) effect, although we do see evidence for some SZ effect for our highest redshift samples. Assuming a flat universe, our preliminary detection of the ISW effect provides independent physical evidence for the existence of dark energy.

This paper presents a scheme to deal accurately and efficiently with complex angular masks, such as occur typically in galaxy surveys. An angular mask is taken to be an arbitrary union of arbitrarily weighted angular regions bounded by arbitrary numbers of edges. The restrictions on the mask are (i) that each edge must be part of some circle on the sphere (but not necessarily a great circle), and (ii) that the weight within each subregion of the mask must be constant. The scheme works by resolving a mask into disjoint polygons, convex angular regions bounded by arbitrary numbers of edges. The polygons may be regarded as the `pixels' of a mask, with the feature that the pixels are allowed to take a rather general shape, rather than following some predefined regular pattern. Among other things, the scheme includes facilities to compute the spherical harmonics of the angular mask, and Data-Random <DR> and Random-Random <RR> angular integrals. A software package mangle which implements this scheme, along with complete software documentation, is available at http://casa.colorado.edu/~ajsh/mangle/ .

The coming flood of CMB polarization experiments, spurred by the recent detection of CMB polarization by DASI and WMAP, will be confronted by many new analysis tasks specific to polarization. For the analysis of CMB polarization data sets, the devil is truly in the details. With this in mind, we present the details of the data analysis for the POLAR experiment, which recently led to the tightest upper limits on the polarization of the Cosmic Microwave Background Radiation at large angular scales. We discuss the data selection process, mapmaking algorithms, offset removal, and the likelihood analysis which were used to find upper limits on the polarization. Stated using the modern convention for reporting CMB Stokes parameters, these limits are 5.0 \uK on both E-type and B-type polarization at 95% confidence. Finally, we discuss simulations used to test our analysis techniques and to probe the fundamental limitations of the experiment.

Understanding the clustering of galaxies has long been a goal of modern observational cosmology. Utilizing our photometric redshift technique a volume limited sample containing more than 2 million galaxies is constructed from the SDSS galaxy catalog. In the largest such analysis to date, we study the angular clustering as a function of luminosity and spectral type. Using Limber's equation we calculate the clustering length for the full data set as r0=5.77+/-0.10 Mpc/h. We find that r0 increases with luminosity by a factor of 1.6 over the sampled luminosity range, in agreement with previous redshift surveys. We also find that both the clustering length and the slope of the correlation function depend on the galaxy type. In particular, by splitting the galaxies in four groups by their rest-frame type we find a bimodal behavior in their clustering properties. Galaxies with spectral types similar to elliptical galaxies have a correlation length of 6.59 +/- 0.17 Mpc/h and a slope of the angular correlation function of 0.96 +/- 0.05 while blue galaxies have a clustering length of 4.51 +/- 0.19 Mpc/h and a slope of 0.68 +/- 0.09. The two intermediate color groups behave like their more extreme 'siblings', rather than showing a gradual transition in slope. We discuss these correlations in the context of current cosmological models for structure formation.

Most of the useful information about inflationary gravitational waves and reionization is on large angular scales where Galactic foreground contamination is the worst, so a key challenge is to model, quantify and remove polarized foregrounds. We use the Leiden radio surveys to quantify the polarized synchrotron radiation at large angular scales, which is likely to be the most challenging polarized contaminant for the WMAP satellite. We find that the synchrotron E- and B-contributions are equal to within 10% from 408-820MHz with a hint of E-domination at higher frequencies. We quantify Faraday Rotation & Depolarization effects and show that they cause the synchrotron polarization percentage to drop both towards lower frequencies and towards lower multipoles.

I survey physics theories involving parallel universes, which form a natural four-level hierarchy of multiverses allowing progressively greater diversity. Level I: A generic prediction of inflation is an infinite ergodic universe, which contains Hubble volumes realizing all initial conditions - including an identical copy of you about 10^{10^29} meters away. Level II: In chaotic inflation, other thermalized regions may have different effective physical constants, dimensionality and particle content. Level III: In unitary quantum mechanics, other branches of the wavefunction add nothing qualitatively new, which is ironic given that this level has historically been the most controversial. Level IV: Other mathematical structures give different fundamental equations of physics. The key question is not whether parallel universes exist (Level I is the uncontroversial cosmological concordance model), but how many levels there are. I discuss how multiverse models can be falsified and argue that there is a severe "measure problem" that must be solved to make testable predictions at levels II-IV.

We report on the linear matter power spectrum reconstructed from the Lyman alpha forest, on baryonic wiggles in the galaxy power spectrum, and on how to measure real space (as opposed to redshift space) power.

Cosmic shear measurements have now improved to the point where they deserve to be treated on par with CMB and galaxy clustering data for cosmological parameter analysis, using the full measured aperture mass variance curve rather than a mere phenomenological parametrization thereof. We perform a detailed 9-parameter analysis of recent lensing (RCS), CMB (up to Archeops) and galaxy clustering (2dF) data, both separately and jointly. CMB and 2dF data are consistent with a simple flat adiabatic scale-invariant model with Omega_Lambda=0.72+/-0.09, omega_cdm=0.115+/- 0.013, omega_b=0.024+/-0.003, and a hint of reionization around z~8. Lensing helps further tighten these constraints, but reveals tension regarding the power spectrum normalization: including the RCS survey results raises sigma8 significantly and forces other parameters to uncomfortable values. Indeed, sigma8 is emerging as the currently most controversial cosmological parameter, and we discuss possible resolutions of this sigma8 problem. We also comment on the disturbing fact that many recent analyses (including this one) obtain error bars smaller than the Fisher matrix bound. We produce a CMB power spectrum combining all existing experiments, and using it for a "MAP versus world" comparison next month will provide a powerful test of how realistic the error estimates have been in the cosmology community.

Despite the interest in dark matter and dark energy, it has never been shown that they are in fact two separate substances. We provide the first strong evidence that they are separate by ruling out a broad class of so-called unified dark matter models that have attracted much recent interest. We find that they produce oscillations or exponential blowup of the matter power spectrum inconsistent with observation. For the particular case of generalized Chaplygin gas models, 99.999% of the previously allowed parameter space is excluded, leaving essentially only the standard Lambda-CDM limit allowed.

Using a catalog of 147,986 galaxy redshifts and fluxes from the Sloan Digital Sky Survey (SDSS) we measure the galaxy luminosity density at z=0.1 in five optical bandpasses corresponding to the SDSS bandpasses shifted to match their restframe shape at z=0.1. We denote the bands {0.1}{u}, {0.1}{g}, {0.1}{r}, {0.1}{i}, {0.1}{z}, with \lambda_{eff} = [3216, 4240, 5595, 6792, 8111] Angstroms respectively. We use a maximum likelihood method which allows for a general form for the shape of the luminosity function, simple luminosity and number evolution, incorporates flux uncertainties, and accounts for the flux limits of the survey. We find luminosity densities at z=0.1 in absolute AB magnitudes in a Mpc^3 of [-14.10 \pm 0.15, -15.18 \pm 0.03, -15.90 \pm 0.03, -16.24 \pm 0.03, -16.56 \pm 0.02] in [{0.1}{u}, {0.1}{g}, {0.1}{r}, {0.1}{i}, {0.1}{z}], respectively, using \Omega_0 =0.3, \Omega_\Lambda=0.7, and h=1, and using Petrosian magnitudes. Similar results are obtained using Sersic model magnitudes, suggesting that flux from outside the Petrosian apertures is not a major correction. In the {0.1}{r} band, the best fit Schechter function to our results has \phi_\ast = (1.49 \pm 0.04) \times 10^{-2} h^3 Mpc^{-3}, M_\ast - 5\log_{10} h = -20.44 \pm 0.01, and \alpha = -1.05\pm 0.01. In solar luminosities, the luminosity density in {0.1}{r} is (1.84 \pm 0.04) h 10^8 L_{{0.1}{r},\odot} Mpc^{-3}. Our results are consistent with other estimates of the luminosity density, from the Two-degree Field Galaxy Redshift Survey and the Millenium Galaxy Catalog. They represent a substantial change (\sim 0.5 mag) from earlier SDSS luminosity density results based on commissioning data, almost entirely because of the inclusion of evolution in the luminosity function model.

We argue for a ``parametrized post-Friedmanian'' approach to linear cosmology, where the history of expansion and perturbation growth is measured without assuming that the Einstein Field Equations hold. As an illustration, a model-independent analysis of 92 type Ia supernovae demonstrates that the curve giving the expansion history has the wrong shape to be explained without some form of dark energy or modified gravity. We discuss how upcoming lensing, galaxy clustering, cosmic microwave background and Lyman alpha forest observations can be combined to pursue this program, which generalizes the quest for a dark energy equation of state, and forecast the accuracy that the proposed SNAP satellite can attain.

We present a method for measuring the cosmic matter budget without assumptions about speculative Early Universe physics, and for measuring the primordial power spectrum P*(k) non-parametrically, either by combining CMB and LSS information or by using CMB polarization. Our method complements currently fashionable ``black box'' cosmological parameter analysis, constraining cosmological models in a more physically intuitive fashion by mapping measurements of CMB, weak lensing and cluster abundance into k-space, where they can be directly compared with each other and with galaxy and Lyman alpha forest clustering. Including the new CBI results, we find that CMB measurements of P(k) overlap with those from 2dF galaxy clustering by over an order of magnitude in scale, and even overlap with weak lensing measurements. We describe how our approach can be used to raise the ambition level beyond cosmological parameter fitting as data improves, testing rather than assuming the underlying physics.

We present the topology of a volume-limited sample of 11,884 galaxies, selected from an apparent-magnitude limited sample of over 100,000 galaxies observed as part of the Sloan Digital Sky Survey (SDSS). The data currently cover three main regions on the sky: one in the Galactic north and one in the south, both at zero degrees declination, and one area in the north at higher declination. Each of these areas covers a wide range of survey longitude but a narrow range of survey latitude, allowing the two dimensional genus to be measured. The genus curves of the SDSS sub-samples are similar, after appropriately normalizing these measurements for the different areas. We sum the genus curves from the three areas to obtain the total genus curve of the SDSS. The total curve has a shape similar to the genus curve derived from mock catalogs drawn from the Hubble Volume Lambda CDM simulation and is similar to that of a Gaussian random field. Likewise, comparison with the genus of the 2dFGRS, after normalization for the difference in area, reveals remarkable similarity in the topology of these samples. We test for the effects of galaxy type segregation by splitting the SDSS data into thirds, based on the u^*- r^* colors of the galaxies, and measure the genus of the reddest and bluest sub-samples. This red/blue split in u^*- r^* is essentially a split by morphology (Strateva et al. 2001). We find that the genus curve for the reddest galaxies exhibits a ``meatball'' shift of the topology -- reflecting the concentration of red galaxies in high density regions -- compared to the bluest galaxies and the full sample, in agreement with predictions from simulations.

Separation of the E and B components of a microwave background polarization map or a weak lensing map is an essential step in extracting science from it, but when the map covers only part of the sky and/or is pixelized, this decomposition cannot be done perfectly. We present a method for decomposing an arbitrary sky map into a sum of three orthogonal components that we term ``pure E'', ``pure B'' and ``ambiguous''. This method is useful both for providing intuition for experimental design and for analyzing data sets in practice. We show how to find orthonormal bases for all three components in terms of bilaplacian eigenfunctions. The number of ambiguous modes is proportional to the length of the map boundary so fairly round maps are preferred. For real-world data analysis, we present a simple matrix eigenvalue method for calculating nearly pure E and B modes in pixelized maps. We find that the dominant source of leakage between E and B is aliasing of small-scale power caused by the pixelization. This problem can be eliminated by heavily oversampling the map, but is exacerbated by the fact that the E power spectrum is expected to be much larger than the B power spectrum and extremely blue. We found that a factor of 2 to 3 more pixels are needed in a polarization map to achieve the same level of contamination by aliased power than in a temperature map.

Nerd abstract: Observational constraints on spacetime are reviewed, focusing on how the underlying physics (dark matter, dark energy, gravity) can be tested rather than assumed. Popular abstract: Space is not a boring static stage on which events unfold over time, but a dynamic entity with curvature, fluctuations and a rich life of its own which is a booming area of study. Spectacular new measurements of the cosmic microwave background, gravitational lensing, type Ia supernovae, large-scale structure, spectra of the Lyman alpha forest, stellar dynamics and x-ray binaries are probing the properties of spacetime over 22 orders of magnitude in scale. Current measurements are consistent with an infinite flat everlasting Universe containing about 30% cold dark matter, 65% dark energy and at least two distinct populations of black holes.

The angular distribution of galaxies encodes a wealth of information about large scale structure. Ultimately, the Sloan Digital Sky Survey (SDSS) will record the angular positions of order 10^8 galaxies in five bands, adding significantly to the cosmological constraints. This is the first in a series of papers analyzing a rectangular stripe 2.5x90 degrees from early SDSS data. We present the angular correlation function for galaxies in four separate magnitude bins on angular scales ranging from 0.003 degrees to 15 degrees. Much of the focus of this paper is on potential systematic effects. We show that the final galaxy catalog -- with the mask accounting for regions of poor seeing, reddening, bright stars, etc. -- is free from external and internal systematic effects for galaxies brighter than r* = 22. Our estimator of the angular correlation function includes the effects of the integral constraint and the mask. The full covariance matrix of errors in these estimates is derived using mock catalogs with further estimates using a number of other methods.

We discuss the tests of nonGaussianity in CMB maps using morphological statistics known as Minkowski functionals. As an example we test degree-scale cosmic microwave background (CMB) anisotropy for Gaussianity by studying the \qmask map that was obtained from combining the QMAP and Saskatoon data. We compute seven morphological functions $M_i(\dt)$, $i=1,...,7$: six \mf and the number of regions $N_c$ at a hundred $\dt$ levels. We also introduce a new parameterization of the morphological functions $M_i(A)$ in terms of the total area $A$ of the excursion set. We show that the latter considerably decorrelates the morphological statistics and makes them more robust because they are less sensitive to the measurements at extreme levels. We compare these results with those from 1000 Gaussian Monte Carlo maps with the same sky coverage, noise properties and power spectrum, and conclude that the \qmask map is neither a very typical nor a very exceptional realization of a Gaussian field. At least about 20% of the 1000 Gaussian Monte Carlo maps differ more than the \qmask map from the mean morphological parameters of the Gaussian fields.

We compute the real-space power spectrum and the redshift-space distortions of galaxies in the 2dF 100k galaxy redshift survey using pseudo-Karhunen-Loeve eigenmodes and the stochastic bias formalism. Our results agree well with those published by the 2dFGRS team, and have the added advantage of producing easy-to-interpret uncorrelated minimum-variance measurements of the galaxy-galaxy, galaxy-velocity and velocity-velocity power spectra in 27 k-bands, with narrow and well-behaved window functions in the range 0.01h/Mpc < k < 0.8h/Mpc. We find no significant detection of baryonic wiggles, although our results are consistent with a standard flat Omega_Lambda=0.7 ``concordance'' model and previous tantalizing hints of baryonic oscillations. We measure the galaxy-matter correlation coefficient r > 0.4 and the redshift-distortion parameter beta=0.49+/-0.16 for r=1 (beta=0.47+/- 0.16 without finger-of-god compression). Since this is an apparent-magnitude limited sample, luminosity-dependent bias may cause a slight red-tilt in the power spectum. A battery of systematic error tests indicate that the survey is not only impressive in size, but also unusually clean, free of systematic errors at the level to which our tests are sensitive. Our measurements and window functions are available at http://www.hep.upenn.edu/~max/2df.html together with the survey mask, radial selection function and uniform subsample of the survey that we have constructed.

We compute upper limits on CMB cross-polarization by cross-correlating the PIQUE and Saskatoon experiments. We also discuss theoretical and practical issues relevant to measuring cross-polarization and illustrate them with simulations of the upcoming BOOMERanG 2002 experiment. We present a method that separates all six polarization power spectra (TT, EE, BB, TE, TB, EB) without any other "leakage" than the familiar EE-BB mixing caused by incomplete sky coverage. Since E and B get mixed, one might expect leakage between TE and TB, between EE and EB and between BB and EB - our method eliminates this by preserving the parity symmetry under which TB and EB are odd and the other four power spectra are even.

We perform a detailed analysis of the latest CMB measurements (including BOOMERaNG, DASI, Maxima and CBI), both alone and jointly with other cosmological data sets involving, e.g., galaxy clustering and the Lyman Alpha Forest. We first address the question of whether the CMB data are internally consistent once calibration and beam uncertainties are taken into account, performing a series of statistical tests. With a few minor caveats, our answer is yes, and we compress all data into a single set of 24 bandpowers with associated covariance matrix and window functions. We then compute joint constraints on the 11 parameters of the ``standard'' adiabatic inflationary cosmological model. Out best fit model passes a series of physical consistency checks and agrees with essentially all currently available cosmological data. In addition to sharp constraints on the cosmic matter budget in good agreement with those of the BOOMERaNG, DASI and Maxima teams, we obtain a heaviest neutrino mass range 0.04-4.2 eV and the sharpest constraints to date on gravity waves which (together with preference for a slight red-tilt) favors ``small-field'' inflation models.

We compute the angular power spectrum C_l from 1.5 million galaxies in early SDSS data on large angular scales, l<600. The data set covers about 160 square degrees, with a characteristic depth of order 1 Gpc/h in the faintest (21<r<22) of our four magnitude bins. Cosmological interpretations of these results are presented in a companion paper by Dodelson et al (2001). The data in all four magnitude bins are consistent with a simple flat ``concordance'' model with nonlinear evolution and linear bias factors of order unity. Nonlinear evolution is particularly evident for the brightest galaxies. A series of tests suggest that systematic errors related to seeing, reddening, etc., are negligible, which bodes well for the sixtyfold larger sample that the SDSS is currently collecting. Uncorrelated error bars and well-behaved window functions make our measurements a convenient starting point for cosmological model fitting.

We measure the cosmic microwave background (CMB) power spectrum on angular scales l~30-200 (1-6 degrees) from the QMASK map, which combines the data from the QMAP and Saskatoon experiments. Since the accuracy of recent measurements leftward of the first acoustic peak is limited by sample-variance, the large area of the QMASK map (648 square degrees) allows us to place among the sharpest constraints to date in this range, in good agreement with BOOMERanG and (on the largest scales) COBE/DMR. By band-pass-filtering the QMAP and Saskatoon maps, we are able to spatially compare them scale-by-scale to check for beam- and pointing-related systematic errors.

We present a new puzzle involving Galactic microwave emission and attempt to resolve it. On one hand, a cross-correlation analysis of the WHAM H-alpha map with the Tenerife 10 and 15 GHz maps shows that the well-known DIRBE correlated microwave emission cannot be dominated by free-free emission. On the other hand, recent high resolution observations in the 8-10 GHz range with the Green Bank 140 ft telescope by Finkbeiner et al. failed to find the corresponding 8 sigma signal that would be expected in the simplest spinning dust models. So what physical mechanism is causing this ubiquitous dust-correlated emission? We argue for a model predicting that spinning dust is the culprit after all, but that the corresponding small grains are well correlated with the larger grains seen at 100 micron only on large angular scales. In support of this grain segregation model, we find the best spinning dust template to involve higher frequency maps in the range 12-60 micron, where emission from transiently heated small grains is important. Upcoming CMB experiments such as ground-based interferometers, MAP and Planck LFI with high resolution at low frequencies should allow a definitive test of this model.

We report a measurement of the real space (not redshift space) power spectrum of galaxies over four and a half decades of wavenumber, 0.01 to 300 h/Mpc, from the IRAS Point Source Catalog Redshift Survey (PSCz). Since estimates of power are highly correlated in the nonlinear regime, we also report results for the prewhitened power spectrum, which is less correlated. The inferred bias between optically-selected APM and IRAS-selected PSCz galaxies is about 1.15 at linear scales < 0.3 h/Mpc, increasing to about 1.4 at nonlinear scales > 1 h/Mpc. The nonlinear power spectrum of PSCz shows a near power-law behaviour to the smallest scales measured, with mild upward curvature in the broad vicinity of 1.5 h/Mpc. Contrary to the prediction of unbiased Dark Matter models, there is no prominent inflection at the linear-nonlinear transition scale, and no turnover at the transition to the stable clustering regime. The nonlinear power spectrum of PSCz requires scale-dependent bias: all Dark Matter models without scale-dependent bias are ruled out with high confidence.

We present measurements of parameters of the 3-dimensional power spectrum of galaxy clustering from 222 square degrees of early imaging data in the Sloan Digital Sky Survey. The projected galaxy distribution on the sky is expanded over a set of Karhunen-Loeve eigenfunctions, which optimize the signal-to-noise ratio in our analysis. A maximum likelihood analysis is used to estimate parameters that set the shape and amplitude of the 3-dimensional power spectrum. Our best estimates are Gamma=0.188 +/- 0.04 and sigma_8L = 0.915 +/- 0.06 (statistical errors only), for a flat Universe with a cosmological constant. We demonstrate that our measurements contain signal from scales at or beyond the peak of the 3D power spectrum. We discuss how the results scale with systematic uncertainties, like the radial selection function. We find that the central values satisfy the analytically estimated scaling relation. We have also explored the effects of evolutionary corrections, various truncations of the KL basis, seeing, sample size and limiting magnitude. We find that the impact of most of these uncertainties stay within the 2-sigma uncertainties of our fiducial result.

We present a method for measuring CMB polarization power spectra given incomplete sky coverage and test it with simulated examples such as Boomerang 2001 and MAP. By augmenting the quadratic estimator method with an additional step, we find that the E and B power spectra can be effectively disentangled on angular scales substantially smaller than the width of the sky patch in the narrowest direction. We find that the basic quadratic and maximum-likelihood methods display a unneccesary sensitivity to systematic errors when T-E cross-correlation is involved, and show how this problem can be eliminated at negligible cost in increased error bars. We also test numerically the widely used approximation that sample variance scales inversely with sky coverage, and find it to be an excellent approximation on scales substantially smaller than the sky patch.

The Sloan Digital Sky Survey is one of the first multicolor photometric and spectroscopic surveys designed to measure the statistical properties of galaxies within the local Universe. In this Letter we present some of the initial results on the angular 2-point correlation function measured from the early SDSS galaxy data. The form of the correlation function, over the magnitude interval 18<r*<22, is shown to be consistent with results from existing wide-field, photographic-based surveys and narrower CCD galaxy surveys. On scales between 1 arcminute and 1 degree the correlation function is well described by a power-law with an exponent of ~ -0.7. The amplitude of the correlation function, within this angular interval, decreases with fainter magnitudes in good agreement with analyses from existing galaxy surveys. There is a characteristic break in the correlation function on scales of approximately 1-2 degrees. On small scales, < 1', the SDSS correlation function does not appear to be consistent with the power-law form fitted to the 1'< theta <0.5 deg data. With a data set that is less than 2% of the full SDSS survey area, we have obtained high precision measurements of the power-law angular correlation function on angular scales 1' < theta < 1 deg, which are robust to systematic uncertainties. Because of the limited area and the highly correlated nature of the error covariance matrix, these initial results do not yet provide a definitive characterization of departures from the power-law form at smaller and larger angles. In the near future, however, the area of the SDSS imaging survey will be sufficient to allow detailed analysis of the small and large scale regimes, measurements of higher-order correlations, and studies of angular clustering as a function of redshift and galaxy type.

Early photometric data from the Sloan Digital Sky Survey (SDSS) contain angular positions for 1.5 million galaxies. In companion papers, the angular correlation function $w(\theta)$ and 2D power spectrum $C_l$ of these galaxies are presented. Here we invert Limber's equation to extract the 3D power spectrum from the angular results. We accomplish this using an estimate of $dn/dz$, the redshift distribution of galaxies in four different magnitude slices in the SDSS photometric catalog. The resulting 3D power spectrum estimates from $w(\theta)$ and $C_l$ agree with each other and with previous estimates over a range in wavenumbers $0.03 < k/{\rm h Mpc}^{-1} < 1$. The galaxies in the faintest magnitude bin ($21 < \rstar < 22$, which have median redshift $z_m=0.43$) are less clustered than the galaxies in the brightest magnitude bin ($18 < \rstar < 19$ with $z_m=0.17$), especially on scales where nonlinearities are important. The derived power spectrum agrees with that of Szalay et al. (2001) who go directly from the raw data to a parametric estimate of the power spectrum. The strongest constraints on the shape parameter $\Gamma$ come from the faintest galaxies (in the magnitude bin $21 < \rstar < 22$), from which we infer $\Gamma = 0.14^{+0.11}_{-0.06}$ (95% C.L.).

We present an upper limit on the polarization of the Cosmic Microwave Background at 7 degree angular scales in the frequency band between 26 and 36 GHz, produced by the POLAR experiment. The campaign produced a map of linear polarization over the R.A. range 112 degrees - 275 degrees at declination 43degrees. The model-independent upper limit on the E-mode polarization component of the CMB at angular scales l = 2 - 20 is 10 microKelvin (95% confidence). The corresponding limit for the B-mode is also 10 microKelvin. Constraining the B-mode power to be zero, the 95% confidence limit on E-mode power alone is 8 microKelvin.

We investigate the frequency of two- and four-image gravitational lens systems in the Jodrell-VLA Astrometric Survey (JVAS) and Cosmic Lens All-Sky Survey (CLASS), and the possible implications for dark matter halo properties. A simple lensing statistics model, which describes lens galaxies as singular isothermal ellipsoids with a projected axis ratio distribution derived from the surface brightness ellipticities of early-type galaxies in the Coma cluster, is ruled out at the 98% level since it predicts too few four-image lenses (quads). We consider a range of factors that may be increasing the frequency of radio quads, including external shear fields, mass distributions flatter than the light, shallow lensing mass profiles, finite core radii, satellite galaxies, and alterations to the luminosity function for faint flat-spectrum radio sources. We find that none of these mechanisms provide a compelling solution to the quad problem on their own while remaining consistent with other observational constraints.

The combined CfA2/SSRS redshift catalog has recently been improved and made public. We compute the redshift-space power spectrum of this catalog using the regions at -2.5 < delta < 50 north (8h < alpha < 17h) and south (20h < alpha < 3h, b < -13) of the Galactic plane, where it is 98% complete down to Zwicky magnitude 15.5 and contains 13,681 galaxies. Our analysis uses Heavens-Taylor mode expansion, Karhunen-Loeve data compression and the Fisher matrix technique to compute quadratic band power estimates. This allows an exact calculation of window functions, including the integral constraint, in addition to the production of a power spectrum with uncorrelated error bars. Our results with this larger data set agree well with previous studies. We analyze 101Mpc/h and 130Mpc/h volume-limited subsets in addition to the full magnitude-limited sample, and our results are well fit by, e.g., Lambda-CDM models with bias b=1.2, 1.4 and 1.4, respectively. We estimate the effect of extinction using the Schlegel, Finkbeiner & Davis dust map. Our results are exclusively for the redshift space galaxy power spectrum, so they can only be compared with theoretical predictions if appropriate corrections are made for biasing and redshift space distortions.

We present results from a first test of the Gaussianity of degree-scale cosmic microwave background (CMB) anisotropy. We investigate Gaussianity of the CMB anisotropy by studying the topology of CMB anisotropy maps from the QMAP and Saskatoon experiments. We also study the QMASK map, a combination map of the QMAP and Saskatoon data. We measure the genus from noise-suppressed Wiener-filtered maps at an angular scale of about 1.5 degrees. To test the Gaussianity of the observed anisotropy, we compare these results to those derived from a collection of simulated maps for each experiment in a Gaussian spatially-flat cosmological constant dominated cold dark matter model. The genus-threshold level relations of the QMAP and Saskatoon maps are consistent with Gaussianity. While the combination QMASK map has a mildly non-Gaussian genus curve which is not a consequence of known foreground contamination, this result is not statistically significant at the 2 sigma level. These results extend previous upper limits on the non-Gaussianity of the large angular scale (> 10 degrees) CMB anisotropy (measured by the COBE DMR experiment) down to degree angular scales.

We present a method for comparing and combining maps with different resolutions and beam shapes, and apply it to the Saskatoon, QMAP and COBE/DMR data sets. Although the Saskatoon and QMAP maps detect signal at the 21 sigma and 40 sigma levels, respectively, their difference is consistent with pure noise, placing strong limits on possible systematic errors. In particular, we obtain quantitative upper limits on relative calibration and pointing errors. Splitting the combined data by frequency shows similar consistency between the Ka- and Q-bands, placing limits on foreground contamination. The visual agreement between the maps is equally striking. Our combined QMAP+Saskatoon map, nicknamed QMASK, is publicly available at www.hep.upenn.edu/~xuyz/qmask.html together with its 6495x6495 noise covariance matrix. This thoroughly tested data set covers a large enough area (648 square degrees -- currently the largest degree-scale map available) to allow a statistical comparison with COBE/DMR, showing good agreement.

We use published measurements of the transmission power spectrum of the Lyman alpha forest to constrain several parameters that describe cosmology and thermal properties of the intergalactic medium (IGM). A 6 parameter grid is constructed using Particle-Mesh dark matter simulations together with scaling relations to make predictions for the gas properties. We fit for all parameters simultaneously and identify several degeneracies. We find that the temperature of the IGM can be well determined from the fall-off of the power spectrum at small scales. We find a temperature around 20.000 K, dependent on the slope of the gas equation of state. We see no evidence for evolution in the IGM temperature. We place constraints on the amplitude of the dark matter fluctuations. However, contrary to previous results, the slope of the dark matter power spectrum is poorly constrained. This is due to uncertainty in the effective Jeans smoothing scale, which depends on the temperature as well as the thermal history of the gas.

We present a method for calculating large numbers of power spectra C_l and P(k) that accelerates CMBfast by a factor around 10^3 without appreciable loss of accuracy, then apply it to constrain 11 cosmological parameters from current Cosmic Microwave Background (CMB) and Large Scale Structure (LSS) data. While the CMB alone still suffers from several degeneracies, allowing, e.g., closed models with strong tilt and tensor contributions, the shape of the real space power spectrum of galaxies from the IRAS Point Source Catalogue Redshift (PSCz) survey breaks these degeneracies and helps place strong constraints on most parameters. At 95% confidence, the combined CMB and LSS data imply a baryon density 0.020 < omega_b < 0.037, dark matter density 0.10 < omega_d < 0.32 with a neutrino fraction f_nu < 38%, vacuum density Omega_Lambda < 0.76, curvature -0.19 < Omega_k < 0.10, scalar tilt 0.86 < n_s < 1.16 and reionization optical depth tau < 0.44. These joint constraints are quite robust, changing little when we impose priors on the Hubble parameter, tilt, flatness, gravity waves or reionization. Adding nucleosynthesis and neutrino priors on the other hand tightens parameters considerably, requiring Omega_Lambda > 0.49 and a red-tilt. The analysis allows a number of consistency tests to be made, all of which pass. At the 95% level, the flat scalar ``concordance model'' with Omega_Lambda=0.62, omega_d=0.13, omega_b=0.02, f_nu~0, n_s=0.9, tau=0.1, h=0.63 is consistent with external constraints. The inferred PSCz bias b~1.2 agrees with the value estimated independently from redshift space distortions. The inferred cosmological constant value agrees with the one independently derived from SN 1a studies. Cosmology seems to be on the right track.

We show that recent measurements of the power spectrum of cosmic microwave background anisotropies by BOOMERanG and MAXIMA can be characterized by four observables, the position of the first acoustic peak l_1= 206 pm 6, the height of the first peak relative to COBE normalization H_1= 7.6 pm 1.4, the height of the second peak relative to the first H_2 = 0.38 pm 0.04, and the height of the third peak relative to the first H_3 = 0.43 pm 0.07. This phenomenological representation of the measurements complements more detailed likelihood analyses in multidimensional parameter space, clarifying the dependence on prior assumptions and the specific aspects of the data leading to the constraints. We illustrate their use in the flat LCDM family of models, where we find Omega_m h^{3.8} > 0.079 (or nearly equivalently, the age of the universe t_0 < 13-14 Gyr) from l_1 and a baryon density Omega_b h^2 > 0.019, a matter density Omega_m h^2 < 0.42 and tilt n>0.85 from the peak heights (95% CL). With the aid of several external constraints, notably nucleosynthesis, the age of the universe and the cluster abundance and baryon fraction, we construct the allowed region in the (Omega_m,h) plane; it points to high h (0.6< h < 0.9) and moderate Omega_m (0.25 < Omega_m < 0.6).

We compute the constraints on a ``standard'' 10 parameter cold dark matter (CDM) model from the most recent CMB and data and other observations, exploring 30 million discrete models and two continuous parameters. Our parameters are the densities of CDM, baryons, neutrinos, vacuum energy and curvature, the reionization optical depth, and the normalization and tilt for both scalar and tensor fluctuations. Our strongest constraints are on spatial curvature, -0.24 < Omega_k < 0.38, and CDM density, h^2 Omega_cdm <0.3, both at 95%. Including SN 1a constraints gives a positive cosmological constant at high significance. We explore the robustness of our results to various assumptions. We find that three different data subsets give qualitatively consistent constraints. Some of the technical issues that have the largest impact are the inclusion of calibration errors, closed models, gravity waves, reionization, nucleosynthesis constraints and 10-dimensional likelihood interpolation.

We compute the joint constraints on ten cosmological parameters from the latest CMB measurements. The lack of a significant second acoustic peak in the latest Boomerang and Maxima data favors models with more baryons than Big Bang nucleosynthesis predicts, almost independently of what prior information is included. The simplest flat inflation models with purely scalar scale-invariant fluctuations prefer a baryon density 0.022 <h^2 Omega_b < 0.040 and a total nonbaryonic (hot + cold) dark matter density 0.14 < h^2 Omega_dm < 0.32 at 95% confidence, and allow reionization no earlier than z~30.

We present a state-of-the-art linear redshift distortion analysis of the recently published IRAS Point Source Catalog Redshift Survey (PSCz). The procedure involves linear compression into 4096 Karhunen-Loeve modes culled from a potential pool of about 3 x 10^5 modes, followed by quadratic compression into three separate power spectra, the galaxy-galaxy, galaxy-velocity, and velocity-velocity power spectra. Least squares fitting to the decorrelated power spectra yields a linear redshift distortion parameter beta = Omega_m^0.6/b = 0.41(+0.13,-0.12).

As experimentalists step up their pursuit of cold dark matter particles and neutrino masses, cosmological constraints are tightening. We compute the joint constraints on 11 cosmological parameters from cosmic microwave background and large scale structure data, and find that at 95% confidence, the total (cold+hot) dark matter density is h^2 Omega_dark = 0.20 +.12 -.10 with at most 38% of it being hot (due to neutrinos). A few assumptions, including negligible neutrinos, tighten this measurement to h^2 Omega_dark = 0.13 +0.04 -0.02, i.e, 2.4e-27 kg/m^3 give or take 20%.

We present Cosmic Microwave Background (CMB) maps from the Santa Barbara HACME balloon experiment (Staren etal 2000), covering about 1150 square degrees split between two regions in the northern sky, near the stars gamma Ursae Minoris and alpha Leonis, respectively. The FWHM of the beam is about 0.77 degrees in three frequency bands centered on 39, 41 and 43 GHz. The results demonstrate that the thoroughly interconnected scan strategy employed allows efficient removal of 1/f-noise and slightly variable scan-synchronous offsets. The maps display no striping, and the noise correlations are found to be virtually isotropic, decaying on an angular scale around one degree. The noise performance of the experiment resulted in an upper limit on CMB anisotropy. However, our results demonstrate that atmospheric contamination and other systematics resulting from the circular scanning strategy can be accurately controlled, and bodes well for the planned follow-up experiments BEAST and ACE, since they show that even with the overly cautious assumption that 1/f-noise and offsets will be as dominant as for HACME, the problems they pose can be readily overcome with the mapmaking algorithm discussed. Our prewhitened notch-filter algorithm for destriping and offset removal should be useful also for other balloon- and ground-based experiments whose scan strategies involve substantial interleaving.

It is shown how to decorrelate the (prewhitened) power spectrum measured from a galaxy survey into a set of high resolution uncorrelated band-powers. The treatment includes nonlinearity, but not redshift distortions. Amongst the infinitely many possible decorrelation matrices, the square root of the Fisher matrix, or a scaled version thereof, offers a particularly good choice, in the sense that the band-power windows are narrow, approximately symmetric, and well-behaved in the presence of noise. We use this method to compute band-power windows for, and the information content of, the Sloan Digital Sky Survey, the Las Campanas Redshift Survey, and the IRAS 1.2 Jy Survey.

We quantify the level of foreground contamination in the QMAP Cosmic Microwave Background (CMB) data with two objectives: (a) measuring the level to which the QMAP power spectrum measurements need to be corrected for foregrounds and (b) using this data set to further refine current foreground models. We cross-correlate the QMAP data with a variety of foreground templates. The 30 GHz Ka-band data is found to be significantly correlated with the Haslam 408 MHz and Reich and Reich 1420 MHz synchrotron maps, but not with the Diffuse Infrared Background Experiment (DIRBE) 240, 140 and 100 micron maps or the Wisconsin H-Alpha Mapper (WHAM) survey. The 40 GHz Q-band has no significant template correlations. We discuss the constraints that this places on synchrotron, free-free and dust emission. We also reanalyze the foreground-cleaned Ka-band data and find that the two band power measurements are lowered by 2.3% and 1.3%, respectively.

We study the prospects for extracting detailed statistical properties of the Sunyaev-Zel'dovich (SZ) effect associated with large scale structure using upcoming multifrequency CMB experiments. The greatest obstacle to detecting the large-angle signal is the confusion noise provided by the primary anisotropies themselves, and to a lesser degree galactic and extragalactic foregrounds. We employ multifrequency subtraction techniques and the latest foregrounds models to determine the detection threshold for the Boomerang, MAP (several microK) and Planck CMB (sub microK) experiments. Calibrating a simplified biased-tracer model of the gas pressure off recent hydrodynamic simulations, we estimate the SZ power spectrum, skewness and bispectrum through analytic scalings and N-body simulations of the dark matter. We show that the Planck satellite should be able to measure the SZ effect with sufficient precision to determine its power spectrum and higher order correlations, e.g. the skewness and bispectrum. Planck should also be able to detect the cross correlation between the SZ and gravitational lensing effect in the CMB. Detection of these effects will help determine the properties of the as yet undetected gas, including the manner in which the gas pressure traces the dark matter.

The recent discovery of dust-correlated diffuse microwave emission has prompted two rival explanations: free-free emission and spinning dust grains. We present new detections of this component at 10 and 15 GHz by the switched-beam Tenerife experiment. The data show a turnover in the spectrum and thereby supports the spinning dust hypothesis. We also present a significant detection of synchrotron radiation at 10 GHz, useful for normalizing foreground contamination of CMB experiments at high-galactic latitudes.

We describe the HEMT Advanced Cosmic Microwave Explorer (HACME), a balloon borne experiment designed to measure sub-degree scale Cosmic Microwave Background anisotropy over hundreds of square degrees, using a unique two dimensional scanning strategy. A spinning flat mirror that is canted relative to its spin axis modulates the direction of beam response in a nearly elliptical path on the sky. The experiment was successfully flown in February of 1996, achieving near laboratory performance for several hours at float altitude. A map free of instrumental systematic effects is produced for a 3.5 hour observation of 630 square degrees, resulting in a flat band power upper limit of (l(l+1)C_l/2 pi)^0.5 < 77 microK at l = 38 (95% confidence). The experiment design, flight operations and data, including atmospheric effects and noise performance, are discussed.

One of the main challenges facing upcoming CMB experiments will be to distinguish the cosmological signal from foreground contamination. We present a comprehensive treatment of this problem and study how foregrounds degrade the accuracy with which the Boomerang, MAP and Planck experiments can measure cosmological parameters. Our foreground model includes not only the normalization, frequency dependence and scale dependence for each physical component, but also variations in frequency dependence across the sky. When estimating how accurately cosmological parameter can be measured, we include the important complication that foreground model parameters (we use about 500) must be simultaneously measured from the data as well. Our results are quite encouraging: despite all these complications, precision measurements of most cosmological parameters are degraded by less than a factor of 2 for our main foreground model and by less than a factor of 5 in our most pessimistic scenario. Parameters measured though large-angle polarization signals suffer more degradation: up to 5 in the main model and 25 in the pessimistic case. The foregrounds that are potentially most damaging and therefore most in need of further study are vibrating dust emission and point sources, especially those in the radio frequencies. It is well-known that E and B polarization contain valuable information about reionization and gravity waves, respectively. However, the cross-correlation between polarized and unpolarized foregrounds also deserves further study, as we find that it carries the bulk of the polarization information about most other cosmological parameters.

Two recent papers have claimed detection of non-Gaussian features in the COBE DMR sky maps of the cosmic microwave background. We confirm these results, but argue that Gaussianity is still not convincingly ruled out. Since a score of non-Gaussianity tests have now been published, one might expect some mildly significant results even by chance. Moreover, in the case of one measure which yields a detection, a bispectrum statistic, we find that if the non-Gaussian feature is real, it may well be due to detector noise rather than a non-Gaussian sky signal, since a signal-to-noise analysis localizes it to angular scales smaller than the beam. We study its spatial origin in case it is nonetheless due to a sky signal (eg, a cosmic string wake or flat-spectrum foreground contaminant). It appears highly localized in the direction b=39.5, l=257, since removing a mere 5 pixels inside a single COBE beam area centered there makes the effect statistically insignificant. We also test Guassianity with an eigenmode analysis which allows a sky map to be treated as a random number generator. A battery of tests of this generator all yield results consistent with Gaussianity.

We show that the galaxy density in the Las Campanas Redshift Survey (LCRS) cannot be perfectly correlated with the underlying mass distribution since various galaxy subpopulations are not perfectly correlated with each other, even taking shot noise into account. This rules out the hypothesis of simple linear biasing, and suggests that the recently proposed stochastic biasing framework is necessary for modeling actual data.

The clustering of galaxies relative to the mass distribution declines with time because: first, nonlinear peaks become less rare events; second, the densest regions stop forming new galaxies because gas there becomes too hot to cool and collapse; third, after galaxies form, they are gravitationally ``debiased'' because their velocity field is the same as the dark matter. To show these effects, we perform a hydrodynamic cosmological simulation and examine the density field of recently formed galaxies as a function of redshift. We find the bias b_* of recently formed galaxies (the ratio of the rms fluctuations of these galaxies and mass), evolves from 4.5 at z=3 to around 1 at z=0, on 8 h^{-1} Mpc comoving scales. The correlation coefficient r_* between recently formed galaxies and mass evolves from 0.9 at z=3 to 0.25 at z=0. As gas in the universe heats up and prevents star formation, star-forming galaxies become poorer tracers of the mass density field. After galaxies form, the linear continuity equation is a good approximation to the gravitational debiasing, even on nonlinear scales. The most interesting observational consequence of the simulations is that the linear regression of the star-formation density field on the galaxy density field evolves from about 0.9 at z=1 to 0.35 at z=0. These effects also provide a possible explanation for the Butcher-Oemler effect, the excess of blue galaxies in clusters at redshift z ~ 0.5. Finally, we examine cluster mass-to-light ratio estimates of Omega, finding that while Omega(z) increases with z, one's estimate Omega_est(z) decreases. (Abridged)

One of the best ways of spotting previously undetected systematic errors in CMB experiments is to compare two independent observations of the same region. We derive a set of tools for comparing and combining CMB data sets, applicable also in the common case where the two have different resolution or beam shape and therefore do not measure the same signal. We present a consistency test that is better than a chi-squared test at detecting systematic errors. We show how two maps of different angular resolution can be combined without smoothing the higher resolution down to the lower one, and generalize this to arbitrary beam configurations. We also show how lossless foreground removal can be performed even for foreground models involving scale dependence, latitude dependence and spectral index variations in combination.

We describe constraints on a ``standard'' 8 parameter open cold dark matter (CDM) model from the most recent CMB and SN1a data. Our parameters are the densities of CDM, baryons, vacuum energy and curvature, the reionization optical depth, and the normalization and tilt for both scalar and tensor fluctuations. We find that although the possibility of reionization and gravity waves substantially weakens the constraints on CDM and baryon density, tilt, Hubble constant and curvature, allowing e.g. a closed Universe, open models with vanishing cosmological constant are still strongly disfavored.

Weak lensing of galaxies by large scale structure can potentially measure cosmological quantities as accurately as the cosmic microwave background (CMB). However, the relation between observables and fundamental parameters is more complex and degenerate, especially in the full space of adiabatic cold dark matter models considered here. We introduce a Fisher matrix analysis of the information contained in weak lensing surveys to address these issues and provide a simple means of estimating how survey properties and source redshift uncertainties affect parameter measurement. We find that surveys on degree scales and above can improve the accuracy on parameters that affect the growth rate of structure by up to an order of magnitude compared to using the CMB alone even if the characteristic redshift of the sources must be determined from the data itself. Surprisingly, both sparse sampling and increasing the source redshift can weaken the cosmological constraints.

We cross-correlate a 19 GHz full sky Cosmic Microwave Background (CMB) survey with other maps to quantify the foreground contribution. Correlations are detected with the Diffuse Infrared Background Experiment (DIRBE) 240, 140 and 100 micron maps at high latitudes (|b|>30degrees), and marginal correlations are detected with the Haslam 408 MHz and the Reich & Reich 1420 MHz synchrotron maps. The former agree well with extrapolations from higher frequencies probed by the COBE DMR and Saskatoon experiments and are consistent with both free-free and rotating dust grain emission.

We study the evolution of the bias factor b and the mass-galaxy correlation coefficient r in a simple analytic model for galaxy formation and the gravitational growth of clustering. The model shows that b and r can be strongly time-dependent, but tend to approach unity even if galaxy formation never ends as the gravitational growth of clustering debiases the older galaxies. The presence of random fluctuations in the sites of galaxy formation relative to the mass distribution can cause large and rapidly falling bias values at high redshift.

It is becoming increasingly clear that galaxy bias is complicated, with evidence indicating that it can be time-dependent, morphology-dependent, scale-dependent, non-linear and non-deterministic. We discuss strategies for overcoming these complications and performing precision cosmological tests with upcoming redshift surveys.

We study the ability of future CMB anisotropy experiments and redshift surveys to constrain a thirteen-dimensional parameterization of the adiabatic cold dark matter model. Each alone is unable to determine all parameters to high accuracy. However, considered together, one data set resolves the difficulties of the other, allowing certain degenerate parameters to be determined with far greater precision. We treat in detail the degeneracies involving the classical cosmological parameters, massive neutrinos, tensor-scalar ratio, bias, and reionization optical depth as well as how redshift surveys can resolve them. We discuss the opportunities for internal and external consistency checks on these measurements. Previous papers on parameter estimation have generally treated smaller parameter spaces; in direct comparisons to these works, we tend to find weaker constraints and suggest numerical explanations for the discrepancies.

We report the results from the second flight of QMAP, an experiment to map the cosmic microwave background near the North Celestial Pole. We present maps of the sky at 31 and 42 GHz as well as a measurement of the angular power spectrum covering the l-range 40-200. Anisotropy is detected at about 20 sigma and is in agreement with previous results at these angular scales. We also report details of the data reduction and analysis techniques which were used for both flights of QMAP.

We report on the first flight of the balloon-borne QMAP experiment. The experiment is designed to make a map of the cosmic microwave background anisotropy on angular scales from 0.7 to several degrees. Using the map we determine the angular power spectrum of the anisotropy in multipole bands from l~40 to l~140. The results are consistent with the Saskatoon (SK) measurements. The frequency spectral index (measured at low l) is consistent with that of CMB and inconsistent with either Galactic synchrotron or free-free emission. The instrument, measurement, analysis of the angular power spectrum, and possible systematic errors are discussed.

We present results from the QMAP balloon experiment, which maps the Cosmic Microwave Background (CMB) and probes its angular power spectrum on degree scales. In two separate flights, data were taken in six channels at two frequency bands between 26 to 46 GHz. We describe our method for mapmaking (removal of 1/f-noise and scan-synchronous offsets) and power spectrum estimation, as well as the results of a joint analysis of the data from both flights. This produces a 527 square degree map of the CMB around the North Celestial Pole, allowing a wide variety of systematic cross-checks. The frequency dependence of the fluctuations is consistent with CMB and inconsistent with Galactic foreground emission. The anisotropy is measured in three multipole bands from l~40 to l~200, and the angular power spectrum shows a distinct rise which is consistent with the Saskatoon results.

For high-precision cosmic microwave background (CMB) experiments, contamination from extragalactic point sources is a major concern. It is therefore useful to be able to detect and discard point source contaminated pixels using the map itself. We show that the sensitivity with which this can be done can often be greatly improved (by factors between 2.5 and 18 for the upcoming Planck mission) by a customized hi-pass filtering that suppresses fluctuations due to CMB and diffuse galactic foregrounds. This means that point source contamination will not severely degrade the cleanest Planck channels unless current source count estimates are off by more than an order of magnitude. A catalog of around 40,000 far infra-red sources at 857 GHz may be a useful by-product of Planck.

Determining the properties of the dark components of the universe remains one of the outstanding challenges in cosmology. We explore how upcoming CMB anisotropy measurements, galaxy power spectrum data, and supernova (SN) distance measurements can observationally constrain their gravitational properties with minimal assumptions on the theoretical side. SN observations currently suggest the existence of dark matter with an exotic equation of state p/rho < -1/3 that accelerates the expansion of the universe. When combined with CMB anisotropy measurements, SN or galaxy survey data can in principle determine the equation of state and density of this component separately, regardless of their value, as long as the universe is spatially flat. Combining these pairs creates a sharp consistency check. If p/rho > -1/2, then the clustering behavior (sound speed) of the dark component can be determined so as to test the scalar-field ``quintessence'' hypothesis. If the exotic matter turns out instead to be simply a cosmological constant (p/rho = -1), the combination of CMB and galaxy survey data should provide a significant detection of the remaining dark matter, the neutrino background radiation (NBR). The gross effect of its density or temperature on the expansion rate is ill-constrained as it is can be mimicked by a change in the matter density. However, anisotropies of the NBR break this degeneracy and should be detectable by upcoming experiments.

We show that galaxy redshift surveys sensitively probe the neutrino mass, with eV mass neutrinos suppressing power by a factor of two. The Sloan Digital Sky Survey can potentially detect $N$ nearly degenerate massive neutrino species with mass m_nu > 0.65 (Omega_m h^2/0.1 N)^{0.8} eV at better than 2sigma once microwave background experiments measure two other cosmological parameters. Significant overlap exists between this region and that implied by the LSND experiment, and even m_nu ~ 0.01-0.1 eV, as implied by the atmospheric anomaly, can affect cosmological measurements.

We show that upcoming CMB satellite experiments and large redshift surveys can be used together to yield 5% determinations of H_0 and Omega_m, an order of magnitude improvement over CMB data alone. CMB anisotropies provide the sound horizon at recombination as a standard ruler. For reasonable baryon fractions, this scale is imprinted on the galaxy power spectrum as a series of spectral features. Measuring these features in redshift space determines the Hubble constant, which in turn yields Omega_m once combined with CMB data. Since the oscillations in both power spectra are frozen in at recombination, this test is insensitive to low-redshift cosmology.

We assess the accuracy with which Omega and Lambda can be measured by combining various types of upcoming experiments. Useful expressions for the Fisher information matrix are derived for classical cosmological tests involving luminosity (eg, SN Ia), angular size, age and number counts. These geometric probes are found to be quite complementary both to each other and to inferences from cluster abundance and the cosmic microwave background (CMB). For instance, a joint analysis of SN Ia and CMB reduces the error bars by about an order of magnitude compared to a separate analysis of either data set.

We compute the accuracy with which Omega and Lambda can be measured by combining future SN Ia and CMB experiments, deriving a handy expression for the SN Ia Fisher information matrix. The two data sets are found to be highly complementary: a joint analysis reduces the error bars by more than an order of magnitude compared to a separate analysis of either data set.

Most work on foreground removal has treated the case where the frequency dependence of all components is perfectly known and independent of position. In contrast, real-world foregrounds are generally not perfectly correlated between frequencies, with the spectral index varying slightly with position and (in the case of some radio sources) with time. A method incorporating this complication in presented, and illustrated with an application to the upcoming satellite missions MAP and Planck. We find that even spectral index variations as small as 0.1 can have a substantial impact on how channels should be combined and on attainable accuracy.

Precision measurements of the galaxy power spectrum P(k) require a data analysis pipeline that is both fast enough to be computationally feasible and accurate enough to take full advantage of high-quality data. We present a rigorous discussion of different methods of power spectrum estimation, with emphasis on the traditional Fourier method, the linear (Karhunen-Loeve; KL), and quadratic data compression schemes, showing in what approximations they give the same result. To improve speed, we show how many of the advantages of KL data compression and power spectrum estimation may be achieved with a computationally faster quadratic method. To improve accuracy, we derive analytic expressions for handling the integral constraint, since it is crucial that finite volume effects are accurately corrected for on scales comparable to the depth of the survey. We also show that for the KL and quadratic techniques, multiple constraints can be included via simple matrix operations, thereby rendering the results less sensitive to galactic extinction and mis-estimates of the radial selection function. We present a data analysis pipeline that we argue does justice to the increases in both quality and quantity of data that upcoming redshift surveys will provide. It uses three analysis techniques in conjunction: a traditional Fourier approach on small scales, a pixelized quadratic matrix method on large scales and a pixelized KL eigenmode analysis to probe anisotropic effects such as redshift-space distortions.

We assess the accuracy with which future galaxy surveys can measure cosmological parameters by deriving a handy approximation that we validate numerically. We find that galaxy surveys are quite complementary to future Cosmic Microwave Background (CMB) experiments. By breaking parameter degeneracies of the Planck CMB satellite in a cold dark matter cosmology, the Sloan Digital Sky Survey might be able to reduce the Planck error bars by about an order of magnitude on the large-scale power normalization and the reionization optical depth, down to percent levels. However, pinpointing attainable accuracies to within better than a factor of a few depends crucially on whether it will be possible to extract useful information from the mildly nonlinear regime.

We explore the qualitative changes that would occur if the amplitude Q ~ 10^{-5} of cosmological density fluctuations were different. If is less than about 10^{-6}, the cosmological objects that form would have so low virial temperatures that they may be unable to cool and form stars, and would be so loosely bound that even if they could produce a supernova explosion, they might be unable to retain the heavy elements necessary for planetary life. If Q is greater than about 10^{-4}, dense supermassive galaxies would form, and biological evolution could be marred by short disruption timescales for planetary orbits. If Q were still larger, most bound systems would collapse directly to supermassive black holes. These constraints on Q can be expressed in terms of fundamental constants alone, and depend only on the electromagnetic and gravitational coupling constants, the electron-proton mass ratio and the matter-to-photon ratio. We discuss the implications for inflation and defect models, and note that the recent anthropic upper bounds on the cosmological constant Lambda would be invalid if both Q and Lambda could vary and there were no anthropic constraints on Q. The same applies to anthropic bounds on the curvature parameter Omega.

Accurate measurements of the cosmic microwave background (CMB) anisotropies with an angular resolution of a few arcminutes can be used to determine fundamental cosmological parameters such as the densities of baryons, cold and hot dark matter, and certain combinations of the cosmological constant and the curvature of the Universe to percent-level precision. Assuming the true theory is a variant of inflationary cold dark matter cosmologies, we calculate the accuracy with which cosmological parameters can be determined by the next generation of CMB satellites, MAP and Planck. We pay special attention to: (a) the accuracy of the computed derivatives of the CMB power spectrum C_L; (b) the number and choices of parameters; (c) the inclusion of prior knowledge of the values of various parameters.

Upcoming large redshift surveys potentially allow precision measurements of the galaxy power spectrum. To accurately measure P(k) on the largest scales, comparable to the depth of the survey, it is crucial that finite volume effects are accurately corrected for in the data analysis. Here we derive analytic expressions for the one such effect that has not previously been worked out exactly: that of the so-called integral constraint. We also show that for data analysis methods based on counts in cells, multiple constraints can be included via simple matrix operations, thereby rendering the results less sensitive to galactic extinction and misestimates of the shape of the radial selection function.

We apply state-of-the art data analysis methods to a number of fictitious CMB mapping experiments, including 1/f noise, distilling the cosmological information from time-ordered data to maps to power spectrum estimates, and find that in all cases, the resulting error bars can we well approximated by simple and intuitive analytic expressions. Using these approximations, we discuss how to maximize the scientific return of CMB mapping experiments given the practical constraints at hand, and our main conclusions are as follows. (1) For a given resolution and sensitivity, it is best to cover a sky area such that the signal-to-noise ratio per resolution element (pixel) is of order unity. (2) It is best to avoid excessively skinny observing regions, narrower than a few degrees. (3) The minimum-variance mapmaking method can reduce the effects of 1/f noise by a substantial factor, but only if the scan pattern is thoroughly interconnected. (4) 1/f noise produces a 1/l contribution to the angular power spectrum for well connected single-beam scanning, as compared to virtually white noise for a two-beam scan pattern such as that of the MAP satellite.

We discuss the problems of applying Maximum Likelihood methods to the CMB and how one can make it both efficient and optimal. The solution is a generalised eigenvalue problem that allows virtually no loss of information about the parameter being estimated, but can allow a substantial compression of the data set. We discuss the more difficult question of simultaneous estimation of many parameters, and propose solutions. A much fuller account of most of this work is available (Tegmark, Taylor & Heavens 1997)

Weak gravitational lensing from large-scale structure enhances and reduces the fluxes from extragalactic point sources with an rms amplitude of order 15%. In cosmic microwave background (CMB) experiments, sources exceeding some flux threshold are removed, which means that lensing will modulate the brightness map of the remaining unresolved sources. Since this mean brightness is of order 100 micro-Kelvin at 30 GHz for a reasonable flux cut, one might be concerned that this modulation could cause substantial problems for future CMB experiments. We present a detailed calculation of this effect and, fortunately, find that its power spectrum is always smaller than the normal point source power spectrum. Thus although this effect should be taken into account when analysing future high-precision CMB measurements, it will not substantially reduce the accuracy with which cosmological parameters can be measured.

The next generation of CMB experiments can measure cosmological parameters with unprecedented accuracy - in principle. To achieve this in practice when faced with such gigantic data sets, elaborate data analysis methods are needed to make it computationally feasible. An important step in the data pipeline is to make a map, which typically reduces the size of the data set my orders of magnitude. We compare ten map-making methods, and find that for the Gaussian case, both the method used by the COBE DMR team and various variants of Wiener filtering are optimal in the sense that the map retains all cosmological information that was present in the time-ordered data (TOD). Specifically, one obtains just as small error bars on cosmological parameters when estimating them from the map as one could have obtained by estimating them directly from the TOD. The method of simply averaging the observations of each pixel (for total-power detectors), on the contrary, is found to generally destroy information, as does the maximum entropy method and most other non-linear map-making techniques. Since it is also numerically feasible, the COBE method is the natural choice for large data sets. Other lossless (e.g. Wiener-filtered) maps can then be computed directly from the COBE method map.

A new method for estimating the angular power spectrum C_l from cosmic microwave background (CMB) maps is presented, which has the following desirable properties: (1) It is unbeatable in the sense that no other method can measure C_l with smaller error bars. (2) It is quadratic, which makes the statistical properties of the measurements easy to compute and use for estimation of cosmological parameters. (3) It is computationally faster than rival high-precision methods such as the nonlinear maximum-likelihood technique, with the crucial steps scaling as n^2 rather than n^3, where n is the number of map pixels. (4) It is applicable to any survey geometry whatsoever, with arbitraty regions masked out and arbitrary noise behaviour. (5) It is not a "black-box" method, but quite simple to understand intuitively: it corresponds to a high-pass filtering and edge softening of the original map followed by a straight expansion in truncated spherical-harmonics. It is argued that this method is computationally feasible even for futute high-resolution CMB experiments with n=10^6-10^7. It is also shown that C_l computed with this method is useful not merely for graphical presentation purposes, but also as an intermediate (and arguably necessary) step in the data analysis pipeline, reducing the data set to a more manageable size before the final step of constraining Gaussian cosmological models and parameters - while retaining all the cosmological information that was present in the original map.

We present a detailed analysis of the cosmic microwave background structure in the Tenerife Dec=+40 degrees data. The effect of local atmospheric contributions on the derived fluctuation amplitude is considered, resulting in an improved separation of the intrinsic CMB signal from noise. Our analysis demonstrates the existence of common structure in independent data scans at 15 and 33 GHz. For the case of fluctuations described by a Gaussian auto-correlation function, a likelihood analysis of our combined results at 15 and 33 GHz implies an intrinsic rms fluctuation level of 48^{+21}_{-15} uK on a coherence scale of 4 degrees; the equivalent analysis for a Harrison-Zel'dovitch model gives a power spectrum normalisation of Q_{rms-ps} = 22^{+10}_{-6} uK. The fluctuation amplitude is seen to be consistent at the 68% confidence level with that reported for the COBE two-year data for primordial fluctuations described by a power law model with a spectral index in the range 1.0 \le n \le 1.6. This limit favours the large scale CMB anisotropy being dominated by scalar fluctuations rather than tensor modes from a gravitational wave background. The large scale Tenerife and COBE results are considered in conjunction with observational results from medium scale experiments in order to place improved limits on the fluctuation spectral index; we find n=1.10 +/- 0.10 assuming standard CDM with H_{0}=50 kms^{-1}Mpc^{-1}.

We describe how to compute estimates of the power spectrum C_l from Cosmic Microwave Background (CMB) maps that not only retain all the cosmological information, but also have uncorrelated error bars and well-behaved window functions. We apply this technique to the 4-year COBE/DMR data, and tabulate eight uncorrelated band-powers.

The minimum mass that a virialized gas cloud must have in order to be able to cool in a Hubble time is computed, using a detailed treatment of the chemistry of molecular hydrogen. With a simple model for halo profiles, we reduce the problem to that of numerically integrating a system of chemical equations. The results agree well with numerically expensive 3D simulations, and our approach has the advantage of rapidly being able to explore large regions of parameter space. The minimum baryonic mass M_b is found to be strongly redshift dependent, dropping from 10^6 solar masses at z=15 to 5000 solar masses at z=100 as molecular cooling becomes effective. For z>>100, M_b rises again, as CMB photons inhibit H_2-formation through the H^- channel. Finally, for z>>200, the H_2^+ channel for H_2-formation becomes effective, driving M_b down towards 10^3 solar masses. With a standard CDM power spectrum with sigma_8=0.7, this implies that a fraction 10^{-3} of all baryons may have formed luminous objects by z=30, which could be sufficient to reheat the universe.

We present a Wiener filtered map of the Cosmic Microwave Background (CMB) fluctuations in a disk with 15 degree diameter, centered at the North Celestial Pole. The map is based on the 1993-1995 data from the Saskatoon (SK) experiment, with an angular resolution around 1 degree in the frequency range 27.6-44.1 GHz. The signal-to-noise ratio in the map is of order two, and some individual hot and cold spots are significant at the 5 sigma level. The spatial features are found to be consistent from year to year, reenforcing the conclusion that the SK results are not dominated by residual atmospheric contamination or other non-celestial signals.

Since cosmology is no longer "the data-starved science", the problem of how to best analyze large data sets has recently received considerable attention, and Karhunen-Loeve eigenvalue methods have been applied to both galaxy redshift surveys and Cosmic Microwave Background (CMB) maps. We present a comprehensive discussion of methods for estimating cosmological parameters from large data sets, which includes the previously published techniques as special cases. We show that both the problem of estimating several parameters jointly and the problem of not knowing the parameters a priori can be readily solved by adding an extra singular value decomposition step. It has recently been argued that the information content in a sky map from a next generation CMB satellite is sufficient to measure key cosmological parameters (h, Omega, Lambda, etc) to an accuracy of a few percent or better - in principle. In practice, the data set is so large that both a brute force likelihood analysis and a direct expansion in signal-to-noise eigenmodes will be computationally unfeasible. We argue that it is likely that a Karhunen-Loeve approach can nonetheless measure the parameters with close to maximal accuracy, if preceded by an appropriate form of quadratic "pre-compression". We also discuss practical issues regarding parameter estimation from present and future galaxy redshift surveys, and illustrate this with a generalized eigenmode analysis of the IRAS 1.2 Jy survey optimized for measuring beta=Omega^{0.6}/b using redshift space distortions.

For power spectrum estimation it's important that the pixelization of a CMB sky map be smooth and regular to high degree. With this criterion in mind the ``COBE sky cube" was defined. This paper has as central theme to further improve on this elegant scheme which uses a cube as projective base -- here an icosahedron is used in its place. Although the sky cube is excellent, a further reduction of 20 percent of the number of pixels can be obtained while the pixel distance is maintained, and without any degradation of accuracy for integration. The pixels are rounder in this scheme, where they are hexagonal rather than square, and the faces are small in this implementation which simplifies area-equalization. The reason distortion is lessened is that the faces are smaller and therefore more flat. To use the method, you can get a FORTRAN code from the Internet.

We generalize previous work on early photoionization to CDM models with Omega<1. Such models have received recent interest because the excess power in the large-scale galaxy distribution is phenomenologically fit if the ``shape parameter" Gamma=h Omega_0 is about 0.25. It has been argued that such models may require early reionization to suppress small-scale anisotropies in order to be consistent with experimental data. We find that if the cosmological constant lambda=0, the extent of this suppression is quite insensitive to Omega_0. Given a sigma_8-normalization today, the loss of small-scale power associated with a lower Omega_0 is partially canceled by higher optical depth from longer lookback times and by structures forming at higher redshifts before the universe becomes curvature-dominated. The maximum angular scale on which fluctuations are suppressed decreases when Omega_0 is lowered, but this effect is also rather weak and unlikely to be measurable in the near future. For flat models, on the other hand, where lambda_0 = 1-Omega_0, the negative effects of lowering Omega_0 dominate, and early reionization is not likely to play a significant role if Omega_0<<1. The same applies to CDM models where the effective Gamma is lowered by increasing the number of relativistic particle species.

Early photoionization of the intergalactic medium is discussed in a nearly model-independent way, in order to investigate whether early structures corresponding to rare Gaussian peaks in a CDM model can photoionize the intergalactic medium sufficiently early to appreciably smooth out the microwave background fluctuations. We conclude that this is indeed possible for a broad range of CDM normalizations and is almost inevitable for unbiased CDM, provided that the bulk of these early structures are quite small, no more massive than about 10^8 solar masses. Typical parameter values predict that reionization occurs around z=50, thereby suppressing fluctuations on degree scales while leaving the larger angular scales probed by COBE reasonably unaffected. However, for non-standard CDM, incorporating mixed dark matter, vacuum density or a tilted primordial power spectrum, early reionization plays no significant role.

We present a new method for reconstructing the cosmological density, peculiar velocity and peculiar gravitational potential on large scales from redshift data. We remove the distorting effects of line-of-sight peculiar motions by using the linear theory of gravitational instability, in which the potential is the solution to a linear partial differential equation first derived by Kaiser. We solve this equation by deriving its Green function; the Green functions for the peculiar velocity and the real-space density follow directly. Reconstruction of the cosmic fields is thus reduced to integration over redshift space with the appropriate Green function kernel. Our algorithm has a single input parameter, beta=Omega^0.6/b, where b is the linear bias factor and Omega is the cosmological density parameter. We discuss the virtues of this method for error control, for estimating beta, and for constraining the bias mechanism.

A model is presented in which supernova-driven winds from early galaxies reionize the intergalactic medium by z=5. This scenario can explain the observed absence of a Gunn-Peterson trough in the spectra of high-redshift quasars providing that the bulk of these early galaxies are quite small, no more massive than about 10^8 solar masses. It also predicts that most of the IGM was enriched to at least 10% of current metal content by z=5 and perhaps as early as z=15. The existence of such early mini-galaxies violates no spectral constraints and is consistent with a pure CDM model with b<2. Since the final radius of a typical ionized bubble is only around 100 kpc, the induced modification of the galaxy autocorrelation function is negligible, as is the induced angular smoothing of the CMBR. Some of the gas swept up by shells may be observable as pressure-supported Lyman-alpha forest clouds.

More than a dozen papers analyzing the COBE data have now appeared. We review the different techniques and compare them to a ``brute force" likelihood analysis where we invert the full 4038 x 4038 Galaxy-cut pixel covariance matrix. This method is optimal in the sense of producing minimal error bars, and is a useful reference point for comparing other analysis techniques. Our maximum-likelihood estimate of the spectral index and normalization are n=1.15 (0.95) and Q=18.2 (21.3) micro-Kelvin including (excluding) the quadrupole. Marginalizing over the normalization C_9, we obtain n=1.10 +/- 0.29 (n=0.90 +/- 0.32). When we compare these results with those of the various techniques that involve a linear ``compression'' of the data, we find that the latter are all consistent with the brute-force analysis and have error bars that are nearly as small as the minimal error bars. We therefore conclude that the data compressions involved in these techniques do indeed retain most of the useful cosmological information.

One still cannot conclusively assert that the universe underwent a neutral phase, despite the new COBE FIRAS limit y<0.000025 on Compton y-distortions of the cosmic microwave background. Although scenarios where the very early (z=1000) ionization is thermal, caused by IGM temperatures exceeding 10000K, are clearly ruled out, there is a significant loophole for cosmologies with typical CDM parameters if the dominant ionization mechanism is photoionization. If the ionizing radiation has a typical quasar spectrum, then the y-constraint implies roughly h^{3/2}\Omega_b Omega_0^{-1/4}<0.1 for fully ionized models. This means that BDM models with Omega_0 around 0.15 and reionization at about z=1000 are strongly constrained even in this very conservative case, and can survive the y test only if most of the baryons form BDM around the reionization epoch.

A method for extracting maximal resolution power spectra from microwave sky maps is presented and applied to the 2 year COBE data, yielding a power spectrum that is consistent with a standard n=1, Q=20 micro-Kelvin model. By using weight functions that fall off smoothly near the galactic cut, it is found that the spectral resolution \Delta l can be more than doubled at l=15 and more than tripled at l=20 compared to simply using galaxy-cut spherical harmonics. For a future high-resolution experiment with reasonable sky coverage, the resolution around the CDM Doppler peaks would be enhanced by a factor of about 100, down to \Delta l\approx 1, thus allowing spectral features such as the locations of the peaks to be determined with great accuracy. The reason that the improvement is so large is basically that functions with a sharp edge at the galaxy cut exhibit considerable "ringing" in the Fourier domain, whereas smooth functions do not. The method presented here is applicable to any survey geometry, chopping strategy and exposure pattern whatsoever. The so called signal-to-noise eigenfunction technique is found to be a special case, corresponding to ignoring the width of the window functions.

The angular power spectrum C_l is extracted from the 4 year COBE DMR data with a 20 degree galactic cut, using the narrowest window functions possible. The average power in eight multipole bands is also computed, and plotted together with a compilation of power spectrum measurements from other experiments. The COBE results are found to be consistent with an n=1 power spectrum with the normalization Qrmsps=18 microkelvin reported by the COBE DMR team. Certain non-standard cosmologies, such as "small universe" models with nontrivial spatial topology, predict power spectra which are not smooth functions. Rather, they contain bumps and wiggles that may not have been detected by other data analysis techniques such as the Hauser-Peebles method (Wright etal 1996), the band-power method (Hinshaw etal 1996) and orthogonalized spherical harmonics (Gorski etal 1996), since these methods all give broader window functions that can smear such features out beyond recognition. On the large angular scales probed by COBE, the Universe thus appears to be kind to us, presenting a power spectrum that is a simple smooth function.

An improved method for subtracting contaminants from Cosmic Microwave Background (CMB) sky maps is presented, and used to estimate how well future experiments will be able to recover the primordial CMB fluctuations. We find that the naive method of subtracting foregrounds (such as dust emission, synchrotron radiation, free-free-emission, unresolved point sources, etc) on a pixel by pixel basis can be improved by more than an order of magnitude by taking advantage of the correlation of the emission in neighboring pixels. The optimal multi-frequency subtraction method improves on simple pixel-by-pixel subtraction both by taking noise-levels into account, and by exploiting the fact that most contaminants have angular power spectra that differ substantially from that of the CMB. The results are natural to visualize in the two-dimensional plane with axes defined by multipole l and frequency v. We present a brief overview of the geography of this plane, showing the regions probed by various experiments and where we expect contaminants to dominate. We illustrate the method by estimating how well the proposed ESA COBRAS/SAMBA mission will be able to recover the CMB fluctuations against contaminating foregrounds.

We compute the angular power spectrum C_l of the BATSE 3B catalog, and find no evidence for clustering on any scale. These constraints bridge the entire range from small scales, probing source clustering and repetition, to large scales constraining possible Galactic anisotropies, or those from nearby cosmological large scale structures.

The power spectrum of fluctuations in the cosmic microwave background (CMB) depends on most of the key cosmological parameters. Accurate future measurements of this power spectrum might therefore allow us to determine h, Omega, Omega_b, Lambda, n, T/S, etc, with hitherto unprecedented accuracy. In these lecture notes, which are intended to be readable without much prior CMB knowledge, I review the various physical processes that generate CMB fluctuations, focusing on how changes in the parameters alters the shape of the power spectrum. I also discuss foregrounds and real-world data analysis issues and how these affect the accuracy with which the parameters can be measured.

The power spectrum estimated from a galaxy redshift survey is the real spectrum convolved with a window function, so when estimating the power on very large scales (for small k), it is important that this window function be as narrow as possible. A method that achieves this is presented. The optimal estimate of P(k) is found to be the Fourier transform of the number density fluctuations (n/nbar - 1) weighted by a function psi_0, and the optimal psi_0 is found to be the ground-state solution of the Schoedinger equation, with the inverse selection function as the potential. This quantum mechanics analogy occurs basically because we want the weight function to be narrow both in Fourier space (to give a narrow window) and in real space (to minimize the variance from shot noise). An optimal method for averaging the estimates at different k-vectors is also presented, generalizing the standard procedure of averaging over shells in k-space. Finally, a discrete version of the method is presented, dividing space into ``fuzzy pixels", which has the advantage of being able to handle redshift distortions in a straightforward way.

We tighten previous upper limits on gamma ray burst repetition by analyzing the angular power spectrum of the BATSE 3B catalog of 1122 bursts. At 95% confidence, we find that no more than 2% of all observed bursts can be labeled as repeaters, even if no sources are observed to repeat more than once. If a fraction f of all observed bursts can be labeled as repeaters that are observed to burst v times each, then all models with (v-1)f>0.05 are ruled out at 99% confidence, as compared to the best previous 99% limit (v-1)f>0.27. At 95% confidence, our new limit is (v-1)f>0.02. Thus even a cluster of 6 events from a single source would have caused excess power above that present in the 3B catalog. We conclude that the current BATSE data are consistent with no repetition of classical gamma ray bursts, and that any repeater model is severely constrained by the near perfect isotropy of their angular distribution.

We compute the angular power spectrum C_l from the BATSE 3B catalog of 1122 gamma-ray bursts, and find no evidence for clustering on any scale. These constraints bridge the entire range from small scales (which probe source clustering and burst repetition) to the largest scales (which constrain possible anisotropies from the Galactic halo or from nearby cosmological large scale structures). We develop an analysis technique that takes the angular position errors into account, which enables us to place tight upper limits on the clustering down to scales l\approx 60, corresponding to a few degrees on the sky. The minimum-variance burst weighting that we employ is graphically visualized as an all-sky map where each burst is smeared out by an amount corresponding to its position uncertainty. We also present separate band-pass filtered sky maps for the quadrupole term and for the multipole-ranges l=3-10 and l=11-30, so that the fluctuations on different angular scales can be separately inspected for visual features such as localized ``hot spots" or structures aligned with the Galactic plane. These filtered maps reveal no apparent deviations from isotropy.

We have investigated the possibility of inferring peculiar velocities for clusters of galaxies from the Doppler shift of scattered cosmic microwave background (CMB) photons. We find that if the core radius of the gas distribution or the beam size of the instrument is larger than 3-7 arcminutes, then the maximum attainable signal-to-noise ratio is determined by confusion with primary fluctuations. For smaller angular scales, ``cosmic confusion'' is less important and instrumental noise and/or foreground emission will be the limiting factor. For a cluster with the optical depth of the Coma cluster and for an optimal filtering technique, typical one-sigma errors span the wide range from 400 to 1600 km/s, depending on the cosmological model, the resolution of the instrument and the core radius of the cluster. The results have important implications for the design of future high-resolution surveys of the CMB. Individual peculiar velocities will be measurable only for a few fast moving clusters at intermediate redshift unless cosmic fluctuations are smaller than most standard cosmological scenarios predict. However, a reliable measurement of bulk velocities of ensembles of X-ray bright clusters will be possible on very large scales (100-500 Mpc/h).

A new method for estimating the power spectrum $C_l$ from cosmic microwave background (CMB) maps was recently presented by the author and applied to the 2 year COBE data. It was found that the spectral resolution $\Delta l$ for COBE could be more than doubled at $l=15$, thereby revealing previously unresolved features in the power spectrum. Whereas that paper was rather technical, the present paper shows that all qualitative features of this method can be understood from a simple analogy with quantum mechanics.

A formalism is presented that allows cosmological experiments to be tested for consistency, and allows a simple frequentist interpretation of the resulting significance levels. As an example of an application, this formalism is used to place constraints on bulk flows of galaxies using the results of the microwave background anisotropy experiments COBE and SP91, and a few simplifying approximations about the experimental window functions. It is found that if taken at face value, with the quoted errors, the recent detection by Lauer and Postman of a bulk flow of 689 km/s on scales of 150$h^{-1}$Mpc is inconsistent with SP91 at a 95\% confidence level within the framework of a Cold Dark Matter (CDM) model. The same consistency test is also used to place constraints that are completely model-independent, in the sense that they hold for any power spectrum whatsoever --- the only assumption being that the random fields are Gaussian. It is shown that the resulting infinite-dimensional optimization problem reduces to a set of coupled non-linear equations that can readily be solved numerically. Applying this technique to the above-mentioned example, we find that the Lauer and Postman result is inconsistent with SP91 even if no assumptions whatsoever are made about the power spectrum.