This is the second paper in a series where we propose a method of indirectly measuring large scale structure using information from small scale perturbations. The idea is to build a quadratic estimator from small scale modes that provides a map of structure on large scales. We demonstrated in the first paper that the quadratic estimator works well on a dark-matter-only N-body simulation at a snapshot of $z = 0$. Here we generalize the theory to the case of a light-cone with halo bias and redshift space distortions taken into consideration. We successfully apply the generalized version of the quadratic estimator to a light cone halo catalog of an N-body simulation of size $\sim5.6\,(h^{-1}\,\rm Gpc)^3$. The most distant point in the light cone is at a redshift of $1.4$, which indicates that we might be able to apply our method to next generation galaxy surveys.

In this work, we explore the application of intensity mapping to detect extended Ly$\alpha$ emission from the IGM via cross-correlation of PAUS images with Ly$\alpha$ forest data from eBOSS and DESI. Seven narrow-band (FWHM=13nm) PAUS filters have been considered, ranging from 455 to 515 nm in steps of 10 nm, which allows the observation of Ly$\alpha$ emission in a range $2.7<z<3.3$. The cross-correlation is simulated first in an area of 100 deg$^2$ (PAUS projected coverage), and second in two hypothetical scenarios: a deeper PAUS (complete up to $i_{\rm AB}<24$ instead of $i_{\rm AB}<23$, observation time x6), and an extended PAUS coverage of 225 deg$^2$ (observation time x2.25), both cross-correlated only with DESI. A hydrodynamic simulation of size 400 Mpc/h has been used to simulate both extended Ly$\alpha$ emission and absorption in the Ly$\alpha$ forest, while the foregrounds in PAUS images have been simulated using a lightcone mock catalogue. The total probability of detection is estimated to be 23% and 33% for PAUS-eBOSS and PAUS-DESI respectively, from a run of 1000 simulated cross-correlations with different realisations of instrumental noise and quasar positions. The hypothetical PAUS scenarios increase this probability to 58% (deeper PAUS) and 60% (extended PAUS), with the extension of angular coverage being far more time-efficient for Ly$\alpha$ intensity mapping than the increase of exposure time. These findings seem to indicate that this methodology could also be applied to broad-band surveys.

We demonstrate a method for reconstructing the weak lensing potential from the Lyman-$\alpha$ forest data. We derive an optimal estimator for the lensing potential on the sky based on the correlation between pixels in real space. This method effectively deals with irregularly spaced data, holes in the survey, missing data and inhomogeneous noise. We demonstrate an implementation of the method with simulated spectra and weak lensing. It is shown that with a source density of $>\sim 0.5$ per square arcminutes and $\sim 200$ pixels in each spectrum ($\lambda / \Delta\lambda = 1300$) the lensing potential can be reconstructed with high fidelity if the relative absorption in the spectral pixels is signal dominated. When noise dominates the measurement of the absorption in each pixel the noise in the lensing potential is higher, but for reasonable numbers of sources and noise levels and a high fidelity map the lensing potential is obtainable. The lensing estimator could also be applied to lensing of the Cosmic Microwave Background (CMB), 21 cm intensity mapping (IM) or any case in which the correlation function of the source can be accurately estimated.

We investigate the possibility that a statistical detection of the galaxy parallax shifts due to the Earth's motion with respect to the CMB frame (cosmic secular parallax) could be made by the Vera Rubin Observatory or by the WFIRST satellite, and used to measure the Hubble constant. We make mock galaxy surveys which extend to redshift z=0.06 from a large N-body simulation, and include astrometric errors from the LSST and WFIRST science requirements, redshift errors and peculiar velocities. We include spectroscopic redshifts for the brightest galaxies (r < 18) in the fiducial case. We use these catalogues to make measurements of parallax versus redshift, for various assumed survey parameters and analysis techniques. We find that in order to make a competitive measurement it will be necessary to model and correct for the peculiar velocity component of galaxy proper motions. It will also be necessary to push astrometry of extended sources into a new regime, and combine information from the different elements of resolved galaxies. For our fiducial survey parameters, we predict an rms error on the direct geometrical measurement of H0 of 2.8 % for LSST and 0.8 % for WFIRST.

Large scale density modes are difficult to measure because they are sensitive to systematic observational errors in galaxy surveys, but we can study them indirectly by observing their impact on small scale perturbations. Cosmological perturbation theory predicts that second-order density inhomogeneities are a convolution of a short- and a long-wavelength mode. This arises physically because small scale structures grow at different rates depending on the large scale environment in which they reside. This induces an off-diagonal term in the two-point statistics in Fourier space that we use as the basis for a quadratic estimator for the large scale field. We demonstrate that this quadratic estimator works well on an N-body simulation of size (2.5 h^{-1} Gpc)^3. In particular, the quadratic estimator successfully reconstructs the long-wavelength modes using only small-scale information. This opens up novel opportunities to study structure on the largest observable scales.

Nebular emission lines associated with galactic HII regions carry information about both physical properties of the ionised gas and the source of ionising photons as well as providing the opportunity of measuring accurate redshifts and thus distances once a cosmological model is assumed. While nebular line emission has been extensively studied at lower redshift there are currently only few constraints within the epoch of reionisation (EoR, $z>6$), chiefly due to the lack of sensitive near-IR spectrographs. However, this will soon change with the arrival of the Webb Telescope providing sensitive near-IR spectroscopy covering the rest-frame UV and optical emission of galaxies in the EoR. In anticipation of Webb we combine the large cosmological hydrodynamical simulation Bluetides with photoionisation modelling to predict the nebular emission line properties of galaxies at $z=8\to 13$. We find good agreement with the, albeit limited, existing direct and indirect observational constraints on equivalent widths though poorer agreement with luminosity function constraints.

The Wide Field Infrared Survey Telescope (WFIRST), with its field of view and high sensitivity will make surveys of cosmological large-scale structure possible at high redshifts. We investigate the possibility of detecting Baryon Acoustic Oscillations (BAO) at redshifts $z>7.6$ for use as a standard ruler. We use data from the hydrodynamic simulation \textsc{BlueTides} in conjunction with the gigaparsec-scale Outer Rim simulation and a model for patchy reionization to create mock WFIRST High Latitude Survey grism data for \lya\ emission line selected galaxies at redshifts $z=7.4$ to $z=10$, covering 2280 square degrees. We measure the monopoles of galaxies in the mock catalogues and fit the BAO features. We find that for a line flux of $L = 7\times 10^{-17} \ {\rm erg/s/cm}^{2}$, the $5 \sigma$ detection limit for the current design, the BAO feature is partially detectable (measured in three out of four survey quadrants analysed independently). The resulting root mean square error on the angular diameter distance to $z=7.7$ is 7.9$\%$. If we improve the detection sensitivity by a factor of two (i.e. $L = 3.5\times 10^{-17} \ {\rm erg/s/cm}^{2}$), the distance error reduces to $1.4\%$. We caution that many more factors are yet to be modelled, including dust obscuration, the damping wing due to the intergalactic medium, and low redshift interlopers. If these issues do not strongly affect the results, or different observational techniques (such as use of multiple lines) can mitigate them, WFIRST or similar instruments may be able to constrain the angular diameter distance to the high redshift Universe.

Trend filtering---first introduced into the astronomical literature in Paper I of this series---is a state-of-the-art statistical tool for denoising one-dimensional signals that possess varying degrees of smoothness. In this work, we demonstrate the broad utility of trend filtering to observational astronomy by discussing how it can contribute to a variety of spectroscopic and time-domain studies. The observations we discuss are (1) the Lyman-$\alpha$ forest of quasar spectra; (2) more general spectroscopy of quasars, galaxies, and stars; (3) stellar light curves with planetary transits; (4) eclipsing binary light curves; and (5) supernova light curves. We study the Lyman-$\alpha$ forest in the greatest detail---using trend filtering to map the large-scale structure of the intergalactic medium along quasar-observer lines of sight. The remaining studies share broad themes of: (1) estimating observable parameters of light curves and spectra; and (2) constructing observational spectral/light-curve templates. We also briefly discuss the utility of trend filtering as a tool for one-dimensional data reduction and compression.

The problem of denoising a one-dimensional signal possessing varying degrees of smoothness is ubiquitous in time-domain astronomy and astronomical spectroscopy. For example, in the time domain, an astronomical object may exhibit a smoothly varying intensity that is occasionally interrupted by abrupt dips or spikes. Likewise, in the spectroscopic setting, a noiseless spectrum typically contains intervals of relative smoothness mixed with localized higher frequency components such as emission peaks and absorption lines. In this work, we present trend filtering, a modern nonparametric statistical tool that yields significant improvements in this broad problem space of denoising $spatially$ $heterogeneous$ signals. When the underlying signal is spatially heterogeneous, trend filtering is superior to any statistical estimator that is a linear combination of the observed data---including kernel smoothers, LOESS, smoothing splines, Gaussian process regression, and many other popular methods. Furthermore, the trend filtering estimate can be computed with practical and scalable efficiency via a specialized convex optimization algorithm, e.g. handling sample sizes of $n\gtrsim10^7$ within a few minutes. In a companion paper, we explicitly demonstrate the broad utility of trend filtering to observational astronomy by carrying out a diverse set of spectroscopic and time-domain analyses.

We present an approach for automatic extraction of measured values from the astrophysical literature, using the Hubble constant for our pilot study. Our rules-based model -- a classical technique in natural language processing -- has successfully extracted 298 measurements of the Hubble constant, with uncertainties, from the 208,541 available arXiv astrophysics papers. We have also created an artificial neural network classifier to identify papers in arXiv which report novel measurements. From the analysis of our results we find that reporting measurements with uncertainties and the correct units is critical information when distinguishing novel measurements in free text. Our results correctly highlight the current tension for measurements of the Hubble constant and recover the $3.5\sigma$ discrepancy -- demonstrating that the tool presented in this paper is useful for meta-studies of astrophysical measurements from a large number of publications.

High-$z$ AGNs hosted in gas rich galaxies are expected to grow through significantly obscured accretion phases. This may limit or bias their observability. In this work, we use \textsc{BlueTides}, a large volume cosmological simulation of galaxy formation to examine quasar obscuration for the highest-redshift ($z \geq 7$) supermassive black holes residing in the center of galaxies. We find that for the bright quasars, most of the high column density gas ($>90\%$) resides in the innermost regions of the host galaxy, (typically within $< 10$ ckpc), while the gas in the outskirts is a minor contributor to the $N_\mathrm H$. The brightest quasars can have large angular variations in galactic obscuration, over 2 orders of magnitude, where the lines of sight with the lowest obscuration are those formed via strong gas outflows driven by AGN feedback. We find that for the overall AGN population, the mean $N_\mathrm H$ is generally larger for high luminosity and BH mass, while the $N_\mathrm H$ distribution is significantly broadened, developing a low $N_\mathrm H $ wing due to the angular variations driven by the AGN outflows/feedback. The obscured fraction P($N_{\rm H} > 10^{23} {\rm cm}^{-2}$) typically range from 0.6 to 1.0 for increasing $L_{X}$ (with $L_X > 10^{43} \rm{ergs/s}$), with no clear trend of redshift evolution. With respect to the galaxy host property, we find a linear relation between $N_{\rm H}$, $M_*$ and $M_{\rm H_2}$ with $\log N_{\rm H} = (0.24 \pm 0.03) \log M_{*} + (20.7 \pm 0.3)$ and $\log N_{\rm H} = (0.47 \pm 0.03) \log M_{\rm H_2} + (18.4 \pm 0.3)$. The dust optical depth in the UV band $\tau_{\mathrm UV}$ has tight positive correlation with $N_{\rm H}$. Our dust extincted UVLF is about 1.5 dex lower than the intrinsic UVLF, implying that more than 99\% of the $z \sim 7$ AGNs are heavily dust extincted and therefore would be missed by the UV band observation.

We examine the properties of the host galaxies of $z=7$ quasars using the large volume, cosmological hydrodynamical simulation BlueTides. We find that the most massive black holes and quasars are hosted by massive galaxies with stellar masses $\log(M_\ast/M_\odot)=10.8\pm0.2$, and $10.2\pm0.4$, which have a wide range of star formation rates, of $50\substack{+119 \\ -35}M_\odot/\rm{yr}$ and $19\substack{+28 \\ -12}M_\odot/\rm{yr}$, respectively. The hosts of the most massive black holes and quasars in BlueTides are generally bulge-dominated, with bulge-to-total mass ratio $B/T\simeq0.85\pm0.1$, however their morphologies are not biased relative to the overall $z=7$ galaxy sample. We find that the hosts of the most massive black holes and quasars are significantly more compact, with half-mass radii $R_{0.5}=0.29\substack{+0.15 \\ -0.10}$ kpc and $0.28\substack{+0.08 \\ -0.06}$ kpc respectively, relative to galaxies with similar masses, which have $R_{0.5}=0.5\substack{+0.3 \\ -0.2}$ kpc. We make mock James Webb Space Telescope (JWST) images of these quasars and their host galaxies. We find that distinguishing the host from the quasar emission will be possible but still challenging with JWST, due to the small sizes of quasar hosts. We find that currently observable quasar samples are biased tracers of the intrinsic black hole--stellar mass relations, following a relation that is 0.27 dex higher than that of the full galaxy sample. Finally, we find that black hole hosts are generally quite isolated. However, the most massive black holes are more likely to be found in denser environments than the typical $M_{\textrm{BH}}>10^{6.5}M_\odot$ black hole, indicating that minor mergers at least play some role in growing black holes in the early Universe.

The expansion of the Universe is understood to have accelerated during two epochs: in its very first moments during a period of Inflation and much more recently, at $z < 1$, when Dark Energy is hypothesized to drive cosmic acceleration. The undiscovered mechanisms behind these two epochs represent some of the most important open problems in fundamental physics. The large cosmological volume at $2 < z < 5$, together with the ability to efficiently target high-$z$ galaxies with known techniques, enables large gains in the study of Inflation and Dark Energy. A future spectroscopic survey can test the Gaussianity of the initial conditions up to a factor of ~50 better than our current bounds, crossing the crucial theoretical threshold of $\sigma(f_{NL}^{\rm local})$ of order unity that separates single field and multi-field models. Simultaneously, it can measure the fraction of Dark Energy at the percent level up to $z = 5$, thus serving as an unprecedented test of the standard model and opening up a tremendous discovery space.

Astrophysical observations currently provide the only robust, empirical measurements of dark matter. In the coming decade, astrophysical observations will guide other experimental efforts, while simultaneously probing unique regions of dark matter parameter space. This white paper summarizes astrophysical observations that can constrain the fundamental physics of dark matter in the era of LSST. We describe how astrophysical observations will inform our understanding of the fundamental properties of dark matter, such as particle mass, self-interaction strength, non-gravitational interactions with the Standard Model, and compact object abundances. Additionally, we highlight theoretical work and experimental/observational facilities that will complement LSST to strengthen our understanding of the fundamental characteristics of dark matter.

The gravitational potential well of an $M>10^{13}$ $M_\odot$ galaxy will lead to a gravitational redshift differential of order 1 km/s between the galaxy core and its outskirts. Current surveys of massive galaxies with spatially resolved spectroscopy have reached a size which makes feasible attempts to detect gravitational redshifts within galaxies. We use spectra from the Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) experiment to attempt a measurement of the averaged stellar redshift profile of large elliptical galaxies. We find that systematic effects (possibly related to charge transfer or wavelength calibration accuracy) make the standard MaNGA data pipeline unsuitable for measuring the relevant sub km/s wavelength shifts. We therefore develop a cross-correlation technique to mitigate these effects, but caution that we are working beyond the design accuracy of the MaNGA experiment. With a sample of $272$ galaxies in halos with $\log (M/M_\odot)>13$, we attempt a measurement of the gravitational redshift profile, achieving $1 \sigma$ errors of size $\sim 0.5$ km/s, but are unable to make a significant detection of the signal. Even without a detection, our measurement can be used to limit the dark matter mass within the half light radius of elliptical galaxies to $1.2$ times the stellar mass, at the 68% confidence level. We also perform weighting scheme tests and split sample tests, and address target selection issues and other relativistic effects, including the transverse Doppler effect and relativistic beaming of stars. Future detections could lead to new constraints on the galaxy mass distribution that are different from kinematic and lensing determinations and open a window on galaxy properties and tests of gravity.

Gravitational redshifts and other relativistic effects are beginning to be studied in the context of galaxy clustering. Distortions consistent with those expected in General Relativity have been measured in galaxy cluster redshift profiles by Wojtak et al. and others and in the the cross-correlation function of galaxy populations by Alam et al. On scales below ~20 Mpc/$h$ simulations have shown that gravitational redshift dominates over other effects. However, this signal is related to the shape and depth of gravitational potentials, and therefore the matter density in galaxies and galaxy clusters that is responsible for them. We investigate the effects of baryonic physics on the gravitational redshift profiles of massive (group and cluster-sized) halos. We compare the profiles of different components in halos taken from the MassiveBlack-II cosmological hydrodynamic simulation and a dark matter-only version of the same simulation. We find that inclusion of baryons, cooling, star formation and feedback significantly alters the relevant inner density profiles. These baryonic effects lead to overall increases in both gravitational redshifts and the transverse relativistic Doppler effects by up to ~50% for group size halos. We show how modified Navarro Frenk White halo profiles can be used to parametrize these differences, and provide relevant halo profile fits.

Marked correlation functions, which are sensitive to the clustering of galaxies in different environments, have been proposed as constraints on modified gravity models. We present measurements of the marked correlation functions of galaxies in redshift space using 361,761 LOWZ ($z_{\rm eff} = 0.32$) galaxies from the Sloan Digital Sky Survey III (SDSS III) Baryon Oscillation Spectroscopic Survey (BOSS) Data Release 12 (DR12) and compare them to $\Lambda$CDM+General Relativity simulations. We apply mass cuts to find the best match between the redshift space autocorrelation function of subhaloes in the simulation and in the observations. We then compare the marked correlation functions, finding no significant evidence for deviations of the marked correlation functions of LOWZ galaxies from $\Lambda$CDM on scales $6 \ h^{-1}$Mpc $ \leq s \leq$ $69 \ h^{-1}$Mpc. The constraining power of marked correlation functions in our analysis is limited by our ability to model the autocorrelation function of galaxies on small scales including the effect of redshift distortions. The statistical errors are well below the differences seen between marked correlation functions of $f$(R) gravity models and $\Lambda$CDM in recent publications (Armijo et al., Hern\'{a}ndez-Aguayo et al.) indicating that improved future theoretical analyses should be able to rule out some models definitively.

We investigate the large-scale structure of Lyman-alpha emission intensity in the Universe at redshifts z=2-3.5 using cross-correlation techniques. Our Lya emission samples are spectra of BOSS Luminous Red Galaxies from Data Release 12 with the best fit model galaxies subtracted. We cross-correlate the residual flux in these spectra with BOSS quasars, and detect a positive signal on scales 1-15 Mpc/h. We identify and remove a source of contamination not previously accounted for, due to the effects of quasar clustering on cross-fibre light. Corrected, our quasar-Lya emission cross-correlation is 50 % lower than that seen by Croft et al. for DR10, but still significant. Because only 3% of space is within 15 Mpc/h of a quasar, the result does not fully explore the global large-scale structure of Lya emission. To do this, we cross-correlate with the Lya forest. We find no signal in this case. The 95% upper limit on the global Lya mean surface brightness from Lya emission-Lya forest cross-correlation is mu < 1.2x10^-22 erg/s/cm^2/A/arcsec^2 This null result rules out the scenario where the observed quasar-Lya emission cross-correlation is primarily due to the large scale structure of star forming galaxies, Taken in combination, our results suggest that Lya emitting galaxies contribute, but quasars dominate within 15 Mpc/h. A simple model for Lya emission from quasars based on hydrodynamic simulations reproduces both the observed forest-Lya emission and quasar-Lya emission signals. The latter is also consistent with extrapolation of observations of fluorescent emission from smaller scales r < 1 Mpc.

The most distant known quasar recently discovered by Ba\~nados et al. (2018) is at $z=7.5$ (690 Myr after the Big Bang), at the dawn of galaxy formation. We explore the host galaxy of the brightest quasar in the large volume cosmological hydrodynamic simulation BlueTides, which in Phase II has reached these redshifts. The brightest quasar in BlueTides has a luminosity of a $\sim$ few $10^{13} L_{\odot}$ and a black hole mass of $6.4 \times 10^{8} M_{\odot}$ at $z \sim 7.5$, comparable to the observed quasar (the only one in this large volume). The quasar resides in a rare halo of mass $M_{H} \sim 10^{12} M_{\odot}$ and has a host galaxy of stellar mass of $4 \times 10^{10}M_{\odot}$ with an ongoing (intrinsic) star formation rate of $\sim 80 M_{\odot} yr^{-1}$. The corresponding intrinsic UV magnitude of the galaxy is $-23.1$, which is roughly $2.7$ magnitudes fainter than the quasar's magnitude of $-25.9$. We find that the galaxy is highly metal enriched with a mean metallicity equal to the solar value. We derive quasar and galaxy spectral energy distribution (SED) in the mid and near infrared JWST bands. We predict a significant amount of dust attenuation in the rest-frame UV corresponding to $A_{1500} \sim 1.7$ giving an UV based SFR of $\sim 14 M_{\odot} yr^{-1}$. We present mock JWST images of the galaxy with and without central point source, in different MIRI and NIRCam filters. The host galaxy is detectable in NIRCam filters, but it is extremely compact ($R_{E}=0.35$ kpc). It will require JWST's exquisite sensitivity and resolution to separate the galaxy from the central point source. Finally within the FOV of the quasar in BlueTides there are two more sources that would be detectable by JWST.

Many theoretical models predict that quasar driven outflows account for the observed quenching of star formation in massive galaxies. There is growing observational evidence for quasar-launched massive outflows at z>6, while the details of outflow-host galaxy interaction remain obscure. In this paper, we study the feedback around the highest redshift quasar in the BlueTides simulation, the largest volume cosmological hydrodynamic simulation so far carried out. We present predictions for gas outflows around the brightest $z = 7.54$ quasar which hosts the most massive black hole in the simulation volume, which has grown to black hole mass $6.7\times 10^{8}{\rm M}_\odot$ consistent with the current record holder for high-z quasars. We introduce a method to identify and trace the gas outflowing from the halo. We find that the total mass of the outflow gas is about $3.6\times 10^{9}{\rm M}_\odot$, constituting 6\% of the total gas in the halo. The outflow gas contains a cold, dense molecular component with mass about $2.6\times 10^{8}{\rm M}_\odot$, that originates from the inner region of the halo, within a few kpc of the central black hole. The velocities of the outflow gas reach thousands of km/s, within which the molecular component has mass averaged outward radial velocity of $1300$ km/s, consistent with observations. The averaged outflow rate is about $200-300 {\rm M}_\odot/yr$, with the outflowing gas mainly in a hotter ($T \sim 10^7$ K) and lower density state than the average of the host halo gas.

In La Plante et al. (2017), we presented a new suite of hydrodynamic simulations with the aim of accurately capturing the process of helium II reionization. In this paper, we discuss the observational signatures present in the He II Ly$\alpha$ forest. We show that the effective optical depth of the volume $\tau_\mathrm{eff}$ is not sufficient for capturing the ionization state of helium II, due to the large variance inherent in sightlines. However, the He II flux PDF can be used to determine the timing of helium II reionization. The amplitude of the one-dimensional flux power spectrum can also determine the ionization state of helium II. We show that even given the currently limited number of observations ($\sim$50 sightlines), measurements of the flux PDF can yield information about helium II reionization. Further, measurements using the one-dimensional power spectrum can provide clear indications of the timing of reionization, as well as the relative bias of sources of ionizing radiation.

We have run a new suite of simulations that solve hydrodynamics and radiative transfer simultaneously to study helium II reionization. Our suite of simulations employs various models for populating quasars inside of dark matter halos, which affect the He II reionization history. In particular, we are able to explore the impact that differences in the timing and duration of reionization have on observables. We examine the thermal signature that reionization leaves on the IGM, and measure the temperature-density relation. As previous studies have shown, we confirm that the photoheating feedback from helium II reionization raises the temperature of the IGM by several thousand kelvin. To compare against observations, we generate synthetic Ly$\alpha$ forest sightlines on-the-fly and match the observed effective optical depth $\tau_{\mathrm{eff}}(z)$ of hydrogen to recent observations. We show that when the simulations have been normalized to have the same values of $\tau_\mathrm{eff}$, the effect that helium II reionization has on observations of the hydrogen Ly$\alpha$ forest is minimal. Specifically, the flux PDF and the one-dimensional power spectrum are sensitive to the thermal state of the IGM, but do not show direct evidence for the ionization state of helium. We show that the peak temperature of the IGM typically corresponds to the time of 90%-95% helium ionization by volume, and is a relatively robust indicator of the timing of reionization. Future observations of helium reionization from the hydrogen Ly$\alpha$ forest should thus focus on measuring the temperature of the IGM, especially at mean density. Detecting the peak in the IGM temperature would provide valuable information about the timing of the end of helium II reionization.

Motivated by recent observational constraints on dust reprocessed emission in star forming galaxies at $z\sim 6$ and above we use the very-large cosmological hydrodynamical simulation \bluetides\ to explore predictions for the amount of dust obscured star formation in the early Universe ($z>8$). \bluetides\ matches current observational constraints on both the UV luminosity function and galaxy stellar mass function and predicts that approximately $90\%$ of the star formation in high-mass ($M_{*}>10^{10}\,{\rm M_{\odot}}$) galaxies at $z=8$ is already obscured by dust. The relationship between dust attenuation and stellar mass predicted by \bluetides\ is consistent with that observed at lower redshift. However, observations of several individual objects at $z>6$ are discrepant with the predictions, though it is possible their uncertainties may have been underestimated. We find that the predicted surface density of $z\ge 8$ sub-mm sources is below that accessible to current {\em Herschel}, SCUBA-2, and ALMA sub-mm surveys. However, as ALMA continues to accrue additional surface area the population of $z>8$ dust-obscured galaxies may become accessible in the near future.

Following the first two annual intensity mapping workshops at Stanford in March 2016 and Johns Hopkins in June 2017, we report on the recent advances in theory, instrumentation and observation that were presented in these meetings and some of the opportunities and challenges that were identified looking forward. With preliminary detections of CO, [CII], Lya and low-redshift 21cm, and a host of experiments set to go online in the next few years, the field is rapidly progressing on all fronts, with great anticipation for a flood of new exciting results. This current snapshot provides an efficient reference for experts in related fields and a useful resource for nonspecialists. We begin by introducing the concept of line-intensity mapping and then discuss the broad array of science goals that will be enabled, ranging from the history of star formation, reionization and galaxy evolution to measuring baryon acoustic oscillations at high redshift and constraining theories of dark matter, modified gravity and dark energy. After reviewing the first detections reported to date, we survey the experimental landscape, presenting the parameters and capabilities of relevant instruments such as COMAP, mmIMe, AIM-CO, CCAT-p, TIME, CONCERTO, CHIME, HIRAX, HERA, STARFIRE, MeerKAT/SKA and SPHEREx. Finally, we describe recent theoretical advances: different approaches to modeling line luminosity functions, several techniques to separate the desired signal from foregrounds, statistical methods to analyze the data, and frameworks to generate realistic intensity map simulations.

The fourth generation of the Sloan Digital Sky Survey (SDSS-IV) began observations in July 2014. It pursues three core programs: APOGEE-2, MaNGA, and eBOSS. In addition, eBOSS contains two major subprograms: TDSS and SPIDERS. This paper describes the first data release from SDSS-IV, Data Release 13 (DR13), which contains new data, reanalysis of existing data sets and, like all SDSS data releases, is inclusive of previously released data. DR13 makes publicly available 1390 spatially resolved integral field unit observations of nearby galaxies from MaNGA, the first data released from this survey. It includes new observations from eBOSS, completing SEQUELS. In addition to targeting galaxies and quasars, SEQUELS also targeted variability-selected objects from TDSS and X-ray selected objects from SPIDERS. DR13 includes new reductions of the SDSS-III BOSS data, improving the spectrophotometric calibration and redshift classification. DR13 releases new reductions of the APOGEE-1 data from SDSS-III, with abundances of elements not previously included and improved stellar parameters for dwarf stars and cooler stars. For the SDSS imaging data, DR13 provides new, more robust and precise photometric calibrations. Several value-added catalogs are being released in tandem with DR13, in particular target catalogs relevant for eBOSS, TDSS, and SPIDERS, and an updated red-clump catalog for APOGEE. This paper describes the location and format of the data now publicly available, as well as providing references to the important technical papers that describe the targeting, observing, and data reduction. The SDSS website, http://www.sdss.org, provides links to the data, tutorials and examples of data access, and extensive documentation of the reduction and analysis procedures. DR13 is the first of a scheduled set that will contain new data and analyses from the planned ~6-year operations of SDSS-IV.

General relativistic effects have long been predicted to subtly influence the observed large-scale structure of the universe. The current generation of galaxy redshift surveys have reached a size where detection of such effects is becoming feasible. In this paper, we report the first detection of the redshift asymmetry from the cross-correlation function of two galaxy populations which is consistent with relativistic effects. The dataset is taken from the Sloan Digital Sky Survey DR12 CMASS galaxy sample, and we detect the asymmetry at the $2.7\sigma$ level by applying a shell-averaged estimator to the cross-correlation function. Our measurement dominates at scales around $10$ h$^{-1}$Mpc, larger than those over which the gravitational redshift profile has been recently measured in galaxy clusters, but smaller than scales for which linear perturbation theory is likely to be accurate. The detection significance varies by 0.5$\sigma$ with the details of our measurement and tests for systematic effects. We have also devised two null tests to check for various survey systematics and show that both results are consistent with the null hypothesis. We measure the dipole moment of the cross-correlation function, and from this the asymmetry is also detected, at the $2.8 \sigma$ level. The amplitude and scale-dependence of the clustering asymmetries are approximately consistent with the expectations of General Relativity and a biased galaxy population, within large uncertainties. We explore theoretical predictions using numerical simulations in a companion paper.

Large redshift surveys of galaxies and clusters are providing the first opportunities to search for distortions in the observed pattern of large-scale structure due to such effects as gravitational redshift. We focus on non-linear scales and apply a quasi-Newtonian approach using N-body simulations to predict the small asymmetries in the cross-correlation function of two galaxy different populations. Following recent work by Bonvin et al., Zhao and Peacock and Kaiser on galaxy clusters, we include effects which enter at the same order as gravitational redshift: the transverse Doppler effect, light-cone effects, relativistic beaming, luminosity distance perturbation and wide-angle effects. We find that all these effects cause asymmetries in the cross-correlation functions. Quantifying these asymmetries, we find that the total effect is dominated by the gravitational redshift and luminosity distance perturbation at small and large scales, respectively. By adding additional subresolution modelling of galaxy structure to the large-scale structure information, we find that the signal is significantly increased, indicating that structure on the smallest scales is important and should be included. We report on comparison of our simulation results with measurements from the SDSS/BOSS galaxy redshift survey in a companion paper.

In a galaxy redshift survey the objects to be targeted for spectra are selected from a photometrically observed sample. The observed magnitudes and colours of galaxies in this parent sample will be affected by their peculiar velocities, through relativistic Doppler and relativistic beaming effects. In this paper we compute the resulting expected changes in galaxy photometry. The magnitudes of the relativistic effects are a function of redshift, stellar mass, galaxy velocity and velocity direction. We focus on the CMASS sample from the Sloan Digital Sky Survey (SDSS), Baryon Oscillation Spectroscopic Survey (BOSS), which is selected on the basis of colour and magnitude. We find that 0.10\% of the sample ($\sim 585$ galaxies) has been scattered into the targeted region of colour-magnitude space by relativistic effects, and conversely 0.09\% of the sample ($\sim 532$ galaxies) has been scattered out. Observational consequences of these effects include an asymmetry in clustering statistics, which we explore in a companion paper. Here we compute a set of weights which can be used to remove the effect of modulations introduced into the density field inferred from a galaxy sample. We conclude by investigating the possible effects of these relativistic modulation on large scale clustering of the galaxy sample.

We study the asymmetry in the two-point cross-correlation function of two populations of galaxies focusing in particular on the relativistic effects that include the gravitational redshift. We derive the cross-correlation function on small and large scales using two different approaches: General Relativistic and Newtonian perturbation theory. Following recent work by Bonvin et al., Gaztanaga et al. and Croft, we calculate the dipole and the shell estimator with the two procedures and we compare our results. We find that while General Relativistic Perturbation Theory (GRPT) is able to make predictions of relativistic effects on very large, obviously linear scales (r > 50 Mpc/h), the presence of non-linearities physically occurring on much smaller scales (down to those describing galactic potential wells) can strongly affect the asymmetry estimators. These can lead to cancellations of the relativistic terms, and sign changes in the estimators on scales up to r ~ 50 Mpc/h. On the other hand, with an appropriate non-linear gravitational potential, the results obtained using Newtonian theory can successfully describe the asymmetry on smaller, non-linear scales (r < 20 Mpc/h) where gravitational redshift is the dominant term. On larger scales the asymmetry is much smaller in magnitude, and measurement is not within reach of current observations. This is in agreement with the observational results obtained by Gaztnaga et al. and the first detection of relativistic effects (on (r < 20 Mpc/h) scales) by Alam et al.

Supermassive blackholes with masses of a billion solar masses or more are known to exist up to $z=7$. However, the present-day environments of the descendants of first quasars is not well understood and it is not known if they live in massive galaxy clusters or more isolated galaxies at $z=0$. We use a dark matter-only realization (BTMassTracer) of the BlueTides cosmological hydrodynamic simulation to study the halo properties of the descendants of the most massive black holes at $z=8$. We find that the descendants of the quasars with most massive black holes are not amongst the most massive halos. They reside in halos of with group-like ($\sim 10^{14}M_{\odot}$) masses, while the most massive halos in the simulations are rich clusters with masses $\sim 10^{15} M_{\odot}$. The distribution of halo masses at low redshift is similar to that of the descendants of least massive black holes, for a similar range of halo masses at $z=8$, which indicates that they are likely to exist in similar environments. By tracing back to the $z = 8$ progenitors of the most massive (cluster sized) halos at $z=0$; we find that their most likely black hole mass is less than $10^7 M_{\odot}$; they are clearly not amongst the most massive black holes. We also provide estimates for the likelihood of finding a high redshift quasar hosting a black hole with masses above $10^{7} M_{\odot}$ for a given halo mass at $z=0$. For halos above $10^{15} M_{\odot}$, there is only $20 \%$ probability that their $z=8$ progenitors hosted a black hole with mass above $10^{7} M_{\odot}$.

We describe the Sloan Digital Sky Survey IV (SDSS-IV), a project encompassing three major spectroscopic programs. The Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2) is observing hundreds of thousands of Milky Way stars at high resolution and high signal-to-noise ratio in the near-infrared. The Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey is obtaining spatially-resolved spectroscopy for thousands of nearby galaxies (median redshift of z = 0.03). The extended Baryon Oscillation Spectroscopic Survey (eBOSS) is mapping the galaxy, quasar, and neutral gas distributions between redshifts z = 0.6 and 3.5 to constrain cosmology using baryon acoustic oscillations, redshift space distortions, and the shape of the power spectrum. Within eBOSS, we are conducting two major subprograms: the SPectroscopic IDentification of eROSITA Sources (SPIDERS), investigating X-ray AGN and galaxies in X-ray clusters, and the Time Domain Spectroscopic Survey (TDSS), obtaining spectra of variable sources. All programs use the 2.5-meter Sloan Foundation Telescope at Apache Point Observatory; observations there began in Summer 2014. APOGEE-2 also operates a second near-infrared spectrograph at the 2.5-meter du Pont Telescope at Las Campanas Observatory, with observations beginning in early 2017. Observations at both facilities are scheduled to continue through 2020. In keeping with previous SDSS policy, SDSS-IV provides regularly scheduled public data releases; the first one, Data Release 13, was made available in July 2016.

We have proposed a method for measuring weak lensing using the Lyman-alpha forest. Here we estimate the noise expected in weak lensing maps and power spectra for different sets of observational parameters. We find that surveys of the size and quality of the ones being done today and ones planned for the future will be able to measure the lensing power spectrum at a source redshift of z~2.5 with high precision and even be able to image the distribution of foreground matter with high fidelity on degree scales. For example, we predict that Lyman-alpha forest lensing measurement from the Dark Energy Spectroscopic Instrument survey should yield the mass fluctuation amplitude with statistical errors of 1.5%. By dividing the redshift range into multiple bins some tomographic lensing information should be accessible as well. This would allow for cosmological lensing measurements at higher redshift than are accessible with galaxy shear surveys and correspondingly better constraints on the evolution of dark energy at relatively early times.

The angular positions of quasars are deflected by the gravitational lensing effect of foreground matter. The Lyman-alpha forest seen in the spectra of these quasars is therefore also lensed. We propose that the signature of weak gravitational lensing of the forest could be measured using similar techniques that have been applied to the lensed Cosmic Microwave Background, and which have also been proposed for application to spectral data from 21cm radio telescopes. As with 21cm data, the forest has the advantage of spectral information, potentially yielding many lensed "slices" at different redshifts. We perform an illustrative idealized test, generating a high resolution angular grid of quasars (of order arcminute separation), and lensing the Lyman-alphaforest spectra at redshifts z=2-3 using a foreground density field. We find that standard quadratic estimators can be used to reconstruct images of the foreground mass distribution at z~1. There currently exists a wealth of Lya forest data from quasar and galaxy spectral surveys, with smaller sightline separations expected in the future. Lyman-alpha forest lensing is sensitive to the foreground mass distribution at redshifts intermediate between CMB lensing and galaxy shear, and avoids the difficulties of shape measurement associated with the latter. With further refinement and application of mass reconstruction techniques, weak gravitational lensing of the high redshift Lya forest may become a useful new cosmological probe.

We employ the very large cosmological hydrodynamical simulation BLUETIDES to investigate the predicted properties of the galaxy population during the epoch of reionisation ($z>8$). BLUETIDES has a resolution and volume ($(400/h\approx 577)^{3}\,{\rm cMpc^3}$) providing a population of galaxies which is well matched to depth and area of current observational surveys targeting the high-redshift Universe. At $z=8$ BLUETIDES includes almost 160,000 galaxies with stellar masses $>10^{8}\,{\rm M_{\odot}}$. The population of galaxies predicted by BLUETIDES closely matches observational constraints on both the galaxy stellar mass function and far-UV ($150\,{\rm nm}$) luminosity function. Galaxies in BLUETIDES are characterised by rapidly increasing star formation histories. Specific star formation rates decrease with redshift though remain largely insensitive to stellar mass. As a result of the enhanced surface density of metals more massive galaxies are predicted to have higher dust attenuation resulting in a significant steepening of the observed far-UV luminosity function at high luminosities. The contribution of active SMBHs to the UV luminosities of galaxies with stellar masses $10^{9-10}\,{\rm M_{\odot}}$ is around $3\%$ on average. Approximately $25\%$ of galaxies with $M_{*}\approx 10^{10}\,{\rm M_{\odot}}$ are predicted to have active SMBH which contribute $>10\%$ of the total UV luminosity.

A strong correlation has been measured between the observed centripetal accelerations in galaxies and the accelerations implied by the baryonic components of galaxies. This empirical radial acceleration relation must be accounted for in any viable model of galaxy formation. We measure and compare the radial accelerations contributed by baryons and by dark matter in disk galaxies in the MassiveBlack-II hydrodynamic galaxy formation simulation. The sample of 1594 galaxies spans three orders of magnitude in luminosity and four in surface brightness, comparable to the observed sample from the Spitzer Photometry & Accurate Rotation Curves (SPARC) dataset used by McGaugh et al. (2016). We find that radial accelerations contributed by baryonic matter only and by total matter are highly correlated, with only small scatter around their mean or median relation, despite the wide ranges of galaxy luminosity and surface brightness. We further find that the radial acceleration relation in this simulation differs from that of the SPARC sample, and can be described by a simple power law in the acceleration range we are probing.

We present the Data Release 12 Quasar catalog (DR12Q) from the Baryon Oscillation Spectroscopic Survey (BOSS) of the SDSS-III. This catalog includes all SDSS-III/BOSS objects that were spectroscopically targeted as quasar candidates during the full survey and that are confirmed as quasars via visual inspection of the spectra, have luminosities Mi[z=2]<-20.5 (in a LCDM cosmology with H_0 = 70 km/s/Mpc, Omega_M = 0.3, and Omega _L=0.7), and either display at least one emission line with a full width at half maximum (FWHM)larger than 500 km/s or, if not, have interesting/complex absorption features. The catalog also includes previously known quasars (mostly from SDSS-I and II) that were reobserved by BOSS. The catalog contains 297,301 quasars detected over 9,376 square degrees with robust identification and redshift measured by a combination of principal component eigenspectra. The number of quasars with z>2.15 is about an order of magnitude greater than the number of z>2.15 quasars known prior to BOSS. Redshifts and FWHMs are provided for the strongest emission lines (CIV, CIII], MgII). The catalog identifies 29,580 broad absorption line quasars and lists their characteristics. For each object, the catalog presents five-band (u, g, r, i, z) CCD-based photometry together with some information on the optical morphology and the selection criteria. When available, the catalog also provides information on the optical variability of quasars using SDSS and PTF multi-epoch photometry. The catalog also contains X-ray, ultraviolet, near-infrared, and radio emission properties of the quasars, when available, from other large-area surveys. The calibrated digital spectra, covering the wavelength region 3,600-10,500A at a spectral resolution in the range 1,300<R<2,500, can be retrieved from the SDSS Catalog Archive Server.

The growth of the most massive black holes in the early universe, consistent with the detection of highly luminous quasars at $z> 6$ implies sustained, critical accretion of material to grow and power them. Given a black hole seed scenario, it is still uncertain which conditions in the early Universe allow the fastest black hole growth. Large scale hydrodynamical cosmological simulations of structure formation allow us to explore the conditions conducive to the growth of the earliest supermassive black holes. We use the cosmological hydrodynamic simulation BlueTides, which incorporates a variety of baryon physics in a (400 Mpc/h)^3 volume with 0.7 trillion particles to follow the earliest phases of black hole critical growth. At z=8 the most massive black holes (a handful) approach masses of 10^8 Msun with the most massive (with M_BH = 4 x 10^8 Msun ) being found in an extremely compact spheroid-dominated host galaxy. Examining the large-scale environment of hosts, we find that the initial tidal field is more important than overdensity in setting the conditions for early BH growth. In regions of low tidal fields gas accretes 'cold' onto the black hole and falls along thin, radial filaments straight into the center forming the most compact galaxies and most massive black holes at earliest times. Regions of high tidal fields instead induce larger coherent angular momenta and influence the formation of the first population of massive compact disks. The extreme early growth depends on the early interplay of high gas densities and the tidal field that shapes the mode of accretion. Mergers play a minor role in the formation of the first generation, rare massive BHs.

We use the BlueTides simulation to predict the properties of the high-$z$ galaxy and active galactic nuclei (AGN) populations for the planned 2200deg$^2$ Wide-Field Infrared Survey Telescope's (WFIRST)-AFTA High Latitude Survey (HLS). BlueTides is a cosmological hydrodynamic simulation, which incorporates a variety of baryon physics in a $(400h^{-1} \mathrm{Mpc})^3$ volume evolved to $z=8$ with 0.7 trillion particles. The galaxy luminosity functions in the simulation show good agreement with all the current observational constraints (up to $z=11$) and predicts an enhanced number of UV bright galaxies. At the proposed depth of the HLS ($m < 26.75$), BlueTides predicts $10^6$ galaxies at $z=8$ with a few up to $z\sim 15$ due to the enhanced bright end of the galaxy luminosity function. At $z=8$, galaxies in the mock HLS have specific star formation rates of $\sim 10 {\rm Gyr}^{-1}$ and ages of $\sim 80 {\rm Myr}$ (both evolving linearly with redshift) and a non-evolving mass-metallicity relation. BlueTides also predicts $\sim 10^4$ AGN in WFIRST HLS from $z=8$ out to $z\sim 14$. These AGN host black holes of $M\sim 10^6-10^8 M_\odot$ accreting close to their Eddington luminosity. Galaxies and AGN have host halo masses of $M_{halo}\sim 10^{11-12} M_\odot$ and a linear bias $b\approx 13-20$. Given the expected galaxy space densities, their high bias and large volume probed we speculate that it may be feasible for WFIRST HLS detect the Baryon Acoustic Oscillation peak in the galaxy power spectrum out to $z=8-9$.

We use the large cosmological hydro-dynamic simulation BlueTides to predict the photometric properties of galaxies during the epoch of reionisation ($z=8-15$). These properties include the rest-frame UV to near-IR broadband spectral energy distributions, the Lyman continuum photon production, the UV star formation rate calibration, and intrinsic UV continuum slope. In particular we focus on exploring the effect of various modelling assumptions, including the assumed choice of stellar population synthesis model, initial mass function, and the escape fraction of Lyman continuum photons, upon these quantities. We find that these modelling assumptions can have a dramatic effect on photometric properties leading to consequences for the accurate determination of physical properties from observations. For example, at $z=8$ we predict that nebular emission can account for up-to $50\%$ of the rest-frame $R$-band luminosity, while the choice of stellar population synthesis model can change the Lyman continuum production rate up to a factor of $\times 2$.

Using deep Hubble and Spitzer observations Oesch et al. (2016) have identified a bright ($M_{\rm UV}\approx -22$) star forming galaxy candidate at $z \approx 11$. The presence of GN-$z11$ implies a number density $\sim 10^{-6}\,{\rm Mpc^{-3}}$, roughly an order of magnitude higher than the expected value based on extrapolations from lower redshift. Using the unprecedented volume and high resolution of the BlueTides cosmological hydrodynamical simulation, we study the population of luminous rare objects at $z > 10$. The luminosity function in BlueTides implies an enhanced number of massive galaxies, consistent with the observation of GN-$z11$. We find about 30 galaxies at $M_{\rm UV}\approx -22$ at $z = 11$ in the BlueTides volume, including a few objects about 1.5 magnitudes brighter. The probability of observing GN-$z11$ in the volume probed by Oesch et al. (2016) is $\sim 13$ per cent. The predicted properties of the rare bright galaxies at $z = 11$ in BlueTides closely match those inferred from the observations of GN-$z11$. BlueTides predicts a negligible contribution from faint AGN in the observed SED. The enormous increase in volume surveyed by WFIRST will provide observations of $\sim1000$ galaxies with $M_{\rm UV} < -22$ beyond $z = 11$ out to $z = 13.5$.

The Lyman Continuum photon production efficiency ($\xi_{\rm ion}$) is a critical ingredient for inferring the number of photons available to reionise the intergalactic medium. To estimate the theoretical production efficiency in the high-redshift Universe we couple the BlueTides cosmological hydrodynamical simulation with a range of stellar population synthesis models. We find Lyman Continuum photon production efficiencies of $\log_{10}(\xi_{\rm ion}/{\rm erg^{-1}\, Hz})\approx 25.1-25.5$ depending on the choice of stellar population synthesis model. These results are broadly consistent with recent observational constraints at high-redshift though favour a model incorporating the effects of binary evolution

Maps of the large-scale structure of the Universe at redshifts 2-4 can be made with the Lyman-alpha forest which are complementary to low redshift galaxy surveys. We apply the Wiener interpolation method of Caucci et al. to construct three-dimensional maps from sets of Lyman-alpha forest spectra taken from cosmological hydrodynamic simulations. We mimic some current and future quasar redshift surveys (BOSS, eBOSS and MS-DESI) by choosing similar sightline densities. We use these appropriate subsets of the Lyman-alpha absorption sightlines to reconstruct the full three dimensional Lyman-alpha flux field and perform comparisons between the true and the reconstructed fields. We study global statistical properties of the intergalactic medium (IGM) maps with auto-correlation and cross-correlation analysis, slice plots, local peaks and point by point scatter. We find that both the density field and the statistical proper- ties of the IGM are recovered well enough that the resulting IGM maps can be meaningfully considered to represent large-scale maps of the Universe in agreement with Caucci et al., on larger scales and for sparser sightlines than had been tested previously. Quantitatively, for sightline parameters comparable to current and near future surveys the correlation coefficient between true and reconstructed fields is r > 0.9 on scales > 30 h^-1 Mpc. The properties of the maps are relatively insensitive to the precise form of the covariance matrix used. The final BOSS quasar Lyman-alpha forest sample will allow maps to be made with a resolution of ~ 30 h^-1 Mpc over a volume of ~ 15 h^-3 Gpc^3 between redshifts 1.9 and 2.3.

The Extended Baryon Oscillation Spectroscopic Survey (eBOSS) will conduct novel cosmological observations using the BOSS spectrograph at Apache Point Observatory. Observations will be simultaneous with the Time Domain Spectroscopic Survey (TDSS) designed for variability studies and the Spectroscopic Identification of eROSITA Sources (SPIDERS) program designed for studies of X-ray sources. eBOSS will use four different tracers to measure the distance-redshift relation with baryon acoustic oscillations (BAO). Using more than 250,000 new, spectroscopically confirmed luminous red galaxies at a median redshift z=0.72, we project that eBOSS will yield measurements of $d_A(z)$ to an accuracy of 1.2% and measurements of H(z) to 2.1% when combined with the z>0.6 sample of BOSS galaxies. With ~195,000 new emission line galaxy redshifts, we expect BAO measurements of $d_A(z)$ to an accuracy of 3.1% and H(z) to 4.7% at an effective redshift of z= 0.87. A sample of more than 500,000 spectroscopically-confirmed quasars will provide the first BAO distance measurements over the redshift range 0.9<z<2.2, with expected precision of 2.8% and 4.2% on $d_A(z)$ and H(z), respectively. Finally, with 60,000 new quasars and re-observation of 60,000 quasars known from BOSS, we will obtain new Lyman-alpha forest measurements at redshifts z>2.1; these new data will enhance the precision of $d_A(z)$ and H(z) by a factor of 1.44 relative to BOSS. Furthermore, eBOSS will provide improved tests of General Relativity on cosmological scales through redshift-space distortion measurements, improved tests for non-Gaussianity in the primordial density field, and new constraints on the summed mass of all neutrino species. Here, we provide an overview of the cosmological goals, spectroscopic target sample, demonstration of spectral quality from early data, and projected cosmological constraints from eBOSS.

The intrinsic scatter in the ellipticities of galaxies about the mean shape, known as "shape noise," is the most important source of noise in weak lensing shear measurements. Several approaches to reducing shape noise have recently been put forward, using information beyond photometry, such as radio polarization and optical spectroscopy. Here we investigate how well the intrinsic ellipticities of galaxies can be predicted using other, exclusively photometric parameters. These parameters (such as galaxy colours) are already available in the data and do not necessitate additional, often expensive observations. We apply two regression techniques, generalized additive models (GAM) and projection pursuit regression (PPR) to the publicly released data catalog of galaxy properties from CFHTLenS. In our simple analysis we find that the individual galaxy ellipticities can indeed be predicted from other photometric parameters to better precision than the scatter about the mean ellipticity. This means that without additional observations beyond photometry the ellipticity contribution to the shear can be measured to higher precision, comparable to using a larger sample of galaxies. Our best-fit model, achieved using PPR, yields a gain equivalent to having 114.3% more galaxies. Using only parameters unaffected by lensing (e.g.~surface brightness, colour), the gain is only ~12%.

We examine the luminosity function (LF) of [OII] emission-line galaxies in the high-resolution cosmological simulation MassiveBlack-II (MBII). From the spectral energy distribution of each galaxy, we select a sub-sample of star-forming galaxies at $0.06 \le z \le 3.0$ using the [OII] emission line luminosity L([OII]). We confirm that the specific star formation rate matches that in the GAMA survey. We show that the [OII] LF at z=1.0 from the MBII shows a good agreement with the LFs from several surveys below L([OII])=$10^{43.0}$ erg/s while the low redshifts ($z \le 0.3$) show an excess in the prediction of bright [OII] galaxies, but still displaying a good match with observations below L([OII])=$10^{41.6}$ erg/s. Based on the validity in reproducing the properties of [OII] galaxies at low redshift ($z \le 1$), we forecast the evolution of the [OII] LF at high redshift ($z \le 3$), which can be tested by upcoming surveys such as the HETDEX and DESI. The slopes of the LFs at bright and faint ends range from -3 to -2 showing minima at z=2. The slope of the bright end evolves approximately as 1/(z+1) at z=2 while the faint end evolves as ~3/(z+1) at $0.6 \le z \le 2$. In addition, a similar analysis is applied for the evolution of [OIII] LFs, which is to be explored in the forthcoming survey WFIRST-AFTA. Finally, we show that the auto-correlation function of [OII] and [OIII] emitting galaxies shows a rapid evolution from z=2 to 1.

In this paper, we study the filamentary structures and the galaxy alignment along filaments at redshift $z=0.06$ in the MassiveBlack-II simulation, a state-of-the-art, high-resolution hydrodynamical cosmological simulation which includes stellar and AGN feedback in a volume of (100 Mpc$/h$)$^3$. The filaments are constructed using the subspace constrained mean shift (SCMS; Ozertem & Erdogmus (2011) and Chen et al. (2015a)). First, we show that reconstructed filaments using galaxies and reconstructed filaments using dark matter particles are similar to each other; over $50\%$ of the points on the galaxy filaments have a corresponding point on the dark matter filaments within distance $0.13$ Mpc$/h$ (and vice versa) and this distance is even smaller at high-density regions. Second, we observe the alignment of the major principal axis of a galaxy with respect to the orientation of its nearest filament and detect a $2.5$ Mpc$/h$ critical radius for filament's influence on the alignment when the subhalo mass of this galaxy is between $10^9M_\odot/h$ and $10^{12}M_\odot/h$. Moreover, we find the alignment signal to increase significantly with the subhalo mass. Third, when a galaxy is close to filaments (less than $0.25$ Mpc$/h$), the galaxy alignment toward the nearest galaxy group depends on the galaxy subhalo mass. Finally, we find that galaxies close to filaments or groups tend to be rounder than those away from filaments or groups.

We have catalogued and analysed cosmological parameter determinations and their error bars published between the years 1990 and 2010. Our study focuses on the number of measurements, their precision and their accuracy. The accuracy of past measurements is gauged by comparison with the WMAP7 results. The 637 measurements in our study are of 12 different parameters and we place the techniques used to carry them out into 12 different categories. We find that the number of published measurements per year in all 12 cases except for the dark energy equation of state parameter w_0 peaked between 1995 and 2004. Of the individual techniques, only BAO measurements were still rising in popularity at the end of the studied time period. The fractional error associated with most measurements has been declining relatively slowly, with several parameters, such as the amplitude of mass fluctutations sigma_8 and the Hubble constant H_0 remaining close to the 10% precision level for a 10-15 year period. The accuracy of recent parameter measurements is generally what would be expected given the quoted error bars, although before the year 2000, the accuracy was significantly worse, consistent with an average underestimate of the error bars by a factor of ~2. When used as complement to traditional forecasting techniques, our results suggest that future measurements of parameters such as fNL, and w_a will have been informed by the gradual improvment in understanding and treatment of systematic errors and are likely to be accurate. However, care must be taken to avoid the effects of confirmation bias, which may be affecting recent measurements of dark energy parameters. For example, of the 28 measurements of Omega_Lambda in our sample published since 2003, only 2 are more than 1 sigma from the WMAP results. Wider use of blind analyses in cosmology could help to avoid this.

Whether among the myriad tiny proto-galaxies there exists a population with similarities to present day galaxies is an open question. We show, using BlueTides, the first hydrodynamic simulation large enough to resolve the relevant scales, that the first massive galaxies to form are %in fact predicted to have extensive rotationally-supported disks. Although their morphology resembles in some ways Milky-way types seen at much lower redshifts, these high-redshift galaxies are smaller, denser, and richer in gas than their low redshift counterparts. From a kinematic analysis of a statistical sample of 216 galaxies at redshift $z=8-10$ we have found that disk galaxies make up 70\% of the population of galaxies with stellar mass $10^{10} M_\odot$ or greater. Cold Dark Matter cosmology therefore makes specific predictions for the population of large galaxies 500 million years after the Big Bang. We argue that wide-field satellite telescopes (e.g. WFIRST) will in the near future discover these first massive disk galaxies. The simplicity of their structure and formation history should make possible new tests of cosmology.

The third generation of the Sloan Digital Sky Survey (SDSS-III) took data from 2008 to 2014 using the original SDSS wide-field imager, the original and an upgraded multi-object fiber-fed optical spectrograph, a new near-infrared high-resolution spectrograph, and a novel optical interferometer. All the data from SDSS-III are now made public. In particular, this paper describes Data Release 11 (DR11) including all data acquired through 2013 July, and Data Release 12 (DR12) adding data acquired through 2014 July (including all data included in previous data releases), marking the end of SDSS-III observing. Relative to our previous public release (DR10), DR12 adds one million new spectra of galaxies and quasars from the Baryon Oscillation Spectroscopic Survey (BOSS) over an additional 3000 sq. deg of sky, more than triples the number of H-band spectra of stars as part of the Apache Point Observatory (APO) Galactic Evolution Experiment (APOGEE), and includes repeated accurate radial velocity measurements of 5500 stars from the Multi-Object APO Radial Velocity Exoplanet Large-area Survey (MARVELS). The APOGEE outputs now include measured abundances of 15 different elements for each star. In total, SDSS-III added 2350 sq. deg of ugriz imaging; 155,520 spectra of 138,099 stars as part of the Sloan Exploration of Galactic Understanding and Evolution 2 (SEGUE-2) survey; 2,497,484 BOSS spectra of 1,372,737 galaxies, 294,512 quasars, and 247,216 stars over 9376 sq. deg; 618,080 APOGEE spectra of 156,593 stars; and 197,040 MARVELS spectra of 5,513 stars. Since its first light in 1998, SDSS has imaged over 1/3 of the Celestial sphere in five bands and obtained over five million astronomical spectra.

We introduce the BlueTides simulation and report initial results for the luminosity functions of the first galaxies and AGN, and their contribution to reionization. BlueTides was run on the BlueWaters cluster at NCSA from $z=99$ to $z=8.0$ and includes 2$\times$7040$^3$ particles in a $400$Mpc/h per side box, making it the largest hydrodynamic simulation ever performed at high redshift. BlueTides includes a pressure-entropy formulation of smoothed particle hydrodynamics, gas cooling, star formation (including molecular hydrogen), black hole growth and models for stellar and AGN feedback processes. The star formation rate density in the simulation is a good match to current observational data at $z\sim 8-10$. We find good agreement between observations and the predicted galaxy luminosity function in the currently observable range $-18\le M_{\mathrm UV} \le -22.5$ with some dust extinction required to match the abundance of brighter objects. BlueTides implements a patchy reionization model that produces a fluctuating UV background. BlueTides predicts number counts for galaxies fainter than current observational limits which are consistent with extrapolating the faint end slope of the luminosity function with a power law index $\alpha\sim -1.8$ at $z\sim 8$ and redshift dependence of $\alpha\sim (1+z)^{-0.4}$. The AGN population has a luminosity function well fit by a power law with a slope $\alpha\sim -2.4$ that compares favourably with the deepest CANDELS-Goods fields. We investigate how these luminosity functions affect the progress of reionization, and find that a high Lyman-$\alpha$ escape fraction ($f_\mathrm{esc} \sim 0.5$) is required if galaxies dominate the ionising photon budget during reionization. Smaller galaxy escape fractions imply a large contribution from faint AGN (down to $M_\mathrm{UV}=-12$) which results in a rapid reionization, disfavoured by current observations.

(Abridged) We detect the large-scale structure of Lya emission in the Universe at redshifts z=2-3.5 by measuring the cross-correlation of Lya surface brightness with quasars in SDSS/BOSS. We use a million spectra targeting Luminous Red Galaxies at z<0.8, after subtracting a best fit model galaxy spectrum from each one, as an estimate of the high-redshift Lya surface brightness. The quasar-Lya emission cross-correlation we detect has a shape consistent with a LambdaCDM model with Omega_M =0.30^+0.10-0.07. The predicted amplitude of this cross-correlation is proportional to the product of the mean Lya surface brightness, <mu_alpha>, the amplitude of mass fluctuations, and the quasar and Lya emission bias factors. Using known values, we infer <mu_alpha>(b_alpha/3) = (3.9 +/- 0.9) x 10^-21 erg/s cm^-2 A^-1 arcsec^-2, where b_alpha is the Lya emission bias factor. If the dominant sources of Lya emission are star forming galaxies, we infer rho_SFR = (0.28 +/- 0.07) (3/b_alpha) /yr/Mpc^3 at z=2-3.5. For b_alpha=3, this value is a factor of 21-35 above previous estimates from individually detected Lya emitters, although consistent with the total rho_SFR derived from dust-corrected, continuum UV surveys. 97% of the Lya emission in the Universe at these redshifts is therefore undetected in previous surveys of Lya emitters. Our measurement is much greater than seen from stacking analyses of faint halos surrounding previously detected Lya emitters, but we speculate that it arises from similar Lya halos surrounding all luminous star-forming galaxies. We also detect redshift space anisotropy of the quasar-Lya emission cross-correlation, finding evidence at the 3.0 sigma level that it is radially elongated, consistent with distortions caused by radiative-transfer effects (Zheng et al. (2011)). Our measurements represent the first application of the intensity mapping technique to optical observations.

We present the first observations of foreground Lyman-$\alpha$ forest absorption from high-redshift galaxies, targeting 24 star-forming galaxies (SFGs) with $z\sim 2.3-2.8$ within a $5' \times 15'$ region of the COSMOS field. The transverse sightline separation is $\sim 2\,h^{-1}\mathrm{Mpc}$ comoving, allowing us to create a tomographic reconstruction of the 3D Ly$\alpha$ forest absorption field over the redshift range $2.20\leq z\leq 2.45$. The resulting map covers $6\,h^{-1}\mathrm{Mpc} \times 14\,h^{-1}\mathrm{Mpc}$ in the transverse plane and $230\,h^{-1}\mathrm{Mpc}$ along the line-of-sight with a spatial resolution of $\approx 3.5\,h^{-1}\mathrm{Mpc}$, and is the first high-fidelity map of large-scale structure on $\sim\mathrm{Mpc}$ scales at $z>2$. Our map reveals significant structures with $\gtrsim 10\,h^{-1}\mathrm{Mpc}$ extent, including several spanning the entire transverse breadth, providing qualitative evidence for the filamentary structures predicted to exist in the high-redshift cosmic web. Simulated reconstructions with the same sightline sampling, spectral resolution, and signal-to-noise ratio recover the salient structures present in the underlying 3D absorption fields. Using data from other surveys, we identified 18 galaxies with known redshifts coeval with our map volume enabling a direct comparison to our tomographic map. This shows that galaxies preferentially occupy high-density regions, in qualitative agreement with the same comparison applied to simulations. Our results establishes the feasibility of the CLAMATO survey, which aims to obtain Ly$\alpha$ forest spectra for $\sim 1000$ SFGs over $\sim 1 \,\mathrm{deg}^2$ of the COSMOS field, in order to map out IGM large-scale structure at $\langle z \rangle \sim 2.3$ over a large volume $(100\,h^{-1}\mathrm{Mpc})^3$.

The z > 2 Lyman-alpha (Lya) forest traces the underlying dark-matter distribution on large scales and, given sufficient sightlines, can be used to create 3D maps of large-scale structure. We examine the observational requirements to construct such maps and estimate the signal-to-noise as a function of exposure time and sightline density. Sightline densities at z = 2.25 are n_los = [360, 1200,3300] deg^{-2} at limiting magnitudes of g =[24.0, 24.5,25.0], resulting in transverse sightline separations of d_perp = [3.6, 1.9, 1.2] h^{-1} Mpc, which roughly sets the reconstruction scale. We simulate these reconstructions using mock spectra with realistic noise properties, and find that spectra with S/N = 4 per angstrom can be used to generate maps that clearly trace the underlying dark-matter at overdensities of rho/<rho> ~ 1. For the VLT/VIMOS spectrograph, exposure times t_exp = [4, 6, 10] hrs are sufficient for maps with spatial resolution epsilon_3d = [5.0, 3.2, 2.3] h^{-1} Mpc. Assuming ~ 250 h^{-1} Mpc is probed along the line-of-sight, 1 deg^2 of survey area would cover a comoving volume of ~ 10^6 h^{-3} Mpc^3 at <z>=2.3, enabling efficient mapping of large volumes with 8-10m telescopes. These maps could be used to study galaxy environments, detect proto-clusters, and study the topology of large-scale structure at high-z.

(Abridged for arXiv)We investigate the properties of halos, galaxies and blackholes to z=0 in the high resolution hydrodynamical simulation MassiveBlack-II (MBII) which evolves a LCDM cosmology in a comoving volume Vbox=100(Mpc/h)^3. MBII is the highest resolution simulation of this size which includes a self-consistent model for star formation, black hole accretion and associated feedback. We provide a simulation browser web application which enables interactive search and tagging of halos, subhalos and their properties and publicly release our galaxy catalogs. Our analysis of the halo mass function (MF) in MBII reveals that baryons have strong effects, with changes in the halo abundance of 20-35% below the knee of the MF (Mhalo < 10^13.2 Msun/h at z=0) when compared to fits based on dark matter only simulations. We provide a fitting function for the halo MF out to redshift z=11 and discuss how the onset of non-universality in the MF limits the accuracy of our fit. We study the halo occupation distribution and clustering of galaxies, in particular the evolution and scale dependence of stochasticity and bias finding reasonable agreement with observational data. The shape of the cosmic spectral energy distribution predicted by MBII is consistent with observations, but lower in amplitude. The Galaxy Stellar Mass Function (GSMF) function is broadly consistent with observations at z>=2. At z<2, the population of passive low mass (for M*<10^9 Msun) galaxies in MBII makes the GSMF too steep compared to observations whereas at the high mass end (M*>10^11 Msun) galaxies hosting bright AGNs make significant contributions to the GSMF. The quasar bolometric luminosity function is also largely consistent with observations. We note however that more efficient AGN feedback (beyond simple thermal coupling used here) is likely necessary for the largest, rarest objects/clusters at low redshifts.

The Sloan Digital Sky Survey (SDSS) has been in operation since 2000 April. This paper presents the tenth public data release (DR10) from its current incarnation, SDSS-III. This data release includes the first spectroscopic data from the Apache Point Observatory Galaxy Evolution Experiment (APOGEE), along with spectroscopic data from the Baryon Oscillation Spectroscopic Survey (BOSS) taken through 2012 July. The APOGEE instrument is a near-infrared R~22,500 300-fiber spectrograph covering 1.514--1.696 microns. The APOGEE survey is studying the chemical abundances and radial velocities of roughly 100,000 red giant star candidates in the bulge, bar, disk, and halo of the Milky Way. DR10 includes 178,397 spectra of 57,454 stars, each typically observed three or more times, from APOGEE. Derived quantities from these spectra (radial velocities, effective temperatures, surface gravities, and metallicities) are also included.DR10 also roughly doubles the number of BOSS spectra over those included in the ninth data release. DR10 includes a total of 1,507,954 BOSS spectra, comprising 927,844 galaxy spectra; 182,009 quasar spectra; and 159,327 stellar spectra, selected over 6373.2 square degrees.

Visualizing the high-redshift Universe is difficult due to the dearth of available data; however, the Lyman-alpha forest provides a means to map the intergalactic medium at redshifts not accessible to large galaxy surveys. Large-scale structure surveys, such as the Baryon Oscillation Spectroscopic Survey (BOSS), have collected quasar (QSO) spectra that enable the reconstruction of HI density fluctuations. The data fall on a collection of lines defined by the lines-of-sight (LOS) of the QSO, and a major issue with producing a 3D reconstruction is determining how to model the regions between the LOS. We present a method that produces a 3D map of this relatively uncharted portion of the Universe by employing local polynomial smoothing, a nonparametric methodology. The performance of the method is analyzed on simulated data that mimics the varying number of LOS expected in real data, and then is applied to a sample region selected from BOSS. Evaluation of the reconstruction is assessed by considering various features of the predicted 3D maps including visual comparison of slices, PDFs, counts of local minima and maxima, and standardized correlation functions. This 3D reconstruction allows for an initial investigation of the topology of this portion of the Universe using persistent homology.

We use zoom-in techniques to re-simulate three high-redshift (z > 5.5) halos which host 10^9 solar mass blackholes from the ~ Gpc volume, MassiveBlack cosmological hydrodynamic simulation. We examine a number of factors potentially affecting supermassive blackhole growth at high redshift in cosmological simulations. These include numerical resolution, feedback prescriptions and formulation of smoothed particle hydrodynamics. We find that varying the size of the region over which feedback energy is deposited directly, either for fixed number of neighbours or fixed volume makes very little difference to the accretion history of blackholes. Changing mass resolution by factors of up to 64 also does not change the blackhole growth history significantly. We find that switching from the density-entropy formulation to the pressure-entropy formulation of smoothed particle hydrodynamics slightly increases the accretion rate onto blackholes. In general numerical details appear to have small effects on the main fueling mechanism for blackholes at these high redshifts. We examine the fashion by which this occurs, finding that the insensitivity to simulation technique seems to be a hallmark of the cold flow feeding picture of these high-z supermassive blackholes. We show that the gas that participates in critical accretion phases, in these massive objects at z > 6~7 is in all cases colder, denser, and forms more coherent streams than the average gas in the halo. This is also mostly the case when the blackhole accretion is feedback regulated (z < 6), however the distinction is less prominent. For our resimulated halos, cold flows appear to be a viable mechanism for forming the most massive blackholes in the early universe, occurring naturally in LambdaCDM models of structure formation. Not requiring fine tuning of numerical parameters, they seem to be physically inevitable in these objects.

We have developed two independent methods to measure the one-dimensional power spectrum of the transmitted flux in the Lyman-$\alpha$ forest. The first method is based on a Fourier transform, and the second on a maximum likelihood estimator. The two methods are independent and have different systematic uncertainties. The determination of the noise level in the data spectra was subject to a novel treatment, because of its significant impact on the derived power spectrum. We applied the two methods to 13,821 quasar spectra from SDSS-III/BOSS DR9 selected from a larger sample of over 60,000 spectra on the basis of their high quality, large signal-to-noise ratio, and good spectral resolution. The power spectra measured using either approach are in good agreement over all twelve redshift bins from $<z> = 2.2$ to $<z> = 4.4$, and scales from 0.001 $\rm(km/s)^{-1}$ to $0.02 \rm(km/s)^{-1}$. We determine the methodological and instrumental systematic uncertainties of our measurements. We provide a preliminary cosmological interpretation of our measurements using available hydrodynamical simulations. The improvement in precision over previously published results from SDSS is a factor 2--3 for constraints on relevant cosmological parameters. For a $\Lambda$CDM model and using a constraint on $H_0$ that encompasses measurements based on the local distance ladder and on CMB anisotropies, we infer $\sigma_8 =0.83\pm0.03$ and $n_s= 0.97\pm0.02$ based on \ion{H}{i} absorption in the range $2.1<z<3.7$.

We present an analysis of high-resolution N-body simulations of decaying dark matter cosmologies focusing on the statistical properties of the transmitted Lyman-alpha forest flux in the high-redshift intergalactic medium. In this type of model a dark matter particle decays into a slightly less massive stable dark matter daughter particle and a comparably light particle. The small mass splitting provides a non-relativistic kick velocity V_k to the daughter particle resulting in free-streaming and subsequent damping of small-scale density fluctuations. Current Lyman-alpha forest power spectrum measurements probe comoving scales up to ~ 2-3 h^-1 Mpc at redshifts z ~ 2-4, providing one of the most robust ways to probe cosmological density fluctuations on relatively small scales. The suppression of structure growth due to the free-streaming of dark matter daughter particles also has a significant impact on the neutral hydrogen cloud distribution, which traces the underlying dark matter distribution well at high redshift. We exploit Lyman-alpha forest power spectrum measurements to constrain the amount of free-streaming of dark matter in such models and thereby place limits on decaying dark matter based only on the dynamics of cosmological perturbations without any assumptions about the interactions of the decay products. We find that SDSS 1D Lyman-alpha forest power spectrum data place a lifetime-dependent upper limit V_k < 30-70 km/s for decay lifetimes < 10 Gyr. This is the most stringent model-independent bound on invisible dark matter decays with small mass splittings. For large mass splittings (large V_k), Lyman-alpha forest data restrict the dark matter lifetime to Gamma^-1 > 40 Gyr. Forthcoming BOSS data should be able to provide more stringent constraints on exotic dark matter, mainly because the larger BOSS quasar spectrum sample will significantly reduce statistical errors.

By combining optical and near-IR observations from the Hubble Space Telescope with NIR photometry from the Spitzer Space Telescope it is possible to measure the rest-frame UV-optical colours of galaxies at z=4-8. The UV-optical spectral energy distribution of star formation dominated galaxies is the result of several different factors. These include the joint distribution of stellar masses, ages, and metallicities, and the subsequent reprocessing by dust and gas in the ISM. Using a large cosmological hydrodynamical simulation we investigate the predicted spectral energy distributions of galaxies at high-redshift with a particular emphasis on assessing the potential contribution of nebular emission. We find that the average pure stellar UV-optical colour correlates with both luminosity and redshift such that galaxies at lower-redshift and higher-luminosity are typically redder. Assuming the escape fraction of ionising photons is close to zero, the effect of nebular emission is to redden the UV-optical 1500-V_w colour by, on average, 0.4 mag at z=8 declining to 0.25 mag at z=4. Young and low-metallicity stellar populations, which typically have bluer pure stellar UV-optical colours, produce larger ionising luminosities and are thus more strongly affected by the reddening effects of nebular emission. This causes the distribution of 1500-V_w colours to narrow and the trends with luminosity and redshift to weaken. The strong effect of nebular emission leaves observed-frame colours critically sensitive to the source redshift. For example, increasing the redshift by 0.1 can result in observed frame colours changing by up to ~0.6. These predictions reinforce the need to include nebular emission when modelling the spectral energy distributions of galaxies at high-redshift and also highlight the difficultly in interpreting the observed colours of individual galaxies without precise redshifts.

We calculate the distribution of HI within 750 proper kpc/h of a quasar, Lbol = 1.62e13 Lsun, powered by an SMBH, Mbh = 4.47e8 Msun, at z = 3. Our numerical model includes a cosmological hydrodynamic simulation that tracks the self consistent growth and thermal feedback of black holes calculated using GADGET-3 as well as a detailed post-processing ray tracing treatment of the non-uniform ionizing radiation field calculated using SPHRAY, which naturally accounts for the self shielding of optically thick systems. We show that the correct treatment of self shielding introduces a flattening feature into the neutral column density distribution around Log NHI = 20 and that regions with the lowest neutral fractions are not those with the highest density gas. For comparison, we solve a Ricatti equation which determines the equilibrium Hydrogen ionization fractions in the presence of a radiation field that falls off as 1/r^2 with regions above a given gas density threshold completely shielded from ionizing radiation. We demonstrate that these semi analytic models cannot reproduce the HI field calculated using SPHRAY. We conclude by comparing our models of this single proximity zone to observations by Hennawi and Prochaska of the absorption spectra of background quasars which are coincident on the sky with foreground quasars in their Quasars Probing Quasars (QPQ) series of papers. Compared to the QPQ sample, we find a factor of 3 fewer optically thick (Log NHI > 17.2) systems around our quasar, however the dark matter halo that hosts our simulated quasar, Mhalo = 5.25e12 Msun, is less massive than the typical QPQ host halo by a factor of four. Allowing for a linear scaling between halo mass, baryonic overdensity and number of absorbers, we estimate the typical host halo mass in the QPQ sample as 1.92e13 Msun.

The recent measurement of the gravitational redshifts of galaxies in galaxy clusters by Wojtak et al. has opened a new observational window on dark matter and modified gravity. By stacking clusters this determination effectively used the line of sight distortion of the cross-correlation function of massive galaxies and lower mass galaxies to estimate the gravitational redshift profile of clusters out to 4 Mpc/h. Here we use a halo model of clustering to predict the distortion due to gravitational redshifts of the cross-correlation function on scales from 1 - 100 Mpc/h. We compare our predictions to simulations and use the simulations to make mock catalogues relevant to current and future galaxy redshift surveys. Without formulating an optimal estimator, we find that the full BOSS survey should be able to detect gravitational redshifts from large-scale structure at the ~4 sigma level. Upcoming redshift surveys will greatly increase the number of galaxies useable in such studies and the BigBOSS and Euclid experiments should be capable of measurements with precision at the few percent level. As has been recently pointed out by McDonald, Kaiser and Zhao et al, other interesting effects including relativistic beaming and transverse Doppler shift can add additional asymmetric distortions to the correlation function. While these contributions are subdominant to the gravitational redshift on large scales, they represent additional opportunities to probe gravitational physics and indicate that many qualitatively new measurements should soon be possible using large redshift surveys.

We use the Baryon Oscillation Spectroscopic Survey (BOSS) Data Release 9 (DR9) to detect and measure the position of the Baryonic Acoustic Oscillation (BAO) feature in the three-dimensional correlation function in the Lyman-alpha forest flux fluctuations at a redshift z=2.4. The feature is clearly detected at significance between 3 and 5 sigma (depending on the broadband model and method of error covariance matrix estimation) and is consistent with predictions of the standard LCDM model. We assess the biases in our method, stability of the error covariance matrix and possible systematic effects. We fit the resulting correlation function with several models that decouple the broadband and acoustic scale information. For an isotropic dilation factor, we measure 100x(alpha_iso-1) = -1.6 ^{+2.0+4.3+7.4}_{-2.0-4.1-6.8} (stat.) +/- 1.0 (syst.) (multiple statistical errors denote 1,2 and 3 sigma confidence limits) with respect to the acoustic scale in the fiducial cosmological model (flat LCDM with Omega_m=0.27, h=0.7). When fitting separately for the radial and transversal dilation factors we find marginalised constraints 100x(alpha_par-1) = -1.3 ^{+3.5+7.6 +12.3}_{-3.3-6.7-10.2} (stat.) +/- 2.0 (syst.) and 100x(alpha_perp-1) = -2.2 ^{+7.4+17}_{-7.1-15} +/- 3.0 (syst.). The dilation factor measurements are significantly correlated with cross-correlation coefficient of ~ -0.55. Errors become significantly non-Gaussian for deviations over 3 standard deviations from best fit value. Because of the data cuts and analysis method, these measurements give tighter constraints than a previous BAO analysis of the BOSS DR9 Lyman-alpha forest sample, providing an important consistency test of the standard cosmological model in a new redshift regime.

We report a detection of the baryon acoustic oscillation (BAO) feature in the three-dimensional correlation function of the transmitted flux fraction in the \Lya forest of high-redshift quasars. The study uses 48,640 quasars in the redshift range $2.1\le z \le 3.5$ from the Baryon Oscillation Spectroscopic Survey (BOSS) of the third generation of the Sloan Digital Sky Survey (SDSS-III). At a mean redshift $z=2.3$, we measure the monopole and quadrupole components of the correlation function for separations in the range $20\hMpc<r<200\hMpc$. A peak in the correlation function is seen at a separation equal to $(1.01\pm0.03)$ times the distance expected for the BAO peak within a concordance $\Lambda$CDM cosmology. This first detection of the BAO peak at high redshift, when the universe was strongly matter dominated, results in constraints on the angular diameter distance $\da$ and the expansion rate $H$ at $z=2.3$ that, combined with priors on $H_0$ and the baryon density, require the existence of dark energy. Combined with constraints derived from Cosmic Microwave Background (CMB) observations, this result implies $H(z=2.3)=(224\pm8){\rm km\,s^{-1}Mpc^{-1}}$, indicating that the time derivative of the cosmological scale parameter $\dot{a}=H(z=2.3)/(1+z)$ is significantly greater than that measured with BAO at $z\sim0.5$. This demonstrates that the expansion was decelerating in the range $0.7<z<2.3$, as expected from the matter domination during this epoch. Combined with measurements of $H_0$, one sees the pattern of deceleration followed by acceleration characteristic of a dark-energy dominated universe.

The observed UV continuum slope of star forming galaxies is strongly affected by the presence of dust. Its observation is then a potentially valuable diagnostic of dust attenuation, particularly at high-redshift where other diagnostics are currently inaccesible. Interpreting the observed UV continuum slope in the context of dust attenuation is often achieved assuming the empirically calibrated Meurer et al. (1999) relation. Implicit in this relation is the assumption of an intrinsic UV continuum slope ($\beta=-2.23$). However, results from numerical simulations suggest that the intrinsic UV continuum slopes of high-redshift star forming galaxies are bluer than this, and moreover vary with redshift. Using values of the intrinsic slope predicted by numerical models of galaxy formation combined with a Calzetti et al. (2000) reddening law we infer UV attenuations ($A_{1500}$) $0.35-0.5\,{\rm mag}$ ($A_{V}$: $0.14-0.2\,{\rm mag}$ assuming Calzetti et al. 2000) greater than simply assuming the Meurer relation. This has significant implications for the inferred amount of dust attenuation at very-high ($z\approx 7$) redshift given current observational constraints on $\beta$, combined with the Meurer relation, suggest dust attenuation to be virtually zero in all but the most luminous systems.

We describe fitting methods developed to analyze fluctuations in the Lyman-{\alpha} forest and measure the parameters of baryon acoustic oscillations (BAO). We apply our methods to BOSS Data Release 9. Our method is based on models of the three-dimensional correlation function in physical coordinate space, and includes the effects of redshift-space distortions, anisotropic non-linear broadening, and broadband distortions. We allow for independent scale factors along and perpendicular to the line of sight to minimize the dependence on our assumed fiducial cosmology and to obtain separate measurements of the BAO angular and relative velocity scales. Our fitting software and the input files needed to reproduce our main BOSS Data Release 9 results are publicly available.

We investigate the evolution of the galaxy stellar mass function at high-redshift ($z\ge 5$) using a pair of large cosmological hydrodynamical simulations: {\em MassiveBlack} and {\em MassiveBlack-II}. By combining these simulations we can study the properties of galaxies with stellar masses greater than $10^{8}\,{\rm M_{\odot}}\,h^{-1}$ and (co-moving) number densities of $\log_{10}(\phi\, [{\rm Mpc^{-3}\,dex^{-1}}\,h^{3}])>-8$. Observational determinations of the galaxy stellar mass function at very-high redshift typically assume a relation between the observed UV luminosity and stellar mass-to-light ratio which is applied to high-redshift samples in order to estimate stellar masses. This relation can also be measured from the simulations. We do this, finding two significant differences with the usual observational assumption: it evolves strongly with redshift and has a different shape. Using this relation to make a consistent comparison between galaxy stellar mass functions we find that at $z=6$ and above the simulation predictions are in good agreement with observed data over the whole mass range. Without using the correct UV luminosity and stellar mass-to-light ratio, the discrepancy would be up to two orders of magnitude for large galaxies $>10^{10}\,{\rm M_{\odot}}\,h^{-1}$. At $z=5$, however the stellar mass function for low mass $<10^{9}\,{\rm M_{\odot}}\,h^{-1}$ galaxies is overpredicted by factors of a few, consistent with the behaviour of the UV luminosity function, and perhaps a sign that feedback in the simulation is not efficient enough for these galaxies.

We present the BOSS Lyman-alpha (Lya) Forest Sample from SDSS Data Release 9, comprising 54,468 quasar spectra with zqso > 2.15 suitable for Lya forest analysis. This data set probes the intergalactic medium with absorption redshifts 2.0 < z_alpha < 5.7 over an area of 3275 square degrees, and encompasses an approximate comoving volume of 20 h^-3 Gpc^3. With each spectrum, we have included several products designed to aid in Lya forest analysis: improved sky masks that flag pixels where data may be unreliable, corrections for known biases in the pipeline estimated noise, masks for the cores of damped Lya systems and corrections for their wings, and estimates of the unabsorbed continua so that the observed flux can be converted to a fractional transmission. The continua are derived using a principal component fit to the quasar spectrum redwards of restframe Lya (lambda > 1216 Ang), extrapolated into the forest region and normalized by a linear function to fit the expected evolution of the Lya forest mean-flux. The estimated continuum errors are ~5% rms. We also discuss possible systematics arising from uncertain spectrophotometry and artifacts in the flux calibration; global corrections for the latter are provided. Our sample provides a convenient starting point for users to analyze clustering in BOSS Lya forest data, and it provides a fiducial data set that can be used to compare results from different analyses of baryon acoustic oscillations in the Lya forest. The full data set is available from the SDSS-III DR9 web site.

The Baryon Oscillation Spectroscopic Survey (BOSS) is designed to measure the scale of baryon acoustic oscillations (BAO) in the clustering of matter over a larger volume than the combined efforts of all previous spectroscopic surveys of large scale structure. BOSS uses 1.5 million luminous galaxies as faint as i=19.9 over 10,000 square degrees to measure BAO to redshifts z<0.7. Observations of neutral hydrogen in the Lyman alpha forest in more than 150,000 quasar spectra (g<22) will constrain BAO over the redshift range 2.15<z<3.5. Early results from BOSS include the first detection of the large-scale three-dimensional clustering of the Lyman alpha forest and a strong detection from the Data Release 9 data set of the BAO in the clustering of massive galaxies at an effective redshift z = 0.57. We project that BOSS will yield measurements of the angular diameter distance D_A to an accuracy of 1.0% at redshifts z=0.3 and z=0.57 and measurements of H(z) to 1.8% and 1.7% at the same redshifts. Forecasts for Lyman alpha forest constraints predict a measurement of an overall dilation factor that scales the highly degenerate D_A(z) and H^{-1}(z) parameters to an accuracy of 1.9% at z~2.5 when the survey is complete. Here, we provide an overview of the selection of spectroscopic targets, planning of observations, and analysis of data and data quality of BOSS.

We use radiative transfer to study the growth of ionized regions around the brightest, z=8 quasars in a large cosmological hydrodynamic simulation that includes black hole growth and feedback (the MassiveBlack simulation). We find that in the presence of the quasar s the comoving HII bubble radii reach 10 Mpc/h after 20 My while with the stellar component alone the HII bubbles are smaller by at least an order of magnitude. Our calculations show that several features are not captured within an analytical growth model of Stromgren spheres. The X-ray photons from hard quasar spectra drive a smooth transition from fully neutral to partially neutral in the ionization front. However the transition from partially neutral to fully ionized is significantly more complex. We measure the distance to the edge of bubbles as a function of angle and use the standard deviation of these distances as a diagnostic of the isotropy of ionized regions. We find that the overlapping of nearby ionized regions from clustered halos not only increases the anisotropy, but also is the main mechanism which allows the outer radius to grow. We therefore predict that quasar ionized bubbles at this early stage in the reionization process should be both significantly larger and more irregularly shaped than bubbles around star forming galaxies. Before the star formation rate increases and the Universe fully reionizes, quasar bubbles will form the most striking and recognizable features in 21cm maps.

We present the Data Release 9 Quasar (DR9Q) catalog from the Baryon Oscillation Spectroscopic Survey (BOSS) of the Sloan Digital Sky Survey III. The catalog includes all BOSS objects that were targeted as quasar candidates during the survey, are spectrocopically confirmed as quasars via visual inspection, have luminosities Mi[z=2]<-20.5 (in a $\Lambda$CDM cosmology with H0 = 70 km/s/Mpc, $\Omega_{\rm M}$ = 0.3, and $\Omega_{\Lambda}$ = 0.7) and either display at least one emission line with full width at half maximum (FWHM) larger than 500 km/s or, if not, have interesting/complex absorption features. It includes as well, known quasars (mostly from SDSS-I and II) that were reobserved by BOSS. This catalog contains 87,822 quasars (78,086 are new discoveries) detected over 3,275 deg$^{2}$ with robust identification and redshift measured by a combination of principal component eigenspectra newly derived from a training set of 8,632 spectra from SDSS-DR7. The number of quasars with $z>2.15$ (61,931) is ~2.8 times larger than the number of z>2.15 quasars previously known. Redshifts and FWHMs are provided for the strongest emission lines (CIV, CIII], MgII). The catalog identifies 7,533 broad absorption line quasars and gives their characteristics. For each object the catalog presents five-band (u,g,r,i,z) CCD-based photometry with typical accuracy of 0.03 mag, and information on the morphology and selection method. The catalog also contains X-ray, ultraviolet, near-infrared, and radio emission properties of the quasars, when available, from other large-area surveys.

The Sloan Digital Sky Survey III (SDSS-III) presents the first spectroscopic data from the Baryon Oscillation Spectroscopic Survey (BOSS). This ninth data release (DR9) of the SDSS project includes 535,995 new galaxy spectra (median z=0.52), 102,100 new quasar spectra (median z=2.32), and 90,897 new stellar spectra, along with the data presented in previous data releases. These spectra were obtained with the new BOSS spectrograph and were taken between 2009 December and 2011 July. In addition, the stellar parameters pipeline, which determines radial velocities, surface temperatures, surface gravities, and metallicities of stars, has been updated and refined with improvements in temperature estimates for stars with T_eff<5000 K and in metallicity estimates for stars with [Fe/H]>-0.5. DR9 includes new stellar parameters for all stars presented in DR8, including stars from SDSS-I and II, as well as those observed as part of the SDSS-III Sloan Extension for Galactic Understanding and Exploration-2 (SEGUE-2). The astrometry error introduced in the DR8 imaging catalogs has been corrected in the DR9 data products. The next data release for SDSS-III will be in Summer 2013, which will present the first data from the Apache Point Observatory Galactic Evolution Experiment (APOGEE) along with another year of data from BOSS, followed by the final SDSS-III data release in December 2014.

Using a new large-scale (~ 0.75 Gpc)^3 hydrodynamic cosmological simulation we investigate the growth rate of supermassive black holes in the early universe (z > 4.75). Remarkably, we find a clear peak in the typical Eddington ratio at black hole masses of 4-8 * 10^7 solar masses (typically found in halos of ~7 * 10^11 to 10^12 solar masses), independent of redshift and indicative that most of BH growth occurs in the cold-flow dominated regime. Black hole growth is by and large regulated by the evolution of gas density. The typical Eddington ratio at a given mass scales simply as cosmological density (1+z)^3 and the peak is caused by the competition between increased gas density available in more massive hosts, and a decrease due to strong AGN feedback that deprives the black hole of sufficient gas to fuel further rapid growth in the high mass end. In addition to evolution in the mean Eddington ratio, we show that the distribution of Eddington ratio among both mass-selected and luminosity-selected samples is approximately log-normal. We combine these findings into a single log-normal fitting formula for the distribution of Eddington ratios as a function of (M_BH,z). This formula can be used in analytic and semi analytic models for evolving black hole populations, predicting black hole masses of observed quasars, and, in conjunction with the observed distribution of Eddington ratios, can be used to constrain the black hole mass function.

We investigate the formation and properties of galaxies hosting z~6 quasars, in the gigaparsec scale cosmological hydrodynamical simulation: MassiveBlack, which includes a self-consistent model for star formation, black hole accretion and associated feedback. We show that the MassiveBlack reproduces current estimates of the galaxy stellar mass function z=5, 6. We find that quasar hosts in the simulation are compact gas rich systems with high star formations rates of SFR ~ 100-1000 Msun/yr consistent with observed properties of Sloan quasar hosts in the redshift range 5.5 < z < 6.5. We show that the star-forming gas in these galaxies predominantly originates from high density cold streams which are able to penetrate the halo and grow the galaxy at the center. MassiveBlack predicts a deviation from the local Mbh-sigma and Mbh-Mstar relation implying that black holes are relatively more massive for a given stellar host at these redshifts.

We use a large dark matter simulation of a LambdaCDM model to investigate the clustering and environmental dependence of the number of substructures in a halo. Focusing on redshift z=1, we find that the halo occupation distribution is sensitive at the tens of percent level to the surrounding density and to a lesser extent to asymmetry of the surrounding density distribution. We compute the autocorrelation function of halos as a function of occupation, building on the finding of Wechsler et al. (2006) and Gao and White (2007) that halos (at fixed mass) with more substructure are more clustered. We compute the relative bias as a function of occupation number at fixed mass, finding a strong relationship. At fixed mass, halos in the top 5% of occupation can have an autocorrelation function ~ 1.5-2 times higher than the mean. We also compute the bias as a function of halo mass, for fixed halo occupation. We find that for group and cluster sized halos, when the number of subhalos is held fixed, there is a strong anticorrelation between bias and halo mass. Such a relationship represents an additional challenge to the halo model.

Building on the legacy of the Sloan Digital Sky Survey (SDSS-I and II), SDSS-III is a program of four spectroscopic surveys on three scientific themes: dark energy and cosmological parameters, the history and structure of the Milky Way, and the population of giant planets around other stars. In keeping with SDSS tradition, SDSS-III will provide regular public releases of all its data, beginning with SDSS DR8 (which occurred in Jan 2011). This paper presents an overview of the four SDSS-III surveys. BOSS will measure redshifts of 1.5 million massive galaxies and Lya forest spectra of 150,000 quasars, using the BAO feature of large scale structure to obtain percent-level determinations of the distance scale and Hubble expansion rate at z<0.7 and at z~2.5. SEGUE-2, which is now completed, measured medium-resolution (R=1800) optical spectra of 118,000 stars in a variety of target categories, probing chemical evolution, stellar kinematics and substructure, and the mass profile of the dark matter halo from the solar neighborhood to distances of 100 kpc. APOGEE will obtain high-resolution (R~30,000), high signal-to-noise (S/N>100 per resolution element), H-band (1.51-1.70 micron) spectra of 10^5 evolved, late-type stars, measuring separate abundances for ~15 elements per star and creating the first high-precision spectroscopic survey of all Galactic stellar populations (bulge, bar, disks, halo) with a uniform set of stellar tracers and spectral diagnostics. MARVELS will monitor radial velocities of more than 8000 FGK stars with the sensitivity and cadence (10-40 m/s, ~24 visits per star) needed to detect giant planets with periods up to two years, providing an unprecedented data set for understanding the formation and dynamical evolution of giant planet systems. (Abridged)

Using a sample of approximately 14,000 z>2.1 quasars observed in the first year of the Baryon Oscillation Spectroscopic Survey (BOSS), we measure the three-dimensional correlation function of absorption in the Lyman-alpha forest. The angle-averaged correlation function of transmitted flux (F = exp(-tau)) is securely detected out to comoving separations of 60 Mpc/h, the first detection of flux correlations across widely separated sightlines. A quadrupole distortion of the redshift-space correlation function by peculiar velocities, the signature of the gravitational instability origin of structure in the Lyman-alpha forest, is also detected at high significance. We obtain a good fit to the data assuming linear theory redshift-space distortion and linear bias of the transmitted flux, relative to the matter fluctuations of a standard LCDM cosmological model (inflationary cold dark matter with a cosmological constant). At 95% confidence, we find a linear bias parameter 0.16<b<0.24 and redshift-distortion parameter 0.44<beta<1.20, at central redshift z=2.25, with a well constrained combination b(1+\beta)=0.336 +/- 0.012. The errors on beta are asymmetric, with beta=0 excluded at over 5 sigma confidence level. The value of beta is somewhat low compared to theoretical predictions, and our tests on synthetic data suggest that it is depressed (relative to expectations for the Lyman-alpha forest alone) by the presence of high column density systems and metal line absorption. These results set the stage for cosmological parameter determinations from three-dimensional structure in the Lyman-alpha forest, including anticipated constraints on dark energy from baryon acoustic oscillations.

The increasing size of cosmological simulations has led to the need for new visualization techniques. We focus on Smoothed Particle Hydrodynamical (SPH) simulations run with the GADGET code and describe methods for visually accessing the entire simulation at full resolution. The simulation snapshots are rastered and processed on supercomputers into images that are ready to be accessed through a web interface (GigaPan). This allows any scientist with a web-browser to interactively explore simulation datasets in both in spatial and temporal dimensions, datasets which in their native format can be hundreds of terabytes in size or more. We present two examples, the first a static terapixel image of the MassiveBlack simulation, a P-GADGET SPH simulation with 65 billion particles, and the second an interactively zoomable animation of a different simulation with more than one thousand frames, each a gigapixel in size. Both are available for public access through the GigaPan web interface. We also make our imaging software publicly available.

Observations of the most distant bright quasars imply that billion solar mass supermassive black holes (SMBH) have to be assembled within the first eight hundred million years. Under our standard galaxy formation scenario such fast growth implies large gas densities providing sustained accretion at critical or supercritical rates onto an initial black hole seed. It has been a long standing question whether and how such high black hole accretion rates can be achieved and sustained at the centers of early galaxies. Here we use our new cosmological hydrodynamic simulation (MassiveBlack) covering a volume (0.75 \Gpc)^3 appropriate for studying the rare first quasars to show that steady high density cold gas flows responsible for assembling the first galaxies produce the high gas densities that lead to sustained critical accretion rates and hence rapid growth commensurate with the existence of ~10^9 solar mass black holes as early as z~7. We find that under these conditions quasar feedback is not effective at stopping the cold gas from penetrating the central regions and hence cannot quench the accretion until the host galaxy reaches M_halo > 10^{12} solar masses. This cold-flow driven scenario for the formation of quasars implies that they should be ubiquitous in galaxies in the early universe and that major (proto)galaxy mergers are not a requirement for efficient fuel supply and growth, particularly for the earliest SMBHs.

We examine predictions for the quasar luminosity functions (QLF) and quasar clustering at high redshift (z > 4.75) using MassiveBlack, our new hydrodynamic cosmological simulation which includes a self-consistent model for black hole growth and feedback. We show that the model reproduces the Sloan QLF within observational constraints at z >= 5. We find that the high-z QLF is consistent with a redshift-independent occupation distribution of BHs among dark matter halos (which we provide) such that the evolution of the QLF follows that of the halo mass function. The sole exception is the bright-end at z=6 and 7, where BHs in high-mass halos tend to be unusually bright due to extended periods of Eddington growth caused by high density cold flows into the halo center. We further use these luminosity functions to make predictions for the number density of quasars in upcoming surveys, predicting there should be ~119+-28 (~87+-28) quasars detectable in the F125W band of the WIDE (DEEP) fields of the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS) from z=5-6, ~19+-7 (~18+-9) from z=6-7, and ~1.7+-1.5 (~1.5+-1.5) from z=7-8. We also investigate quasar clustering, finding that the correlation length is fully consistent with current constraints for Sloan quasars (r_0~17 h^{-1} Mpc at z=4 for quasars above m_i = 20.2), and grows slowly with redshift up to z=6 (r_0~22 h^{-1} Mpc). Finally, we note that the quasar clustering strength depends weakly on luminosity for low L_BH, but gets stronger at higher L_BH as the BHs are found in higher mass halos.

The SDSS-III Baryon Oscillation Spectroscopic Survey (BOSS), a five-year spectroscopic survey of 10,000 deg^2, achieved first light in late 2009. One of the key goals of BOSS is to measure the signature of baryon acoustic oscillations in the distribution of Ly-alpha absorption from the spectra of a sample of ~150,000 z>2.2 quasars. Along with measuring the angular diameter distance at z\approx2.5, BOSS will provide the first direct measurement of the expansion rate of the Universe at z > 2. One of the biggest challenges in achieving this goal is an efficient target selection algorithm for quasars over 2.2 < z < 3.5, where their colors overlap those of stars. During the first year of the BOSS survey, quasar target selection methods were developed and tested to meet the requirement of delivering at least 15 quasars deg^-2 in this redshift range, out of 40 targets deg^-2. To achieve these surface densities, the magnitude limit of the quasar targets was set at g <= 22.0 or r<=21.85. While detection of the BAO signature in the Ly-alpha absorption in quasar spectra does not require a uniform target selection, many other astrophysical studies do. We therefore defined a uniformly-selected subsample of 20 targets deg^-2, for which the selection efficiency is just over 50%. This "CORE" subsample will be fixed for Years Two through Five of the survey. In this paper we describe the evolution and implementation of the BOSS quasar target selection algorithms during the first two years of BOSS operations. We analyze the spectra obtained during the first year. 11,263 new z>2.2 quasars were spectroscopically confirmed by BOSS. Our current algorithms select an average of 15 z > 2.2 quasars deg^-2 from 40 targets deg^-2 using single-epoch SDSS imaging. Multi-epoch optical data and data at other wavelengths can further improve the efficiency and completeness of BOSS quasar target selection. [Abridged]

The Sloan Digital Sky Survey (SDSS) started a new phase in August 2008, with new instrumentation and new surveys focused on Galactic structure and chemical evolution, measurements of the baryon oscillation feature in the clustering of galaxies and the quasar Ly alpha forest, and a radial velocity search for planets around ~8000 stars. This paper describes the first data release of SDSS-III (and the eighth counting from the beginning of the SDSS). The release includes five-band imaging of roughly 5200 deg^2 in the Southern Galactic Cap, bringing the total footprint of the SDSS imaging to 14,555 deg^2, or over a third of the Celestial Sphere. All the imaging data have been reprocessed with an improved sky-subtraction algorithm and a final, self-consistent photometric recalibration and flat-field determination. This release also includes all data from the second phase of the Sloan Extension for Galactic Understanding and Evolution (SEGUE-2), consisting of spectroscopy of approximately 118,000 stars at both high and low Galactic latitudes. All the more than half a million stellar spectra obtained with the SDSS spectrograph have been reprocessed through an improved stellar parameters pipeline, which has better determination of metallicity for high metallicity stars.

We use a large N-body simulation to examine the detectability of HI in emission at redshift z ~ 1, and the constraints imposed by current observations on the neutral hydrogen mass function of galaxies at this epoch. We consider three different models for populating dark matter halos with HI, designed to encompass uncertainties at this redshift. These models are consistent with recent observations of the detection of HI in emission at z ~ 0.8. Whilst detection of 21 cm emission from individual halos requires extremely long integrations with existing radio interferometers, such as the Giant Meter Radio Telescope (GMRT), we show that the stacked 21 cm signal from a large number of halos can be easily detected. However, the stacking procedure requires accurate redshifts of galaxies. We show that radio observations of the field of the DEEP2 spectroscopic galaxy redshift survey should allow detection of the HI mass function at the 5-12 sigma level in the mass range 10^(11.4) M_sun/h < M_halo < 10^(12.5)M_sun/h, with a moderate amount of observation time. Assuming a larger noise level that corresponds to an upper bound for the expected noise for the GMRT, the detection significance for the HI mass function is still at the 1.7-3 sigma level. We find that optically undetected satellite galaxies enhance the HI emission profile of the parent halo, leading to broader wings as well as a higher peak signal in the stacked profile of a large number of halos. We show that it is in principle possible to discern the contribution of undetected satellites to the total HI signal, even though cosmic variance limitation make this challenging for some of our models.

We use the number density of peaks in the smoothed cosmological density field taken from the 2dF Galaxy Redshift Survey to constrain parameters related to the power spectrum of mass fluctuations, n (the spectral index), dn/d(lnk) (rolling in the spectral index), and the neutrino mass, m_nu. In a companion paper we use N-body simulations to study how the peak density responds to changes in the power spectrum, the presence of redshift distortions and the relationship between galaxies and dark matter halos. In the present paper we make measurements of the peak density from 2dF Galaxy Redshift Survey data, for a range of smoothing filter scales from 4-33 h^-1 Mpc. We use these measurements to constrain the cosmological parameters, finding n=1.36 (+0.75)(-0.64), m_nu < 1.76 eV, dn/d(lnk)=-0.012 (+0.192)(-0.208), at the 68 % confidence level, where m_nu is the total mass of three massive neutrinos. At 95% confidence we find m_nu< 2.48 eV. These measurements represent an alternative way to constrain cosmological parameters to the usual direct fits to the galaxy power spectrum, and are expected to be relatively insensitive to non-linear clustering evolution and galaxy biasing.

(Abridged) A consequence of the Earth's motion with respect to the CMB is that over a 10 year period it will travel a distance of ~800 AU. As first noted by Kardashev in 1986, this baseline can be used to carry out astrometric measurements of quasar parallaxes, so that only microarcsecond precision is necessary to detect parallax shifts of objects at gigaparsec distances. Such precision will soon be approached with the launch of the astrometric satellites Gaia and SIM. We use a Fisher matrix formalism to investigate the constraints that these and future missions may be able to place on the cosmological distance scale and dark energy. We find that by observing around a million quasars as planned, an extended 10 year Gaia mission could detect quasar parallax shifts at the 2.8 sigma level and so measure the Hubble constant to within 25 km/s. For the interferometer SIMLite, a Key Project using 2.4 % of the total mission time to observe 750 quasars could detect the effect at the 2 sigma level. Gaia and a dedicated SIMLite only weakly constrain the presence of a cosmological constant at the ~1 sigma levels. We also investigate future mission concepts, such as an interferometer similar in scope and design to NASA's Terrestrial Planet Finder. This could in principle measure the dark energy parameters w_0 and w_a with high precision, yielding a Figure of Merit larger than the stage IV experiments considered by the the Dark Energy Task Force. Unlike perhaps all other probes of dark energy there appear to be no obvious astrophysical sources of systematic error. There is however uncertainty regarding the statistical errors. As well as measurement error, there will be small additional contributions from image centroiding of variable sources, quasar peculiar motions and weak microlensing by stars along the line of sight.

We explore galaxy properties and their link with environment and clustering using a population of ~1000 galaxies formed in a high resolution hydrodynamic simulation of the Lambda CDM cosmology. At the redshift we concentrate on, z=1, the spatial resolution is 1.4 proper kpc/h and Milky-way sized disk galaxies contain ~10^5 particles within their virial radii. We include supermassive black hole accretion and feedback as well as a multiphase model for star formation. We find that a number of familiar qualitative relationships hold approximately between galaxy properties, for example, galaxies lie between two broad extremes of type, where ``late'' types tend to be smaller in size, have lower circular velocities, younger stars, higher star formation rates, larger disk to bulge ratios and lower Sersic indices than ``early types''. As in previous studies the stellar component of disk galaxies is not as rotationally supported as in observations. Bulges contain too much of the stellar mass, although disks do have scale lengths compatible with observations. The addition of black hole physics to the simulations does not appear to have an impact on the angular momentum results, nor do we find that it is affected in an identical simulation with significantly lower mass resolution. Despite this, we can profitably use the rank order of either disk to total ratio, Sersic index, or galaxy age to separate galaxies into morphological classes and examine the density-morphology relation and morphology dependence of clustering. We find that while at redshift z=0, the well known preponderance of early types in dense environments is seen, at z=1 the density-morphology relation becomes flatter and late type galaxies are even seen to have a higher clustering amplitude than early types (abridged).

We introduce SPHRAY, a Smoothed Particle Hydrodynamics (SPH) ray tracer designed to solve the 3D, time dependent, radiative transfer (RT) equations for arbitrary density fields. The SPH nature of SPHRAY makes the incorporation of separate hydrodynamics and gravity solvers very natural. SPHRAY relies on a Monte Carlo (MC) ray tracing scheme that does not interpolate the SPH particles onto a grid but instead integrates directly through the SPH kernels. Given initial conditions and a description of the sources of ionizing radiation, the code will calculate the non-equilibrium ionization state (HI, HII, HeI, HeII, HeIII, e) and temperature (internal energy/entropy) of each SPH particle. The sources of radiation can include point like objects, diffuse recombination radiation, and a background field from outside the computational volume. The MC ray tracing implementation allows for the quick introduction of new physics and is parallelization friendly. A quick Axis Aligned Bounding Box (AABB) test taken from computer graphics applications allows for the acceleration of the raytracing component. We present the algorithms used in SPHRAY and verify the code by performing all the test problems detailed in the recent Radiative Transfer Comparison Project of Iliev et. al. The Fortran 90 source code for SPHRAY and example SPH density fields are made available on a companion website (www.sphray.org).

The structures produced during the epoch of reionization by the action of radiation on neutral hydrogen are in principle different to those that arise through gravitational growth of initially small perturbations. We explore the difference between the two mechanisms using high resolution cosmological radiative transfer. Our computations use a Monte Carlo code which raytraces directly through SPH kernels without a grid, preserving the high spatial resolution of the underlying hydrodynamic simulation. Because the properties of the first sources of radiation are uncertain, we simulate a range of models with different source properties and recombination physics. We examine the morphology of the neutral hydrogren distribution and the reionization history in these models. We find that at fixed mean neutral fraction, structures are visually most affected by the existence of a lower limit in source luminosity, then by galaxy mass to light ratio, and are minimally affected by changes in the recombination rate and amplitude of mass fluctuations. We concentrate on the autocorrelation function of the neutral hydrogen, xi_HI(r) as a basic quantitive measure of Radiation Induced Structure (RIS.) All the models we test exhibit a characteristic behaviour, with xi_HI(r) becoming initially linearly antibiased with respect to the matter correlation function, reaching a minimum bias factor b~0.5 when the universe is ~10-20% ionized. After this xi_HI(r) increases rapidly in amplitude, overtaking the matter correlation function. It keeps a power law shape, but flattens considerably, reaching an asymptotic logarithmic slope of gamma ~-0.5. The growth rate of HI fluctuations is exponentially more rapid than gravitational growth over a brief interval of redshift Deltaz ~ 2-3.

The UV radiation from a quasar leaves a characteristic pattern in the distribution of ionized hydrogen throughout the surrounding space. This pattern or light echo propagates through the intergalactic medium at the speed of light, and can be observed by its imprint on the Ly-alpha forest spectra of background sources. As the echo persists after the quasar has switched off, it offers the possibility of searching for dead quasars, and constraining their luminosities and lifetimes. We outline a technique to search for and characterize these light echoes. To test the method, we create artificial Ly-alpha forest spectra from cosmological simulations at z=3, apply light echoes and search for them. We show how the simulations can also be used to quantify the significance level of any detection. We find that light echoes from the brightest quasars could be found in observational data. With absorption line spectra of 100 redshift z~3-3.5 quasars or galaxies in a 1 square degree area, we expect that ~10 echoes from quasars with B band luminosities L_B=3x10^45 ergs/s exist that could be found at 95% confidence, assuming a quasar lifetime of ~10^7 yr. Even a null result from such a search would have interesting implications for our understanding of quasar luminosities and lifetimes.

Using cosmological hydrodynamic simulations we measure the mean transmitted flux in the Lyman alpha forest for quasar sightlines that pass near a foreground quasar. We find that the trend of absorption with pixel-quasar separation distance can be fitted using a simple power law form including the usual correlation function parameters r_{0} and \gamma so that (<F(r)> = \sum exp(-tau_eff*(1+(r/r_{0})^(-\gamma)))). From the simulations we find the relation between r_{0} and quasar mass and formulate this as a way to estimate quasar host dark matter halo masses, quantifying uncertainties due to cosmological and IGM parameters, and redshift errors. With this method, we examine data for ~3000 quasars from the Sloan Digital Sky Survey (SDSS) Data Release 3, assuming that the effect of ionizing radiation from quasars (the so-called transverse proximity effect) is unimportant (no evidence for it is seen in the data.) We find that the best fit host halo mass for SDSS quasars with mean redshift z=3 and absolute G band magnitude -27.5 is log10(M/M_sun) = 12.48^{+0.53}_{-0.89}. We also use the Lyman-Break Galaxy (LBG) and Lyman alpha forest data of Adelberger et al in a similar fashion to constrain the halo mass of LBGs to be log10(M/M_sun) = 11.13^{+0.39}_{-0.55}, a factor of ~20 lower than the bright quasars. In addition, we study the redshift distortions of the Lyman alpha forest around quasars, using the simulations. We use the quadrupole to monopole ratio of the quasar-Lyman alpha forest correlation function as a measure of the squashing effect. We find that this does not have a measurable dependence on halo mass, but may be useful for constraining cosmic geometry.

We investigate the possibility of probing dark energy by measuring the isotropy of the galaxy cluster autocorrelation function (an Alcock-Paczynski test). The correlation function is distorted in redshift space because of the cluster peculiar velocities, but if these are known and can be subtracted, the correlation function measurement becomes in principle a pure test of cosmic geometry. Galaxy cluster peculiar velocities can be measured using the kinetic Sunyaev Zel'dovich (kSZ) effect. Upcoming CMB surveys, e.g., ACT, SPT, Planck, are expected to do this with varying levels of accuracy, dependent on systematic errors due to cluster temperature measurements, radio point sources, and the primary CMB anisotropy. We use the Hubble volume N-body simulation and the hydrodynamic simulation results of Nagai et. al (2003) to simulate various kSZ surveys. We find by model fitting that a measurement of the correlation function distortion can be used to recover the cosmological parameters that have been used to generate the simulation. However, the low space density of galaxy clusters requires larger surveys than are taking place at present to place tight constraints on cosmology. For example, with the SPT and ACT surveys, Omega_Lambda could be measured to within 0.1 and 0.2 respectively at one sigma, but only upper limits on the equation of state parameter w will be possible. Nevertheless, with accurate measurements of the kSZ effect, this test can eventually be used to probe the dark energy equation of state and its evolution with redshift, with different systematic errors than other methods.

Alignments of galaxy clusters (the Binggeli effect), as well as of galaxies themselves have long been studied both observationally and theoretically. Here we test the influence of large-scales structures and tidal fields on the shapes and alignments of cluster-size and galaxy-size dark matter halos. We use a high-resolution N-body simulation of a $\Lambda$CDM universe, together with the results of Colberg et al. (2005), who identified filaments connecting pairs of clusters. We find that cluster pairs connected by a filament are strongly aligned with the cluster-cluster axis, whereas unconnected ones are not. For smaller, galaxy-size halos, there also is an alignment signal, but its strength is independent of whether the halo is part of an obvious large-scale structure. Additionally, we find no measureable dependence of galaxy halo shape on membership of a filament. We also quantify the influence of tidal fields and find that these do correlate strongly with alignments of halos. The alignments of most halos are thus caused by tidal fields, with cluster-size halos being strongly aligned through the added mechanism of infall of matter from filaments.

The intergalactic neutral hydrogen which is responsible for the Lyman alpha forest of quasar absorption is a tracer of much larger amounts of ionised hydrogen. The ionised component has yet to be detected directly, but is expected to scatter CMB photons via the Sunyaev-Zel'dovich (SZ) effect. We use hydrodynamic simulations of a LambdaCDM universe to create mock quasar spectra and CMB sky maps. We find that the high-z Lya forest gas causes temperature fluctuations of the order of 1 muK rms in the CMB on arcmin scales. The kinetic and thermal SZ effects have a similar magnitude at z=3, with the thermal effect becoming relatively weaker as expected at higher z. The CMB signal associated with lines of sight having HI column densities > 10^18 cm^-2 is only marginally stronger than that for lower column densities. The strong dependence of rms temperature fluctuation on mean Lya absorbed flux, however, suggests that the CMB signal effectively arises in lower density material. We investigate the use of the cross-correlation of the Lya forest and the microwave background to detect the SZ effect at redshifts 2-4. In so doing we are able to set direct limits on the density of diffuse ionised intergalactic baryons. We carry out a preliminary comparison at a mean redshift z=3 of 3488 quasar spectra from SDSS Data Release 3 and the WMAP first year data. Assuming that the baryons are clustered as in a LambdaCDM cosmology, and have the same mean temperature, the cross-correlation yields a weak limit on the cosmic density of ionised baryons Omega_(b,I), which is Omega_(b,I) < 0.8 at 95% confidence. With data from upcoming CMB telescopes, we anticipate that a direct detection of the high redshift ionised IGM will soon be possible, providing an important consistency check on cosmological models.

X-ray absorption lines of highly-ionized species such as OVII at about zero redshift have been firmly detected in the spectra of several active galactic nuclei. However, the location of the absorbing gas remains a subject of debate. To separate the Galactic and extragalactic contributions to the absorption, we have obtained Chandra LETG-HRC and FUSE observations of the black hole X-ray binary LMC X--3. A joint analysis of the detected OVII and Ne IX Kalpha lines, together with the non-detection of the OVII Kbeta and OVIII Kalpha lines, gives the measurements of the temperature, velocity dispersion, and hot oxygen column density. The X-ray data also allow us to place a 95% confidence lower limit to the Ne/O ratio as 0.14. The OVII line centroid and its relative shift from the Galactic OI Kalpha absorption line, detected in the same observations, are inconsistent with the systemic velocity of LMC X--3 ($+310 {\rm km s^{-1}}$). The far-UV spectrum shows OVI absorption at Galactic velocities, but no OVI absorption is detected at the LMC velocity at $> 3\sigma$ significance. Both the nonthermal broadening and the decreasing scale height with the increasing ionization state further suggest an origin of the highly-ionized gas in a supernova-driven galactic fountain. In addition, we estimate the warm and hot electron column densities from our detected OVII Kalpha line in the LMC X--3 X-ray spectra and from the dispersion measure of a pulsar in the LMC vicinity. We then infer the O/H ratio of the gas to be $\gtrsim 8 \times 10^{-5}$, consistent with the chemically-enriched galactic fountain scenario. We conclude that the Galactic hot interstellar medium should in general substantially contribute to zero-redshift X-ray absorption lines in extragalactic sources.

We study the soft X-ray emission (0.1 - 1 keV) from the Warm-Hot Intergalactic Medium (WHIM) in a hydrodynamic simulation of a Cold Dark Matter universe. Our main goal is to investigate how such emission can be explored with a combination of imaging and spectroscopy, and to motivate future X-ray missions. We first present high resolution images of the X-ray emission in several energy bands, in which emission from different ion species dominates. We pick three different areas to study the high resolution spectra of X-rays from the warm-hot IGM: (a) a galaxy group; (b) a filament and (c) an underluminous region. By taking into account the background X-ray emission from AGNs and foreground emission from the Galaxy, we compute composite X-ray spectra of the selected regions. We briefly investigate angular clustering of the soft-X-ray emission, finding a strong signal. Most interestingly, the combination of high spectral resolution and angular information allows us to map the emission from the WHIM in 3 dimensions. We cross-correlate the positions of galaxies in the simulation with this redshift map of emission and detect the presence of six different ion species (Ne IX, Fe XVII, O VII, O VIII, N VII, C VI) in the large-scale structure traced by the galaxies. Finally we show how such emission can be detected and studied with future X-ray satellites, with particular attention to a proposed mission, the Missing Baryon Explorer, or MBE. We present simulated observations of the WHIM gas with MBE.

We predict the amplitude of the gravitational redshift of galaxies in galaxy clusters using an N-body simulation of a Lambda CDM universe. We examine if it might be possible to detect the gravitational effect on the total redshift observed for galaxies. For clusters of mass M ~10^15 m_sun, the difference in gravitational redshift between the brightest galaxy and the rest of the cluster members is ~10 km/s. The most efficient way to detect gravitational redshifts using information from galaxies only involves using the full gravitational redshift profile of clusters. Massive clusters, while having the largest gravitational redshift suffer from large galaxy peculiar velocities and substructure, which act as a source of noise. This and their low number density make it more reasonable to try averaging over many clusters and groups of relatively low mass. We examine publicly available data for 107 rich clusters from the ESO Nearby Abell Clusters Survey (ENACS), finding no evidence for gravitational redshifts. Tests on our simulated clusters show that we need at least ~2500 clusters/groups with M > 5*10^13 m_sun for a detection of gravitational redshifts at the 2 sigma level.

We investigate the large-scale inhomogeneities of the hydrogen ionizing radiation field in the Universe at redshift z=3. Using a raytracing algorithm, we simulate a model in which quasars are the dominant sources of radiation. We make use of large scale N-body simulations of a LambdaCDM universe, and include such effects as finite quasar lifetimes and output on the lightcone, which affects the shape of quasar light echoes. We create Lya forest spectra that would be generated in the presence of such a fluctuating radiation field, finding that the power spectrum of the Lya forest can be suppressed by as much as 15 % for modes with k=0.05-1 Mpc/h. This relatively small effect may have consequences for high precision measurements of the Lya power spectrum on larger scales than have yet been published. We also investigate another radiation field probe, the cross-correlation of quasar positions and the Lya forest. For both quasar lifetimes which we simulate (10^7 yr and 10^8 yr), we expect to see a strong decrease in the Lya absorption close to other quasars (the ``foreground'' proximity effect). We then use data from the Sloan Digital Sky Survey First Data Release to make an observational determination of this statistic. We find no sign of our predicted lack of absorption, but instead increased absorption close to quasars. If the bursts of radiation from quasars last on average < 10^6 yr, then we would not expect to be able to see the foreground effect. However, the strength of the absorption itself seems to be indicative of rare objects, and hence much longer total times of emission per quasar. Variability of quasars in bursts with timescales > 10^4yr and < 10^6 yr could reconcile these two facts.

We study the gas metallicity of quasar hosts using cosmological hydrodynamic simulations of the Lambda-cold dark matter model. Galaxy formation in the simulations is coupled with a prescription for black hole activity enabling us to study the evolution of the metal enrichment in quasar hosts and hence explore the relationship between star/spheroid formation and black hole growth/activity. We find a steep radial metallicity gradient in quasar host galaxies, with gas metallicities close to solar values in the outer parts but becoming supersolar in the center. The hosts of the rare bright quasars at z~5-6 have star formation rates of several hundred solar masses per year and halo masses of order ~10^12 solar masses. Already at these redshifts they have supersolar (Z ~2-3 solar) central metallicities, with a mild dependence of metallicity on luminosity, consistent with observed trends. The mean value of metallicity is sensitive to the assumed quasar lifetime, providing a useful new probe of this parameter. We find that lifetimes from 10^7-4x10^7yr are favored by comparison to observational data. In both the models and observations, the rate of evolution of the mean quasar metallicity as a function of redshift is generally flat out to z ~ 4-5. Beyond the observed redshift range and out to redshift z ~ 6-8, we predict a slow decline of the mean central metallicity towards solar and slightly subsolar values as we approach the epoch of the first significant star formation activity

(abridged) The observed properties of supermassive black holes suggest a fundamental link between their assembly and the formation of their host spheroids. We model the growth and activity of black holes in galaxies using LambdaCDM cosmological hydrodynamic simulations by following the evolution of the baryonic mass component in galaxy potential wells. We find that the observed steep relation between black hole mass and spheroid velocity dispersion, M_BH propto \sigma^4, is reproduced if the gas mass in bulges is linearly proportional to the black hole mass. In this model, black hole growth saturates because of the competition with star-formation and feedback, both of which determine the gas fraction available for accretion. Unless other processes also operate, we predict that the M_BH-sigma$ relation is not set in primordial structures but is fully established at low redshifts, $z \approxlt 2$, and is shallower at earlier times. We find that that central black hole masses are related to their dark matter halos simply via M_BH ~ M_DM^4/3. We assume that galaxies undergo a quasar phase with a typical lifetime, t_Q ~ 2\times 10^7 yr and show that star-formation regulated depletion of gas in spheroids can explain the decrease of the quasar population at redshift z<3 in the optical blue band. However, with the simplest assumption of a redshift independent quasar lifetime, the model overpredicts optical quasar numbers at high redshifts although it yields the observed evolution of number density of X-ray quasars over the redshift range 1 < z< 6. Finally, we find that the majority of black hole mass is assembled in galaxies by z ~ 3 and that the black hole accretion rate density peaks in rough correspondence to the star formation rate density at z ~ 4-5.

We measure the filling factor, correlation function, and power spectrum of transmitted flux in a large sample of Lya forest spectra, comprised of 30 Keck HIRES spectra and 23 Keck LRIS spectra. We infer the linear matter power spectrum P(k) from the flux power spectrum P_F(k), using an improved version of the method of Croft et al. (1998) that accounts for the influence of z-space distortions, non- linearity, and thermal broadening on P_F(k). The evolution of the shape and amplitude of P(k) over the range z= 2-4 is consistent with gravitational instability, implying that non-gravitational fluctuations do not make a large contribution. Our fiducial measurement of P(k) comes from data with <z> = 2.72. It has amplitude Delta^2(k_p)=0.74^0.20_-0.16 at wavenumber k_p=0.03 (km/s)^-1 and is well described by a power-law of index -2.43 +/- 0.06 or by a CDM-like power spectrum with shape parameter Gamma'=1.3^+0.7_-0.5*10^-3 (km/s) at z=2.72. For Omega_m=0.4, Omega_Lam=0.6, the best-fit Gamma =0.16 (h^-1mpc)^-1, in good agreement with the 2dF Galaxy Redshift Survey, and the best-fit sigma_8=0.82 (Gamma/0.15)^-0.44. Matching the observed cluster mass function and our Delta^2(k_p) in spatially flat models requires Omega_m=0.38^+0.10_-0.08 + 2.2 (Gamma-0.15). Matching Delta^2(k_p) in COBE-normalized, flat CDM models with no tensor fluctuations requires Omega_m = (0.29 +/-0.04) n^-2.89 h_65^-1.9. The Lya forest complements other probes of P(k) by constraining a regime of redshift and lengthscale not accessible by other means, and the consistency of these inferred parameters with independent estimates provides further support for inflation, cold dark matter, and vacuum energy (abridged).

We use non-radiative N-body/SPH simulations of structure formation in a LCDM cosmology to compare the angular momentum distributions of dark matter and gas in a large sample of halos. We show that the two components have identical spin parameter distributions and that their angular momentum distributions within individual halos are very similar, all in excellent agreement with standard assumptions. Despite these similarities, however, we find that the angular momentum vectors of the gas and dark matter are poorly aligned, with a median misalignment angle of about 30 degrees, which might have important implications for spin correlation statistics used in weak lensing studies. We present distributions for the component of the angular momentum that is aligned with the total angular momentum of each halo, and find that for between 5 and 50 percent of the mass this component is negative. This disagrees with the `Universal' angular momentum distribution suggested by Bullock et al. (2001), for which the mass fraction with negative specific angular momentum is zero. We discuss the implications of our results for the formation of disk galaxies. Since galactic disks generally do not contain counter-rotating stars or gas, disk formation cannot occur under detailed conservation of specific angular momentum. We suggest that the material with negative specific angular momentum combines with positive angular momentum material to build a bulge component, and show that in such a scenario the remaining material can form a disk with a density distribution that is very close to exponential.

We use a cosmological hydrodynamic simulation of a cold dark matter universe to investigate theoretically the relationship between high redshift galaxies and the Lyman=alpha forest at redshift z=3. Galaxies in the simulation are surrounded by halos of hot gas, which nevertheless contain enough neutral hydrogen to cause a Ly-alpha flux decrement, its strength increasing with galaxy mass. A comparison with recent observational data by Adelberger et. al on the Ly-alpha forest around galaxies reveals that actual galaxies may have systematically less Ly-alpha absorption within 1 Mpc of them than our simulated galaxies. In order to investigate this possibility, we add several simple prescriptions for galaxy feedback on the IGM to the evolved simulation. These include the effect of photoionizing background radiation coming from galactic sources, galactic winds whose only effect is to deposit thermal energy into the IGM, and another, kinetic model for winds, which are assumed to evacuate cavities in the IGM around galaxies. We find that only the latter is able to produce a large effect, enough to match the tentative observational data, given the energy available from star formation in the simulated galaxies. Another intriguing possibility is that a selection effect is responsible, so that galaxies with low Ly-alpha absorption are preferentially included in the sample. This is also viable, but predicts very different galaxy properties (including clustering) than the other scenarios.

Spherical density profiles and specific angular momentum profiles of Dark Matter halos found in cosmological N-body simulations have been measured extensively. The distribution of the total angular momentum of dark matter halos is also used routinely in semi-analytic modeling of the formation of disk galaxies. However, it is unclear whether the initial (i.e. at the time the halo is assembled) angular momentum distributions of baryons is related to the dark matter at all. Theoretical models for ellipticities in weak lensing studies often rely on an assumed correlation of the angular momentum vectors of dark matter and gas in galaxies. Both of these assumptions are shown to be in reasonable agreement with high resolution cosmological smoothed particle hydrodynamical simulations that follow the dark matter as long as only adiabatic gas physics are included. However, we argue that in more realistic models of galaxy formation one expects pressure forces to play a significant role at turn--around. Consequently the torquing force on DM and baryons will be uncorrelated and their respective angular momenta are not expected to align. An SPH simulation with ad-hoc feedback is presented that illustrates these effects. Massive low redshift elliptical galaxies may be a notable exception where "light may trace mass".

We present new perturbation theory (PT) predictions in the Spherical Collapse (SC) model for the 2-point moments of the large-scale distribution of dark matter density in the universe. We assume that these fluctuations grow under gravity from small Gaussian initial conditions. These predictions are compared with numerical simulations and with previous PT results to assess their domain of validity. We find that the SC model provides in practice a more accurate description of 2-point moments than previous tree-level PT calculations. The agreement with simulations is excellent for a wide range of scales (5-50 Mpc/h) and fluctuations amplitudes (0.02-2 variance). When normalized to unit variance these results are independent of the cosmological parameters and of the initial amplitude of fluctuations. The 2-point moments provide a convenient tool to study the statistical properties of gravitational clustering for fairly non-linear scales and complicated survey geometries, such as those probing the clustering of the Ly-alpha forest. In this context, the perturbative SC predictions presented here, provide a simple and novel way to test the gravitational instability paradigm.

We combine COBE-DMR measurements of cosmic microwave background anisotropy with a recent measurement of the mass power spectrum at redshift z=2.5 from Lya forest data to derive constraints on cosmological parameters and test the inflation+CDM scenario of structure formation. By treating the inflationary spectral index n as a free parameter, we can find successful fits to the COBE and Lya forest constraints in Omega_m=1 models with and without massive neutrinos and in low-Omega_m models with and without a cosmological constant. Within each class of model, the combination of COBE and the Lya forest P(k) constrains a parameter combination of the form (Omega_m h^a n^b Omega_b^c), with different indices for each case. This new constraint breaks some of the degeneracies in cosmological parameter determinations from other measurements. The Lya forest P(k) provides the first measurement of the slope of the linear mass power spectrum on ~Mpc scales, and it confirms a basic prediction of the inflationary CDM scenario: a nearly scale-invariant spectrum of primeval fluctuations (n~1) that bends towards k^{n-4} on small scales. Considering additional observational data, we find that COBE-normalized, Omega_m=1 models that match the Lya forest P(k) do not match the observed masses of rich galaxy clusters and that a low-Omega_m model with a cosmological constant provides the best overall fit, even without the direct evidence for cosmic acceleration from supernovae. Modest improvements in the Lya forest P(k) measurement could greatly restrict the allowable region of parameter space for CDM models, constrain the contribution of tensor fluctuations to CMB anisotropy, and achieve a more stringent test of the current consensus model of structure formation.

(Abridged) We use a hydrodynamic simulation of a LambdaCDM model to predict the extragalactic X-ray background (XRB), focussing on emission from the intergalactic medium (IGM). We also include X-rays from point sources associated with galaxies in the simulation, and make maps of the angular distribution of the emission. We find that filaments in the maps are not evident, being diluted by projection. In the soft (0.5-2 keV) band, the mean intensity of radiation from intergalactic and cluster gas is 2.3*10^-12 ergdeg^-2cm^-2s^-1, 35% of the total soft band emission. This is compatible at the ~1 sigma level with estimates of the unresolved soft background from ROSAT and {\it Chandra}. Only 4% of the hard (2-10 keV) emission is associated with the IGM. Relative to AGN flux, the IGM component peaks at a lower redshift (median z~0.45) so its clustering makes an important contribution to that of the total XRB. The angular correlations on 0.1-10 arcmin scales are significant, with an amplitude roughly consistent with an extrapolation of recent ROSAT results to small scales. A cross-correlation of the XRB against nearby galaxies taken from a simulated redshift survey also yields a strong signal from the IGM. Although some recent papers have argued that the expected soft band intensity from gas in galaxy, group, and cluster halos would exceed XRB limits unless much of the gas is expelled by supernova feedback, we obtain reasonable compatibility with current observations in a simulation that incorporates cooling, star formation, and only modest feedback. A prediction of our model is that the unresolved portion of the soft XRB will remain mostly unresolved.

We explore the possibility that an intrinsic correlation between galaxy ellipticities arising during the galaxy formation process may account for part of the shear signal recently reported by several groups engaged in weak lensing surveys. Using high resolution N-body simulations we measure the projected ellipticities of dark matter halos and their correlations as a function of pair separation. With this simplifying, but not necessarily realistic assumption (halo shapes as a proxy for galaxy shapes), we find a positive detection of correlations up to scales of at least 20 h^-1mpc (limited by the box size). The signal is not strongly affected by variations in the halo finding technique, or by the resolution of the simulations. We translate our 3d results into angular measurements of ellipticity correlation functions and shear variance which can be directly compared to observations. We also measure similar results from simulated angular surveys made by projecting our simulation boxes onto the plane of the sky and applying a radial selection function. Interestingly, the shear variance we measure is a small, but not entirely negligible fraction (from ~10-20 %) of that seen by the observational groups, and the ellipticity correlation functions approximately mimic the functional form expected to be caused by weak lensing. The amplitude depends on the width in redshift of the galaxy distribution. If photometric redshifts are used to pick out a screen of background galaxies with a small width, then the intrinsic correlation may become comparable to the weak lensing signal. Although we are dealing with simulated dark matter halos, whether there is a signal from real galaxies could be checked with a nearby sample with known redshifts.

Recently is has been suggested that the dearth of small halos around the Milky Way arises due to a modification of the primordial power spectrum of fluctuations from inflation. Such modifications would be expected to alter the formation of structure from bottom-up to top-down on scales near where the short-scale power has been suppressed. Using cosmological simulations we study the effects of such a modification of the initial power spectrum. While the halo multiplicity function depends primarily on the linear theory power spectrum, most other probes of power are more sensitive to the non-linear power spectrum. Collapse of large-scale structures as they go non-linear regenerates a ``tail'' in the power spectrum, masking small-scale modifications to the primordial power spectrum except at very high-z. Even the small-scale (k>2h/Mpc) clustering of the Ly-alpha forest is affected by this process, so that CDM models with sufficient power suppression to reduce the number of 10^10 Msun halos by a factor of about 5 give similar Ly-alpha forest power spectrum results. We conclude that other observations that depend more directly on the number density of collapsed objects, such as the number of damped Ly-alpha systems, or the redshift of reionization may provide the most sensitive tests of these models.

Using a high-resolution cosmological hydrodynamic simulation, we present a method to constrain extinction due to intergalactic grey dust based on the observed magnitudes of distant Type IA supernovae. We apply several simple prescriptions to relate the intergalactic dust density to the gas density in the simulation, thereby obtaining dust extinctions that may be directly compared to the observed distribution of supernova magnitudes. Our analysis is sensitive to the spatial distribution of grey dust, but is not dependent on its intrinsic properties such as its opacity or grain size. We present an application of our technique to the supernova data of Perlmutter et al., who find that their high redshift sample is ~0.2 magnitudes fainter than the expectation for a non-accelerating, low-density universe. We find that for grey dust to be responsible, it must be distributed quite smoothly, e.g., tracing intergalactic gas. More realistic dust distributions, such as dust tracing the metal density, are inconsistent with observations at the 1.5-2 sigma level. Upcoming observations and improved modelling of the dust distribution should lead to stronger constraints on intergalactic grey dust extinction.

We present predictions for the one-point probability distribution and cumulants of the transmitted QSO flux in the high redshift Lyman-$\alpha$ Forest. We make use of the correlation between the Lyman-$\alpha$ optical depth and the underlying matter density predicted by gravitational instability theory and seen in numerical hydrodynamic simulations. We have modelled the growth of matter fluctuations using the non-linear shear-free dynamics, an approximation which reproduces well the results of perturbation theory for the cumulants in the linear and weakly non-linear clustering regime. As high matter overdensities tend to saturate in spectra, the statistics of the flux distribution are dominated by weakly non-linear overdensities. As a result, our analytic approach can produce accurate predictions, when tested against N-body simulation results, even when the underlying matter field has rms fluctuations larger than unity. Our treatment can be applied to either Gaussian or non-Gaussian initial conditions. Here we concentrate on the former case, but also include a study of a specific non-Gaussian model. We discuss how the methods and predictions we present can be used as a tool to study the generic clustering properties of the \lya forest at high-redshift. With such an approach, rather than concentrating on simulating specific cosmological models, we may be in the position to directly test our assumptions for the Gaussian nature of the initial conditions, and the gravitational instability origin of structure itself. In a separate paper we present results for two-point statistics.

The Lya forest in quasar spectra probes scales where massive neutrinos can strongly suppress the growth of mass fluctuations. Using hydrodynamic simulations with massive neutrinos, we successfully test techniques developed to measure the mass power spectrum from the forest. A recent observational measurement in conjunction with a conservative implementation of other cosmological constraints places upper limits on the neutrino mass: m_nu < 5.5 eV for all values of Omega_m, and m_nu < 2.4 (Omega_m/0.17 -1) eV, if 0.2 < Omega_m <0.5 as currently observationally favored (both 95 % C.L.).

We outline the physical picture of the high-redshift Ly-alpha forest that has emerged from cosmological simulations, describe statistical characteristics of the forest that can be used to test theories of structure formation, present a preliminary comparison between simulation results and measurements from Keck HIRES spectra, and discuss a recent determination of the slope and amplitude of the linear mass power spectrum P(k) at z=2.5 from moderate resolution spectra. The physical picture is simple if each QSO spectrum is viewed as a continuous non-linear map of the line-of-sight density field rather than a collection of discrete absorption lines. The distribution of flux decrements depends mainly on the amplitude and PDF (Gaussian vs. non-Gaussian) of the primordial density fluctuations. The threshold crossing frequency, analogous to the 3-d genus curve, responds to the shape and amplitude of P(k) and to the values of Omega and Lambda. Open and Lambda-CDM models agree well with the measured flux decrement distribution at smoothing lengths of 25 km/s and 100 km/s and with the threshold crossing frequency at 100 km/s. Discrepancy with the observed threshold crossing frequency at 25 km/s may reflect the combined effects of noise in the data and limited mass resolution of the simulations. The slope of the measured P(k) agrees with the predictions of inflation+CDM models. Combining the amplitude with COBE normalization imposes a constraint on these models of the form Omega h^x n^y Omega_b^z = constant. Assuming Gaussian primordial fluctuations and a power spectrum shape parameter Gamma~0.2, consistency of the measured P(k) with the observed cluster mass function at z=0 requires Omega=0.46^{+0.12}_{-0.10} for an open universe and Omega=0.34^{+0.13}_{-0.09} for a flat universe (1-sigma errors). (Shortened)

We estimate the value of the matter density parameter \Omega_0 by combining constraints from the galaxy cluster mass function with Croft et al.'s recent measurement of the mass power spectrum, P(k), from \lya forest data. The key assumption of the method is that cosmic structure formed by gravitational instability from Gaussian primordial fluctuations. For a specified value of \Omega_0, matching the observed cluster mass function then fixes the value of \sigma_8, the rms amplitude of mass fluctuations in 8\hmpc spheres, and it thus determines the normalization of P(k) at z=0. The value of \Omega_0 also determines the ratio of P(k) at z=0 to P(k) at z=2.5, the central redshift of the \lya forest data; the ratio is different for an open universe (\Lambda=0) or a flat universe. Because the \lya forest measurement only reaches comoving scales 2\pi/k ~ 15-20\hmpc, the derived value of \Omega_0 depends on the value of the power spectrum shape parameter \Gamma, which determines the relative contribution of larger scale modes to \sigma_8. Adopting \Gamma=0.2, a value favored by galaxy clustering data, we find \Omega_0 = 0.46^{+0.12}_{-0.10} for an open universe and \Omega_0=0.34^{+0.13}_{-0.09} for a flat universe (1\sigma errors, not including the uncertainty in cluster normalization). Cluster-normalized models with \Omega_0=1 predict too low an amplitude for P(k) at z=2.5, while models with \Omega_0=0.1 predict too high an amplitude. The more general best fit parameter combination is approximately \Omega_0 + 0.2\Lambda_0 = 0.46 + 1.3(\Gamma-0.2). Analysis of larger, existing samples of QSO spectra could greatly improve the measurement of P(k) from the \lya forest, allowing a determination of \Omega_0 by this method with a precision of ~15%, limited mainly by uncertainty in the cluster mass function.

We measure the linear power spectrum of mass density fluctuations at redshift z=2.5 from the \lya forest absorption in a sample of 19 QSO spectra, using the method introduced by Croft et al. (1998). The P(k) measurement covers the range 2\pi/k ~ 450-2350 km/s (2-12 comoving \hmpc for \Omega=1). We examine a number of possible sources of systematic error and find none that are significant on these scales. In particular, we show that spatial variations in the UV background caused by the discreteness of the source population should have negligible effect on our P(k) measurement. We obtain consistent results from the high and low redshift halves of the data set and from an entirely independent sample of nine QSO spectra with mean redshift z=2.1. A power law fit to our measured P(k) yields a logarithmic slope n=-2.25 +/- 0.18 and an amplitude \Delta^2(k_p) = 0.57^{+0.26}_{-0.18}, where $\Delta^2$ is the contribution to the density variance from a unit interval of lnk and k_p=0.008 (km/s)^{-1}. Direct comparison of our mass P(k) to the measured clustering of Lyman Break Galaxies shows that they are a highly biased population, with a bias factor b~2-5. The slope of the linear P(k), never previously measured on these scales, is close to that predicted by models based on inflation and Cold Dark Matter (CDM). The P(k) amplitude is consistent with some scale-invariant, COBE-normalized CDM models (e.g., an open model with \Omega_0=0.4) and inconsistent with others (e.g., \Omega=1). Even with limited dynamic range and substantial statistical uncertainty, a measurement of P(k) that has no unknown ``bias factors'' offers many opportunities for testing theories of structure formation and constraining cosmological parameters. (Shortened)

We present a comparative study of six different methods for reversing the gravitational evolution of a cosmological density field to recover the primordial fluctuations: linear theory, the Gaussianization mapping scheme, two different quasi-linear dynamical schemes based on the Zel'dovich approximation, a Hybrid dynamical-Gaussianization method and the Path Interchange Zel'dovich Approximation (PIZA). The final evolved density field from an N-body simulation constitutes our test case. We use a variety of statistical measures to compare the initial density field recovered from it to the true initial density field, using each of the six different schemes. These include point-by-point comparisons of the density fields in real space, the individual modes in Fourier space, as well as global statistical properties such as the genus, the PDF of the density, and the distribution of peak heights and their shapes. We find linear theory to be the most inaccurate of all the schemes. The Gaussianization scheme is the least accurate after linear theory. The two quasi-linear dynamical schemes are more accurate than Gaussianization, although they break down quite drastically when used outside their range of validity - the quasi-linear regime. The complementary beneficial aspects of the dynamical and the Gaussianization schemes are combined in the Hybrid method. We find this Hybrid scheme to be more accurate and robust than either Gaussianization or the dynamical method alone. The PIZA scheme performs substantially better than the others in all point-by-point comparisons. However, it produces an oversmoothed initial density field, with a smaller number of peaks than expected, but recovers the PDF of the initial density with impressive accuracy on scales as small as 3Mpc/h.

A convincing physical picture for the Lyman-alpha forest has emerged from simulations and related semi-analytic studies of structure formation models. Observations can be be used in the context of this picture to study cosmology using the structure of the forest. With the availability of well motivated predictions, not only has it become possible to test models directly, but the physical processes involved appear to be simple enough that we can attempt to reconstruct aspects of the underlying cosmology from observations. We briefly summarise the method of Croft et al (1997) (astro-ph/9708018) for recovering the primordial mass power spectrum from Lyman-alpha forest data, emphasising the physical reasons that the derived P(k) is independent of unknown "bias factors". We present an illustrative application of the method to four quasar "spectra" reconstructed from published line lists.

(abridged) We estimate the cluster-galaxy cross-correlation function (Xi_cg), from the APM galaxy and galaxy cluster surveys, both in real space from the inversion of projected statistics and in redshift space using the galaxy and cluster redshift samples. The amplitude of Xi_cg is found to be almost independent of cluster richness. At large separations, r >~5 h^-1 Mpc, Xi_cg has a similar shape to the galaxy-galaxy and cluster-cluster autocorrelation functions. Xi_cg in redshift space can be related to the real space Xi_cg by convolution with an appropriate velocity field model. Here we apply a spherical collapse model, which we have tested against N-body simulations, finding that it provides a surprisingly accurate description of the averaged infall velocity of matter into galaxy clusters. We use this model to estimate beta (Omega^{0.6}/b) and find that it tends to overestimate the true result in simulations by only ~10-30%. Application to the APM results yields beta=0.43 with beta < 0.87 at 95% confidence. We also compare the APM Xi_cg and galaxy autocorrelations to results from popular cosmological models and derive two independent estimates of the galaxy biasing expected as a function of scale. Both low and critical density CDM models require anti-biasing by a factor ~2 on scales r <~ 2 h^-1Mpc and an MDM model is consistent with a constant biasing factor on all scales. We use the velocity fields predicted from the different models to distort the APM real space cross-correlation function. Comparison with the APM redshift space Xi_cg yields an estimate of the value of Omega^0.6 needed in each model. Only the low Omega model is fully consistent with observations, with MDM marginally excluded at the ~2 sigma level.

Lyman-alpha forest lines in QSO spectra have typical widths of 20-50 km/s. Low column density absorbers in cosmological simulations are large, diffuse structures, and the Hubble flow across the spatially extended absorber is usually the dominant contribution to the width of its associated absorption line. Thermal broadening is unimportant over most of the spectrum, and peculiar velocities tend to make absorption features narrower rather than broader. As a consequence of Hubble flow broadening, there is a close relation between local Lyman-alpha optical depth and local neutral hydrogen density, which is well approximated by the Gunn-Peterson formula. The physics that governs the unshocked intergalactic medium leads to a tight correlation between the neutral hydrogen density and the underlying gas and dark matter overdensity. For many purposes, it is simpler to regard a Lyman-alpha forest spectrum as a continuous, non-linear map of the density field rather than a collection of discrete lines. This continuous field view of the Lyman-alpha forest can be applied to measurement of the baryon density parameter, testing of cosmological models, and robust determination of the shape and amplitude of the primordial mass power spectrum.

In gravitational instability models, there is a close relationship between most Lyman-alpha absorption and the matter density along the line of sight. Croft et al. (1997) have shown that it is possible to use this relationship with some additional assumptions (such as Gaussianity of the initial density) to recover the power spectrum of mass fluctuations, P(k), from QSO absorption spectra. The relative uniformity of the ionizing radiation background on the scales of interest is an additional assumption required by the technique. Here we examine the fluctuations in Lyman-alpha spectra caused if the ionizing background radiation is generated by discrete QSO sources. We present our results alongside the preliminary application of the P(k) recovery technique to the QSO Q1422+231. We find the ionizing background fluctuations to have an effect orders of magnitude smaller than the matter fluctuations at the scales on which we measure P(k) (lambda < 10 h^-1Mpc).

We present a method to recover the shape and amplitude of the power spectrum of mass fluctuations, P(k), from observations of the high redshift \lya forest. The method is motivated by the physical picture of the \lya forest that has emerged from hydrodynamic cosmological simulations and related semi-analytic models, which predicts a tight correlation between the \lya optical depth and the underlying matter density. We monotonically map the QSO spectrum to a Gaussian density field, measure its 3-d P(k), and normalize by evolving cosmological simulations with this P(k) until they reproduce the observed power spectrum of the QSO flux. Imposing the observed mean \lya opacity as a constraint makes the derived P(k) normalization insensitive to the choice of cosmological parameters, ionizing background spectrum, or reionization history. Thus, in contrast to estimates of P(k) from galaxy clustering, there are no uncertain "bias parameters" in the recovery of the mass power spectrum. We test the full procedure on SPH simulations of 3 cosmological models and show that it recovers their true mass power spectra on comoving scales ~1-10/h Mpc, the upper scale being set by the size of the simulation boxes. The procedure works even for noisy (S/N ~ 10), moderate resolution (~40 km/s pixels) spectra. We present an illustrative application to Q1422+231; the recovered P(k) is consistent with an \Omega=1, \sigma_8=0.5 CDM model. Application to large QSO samples should yield the power spectrum of mass fluctuations on small scales at z ~ 2-4. (Compressed)

Observations from the HUT and the HST have recently detected HeII absorption along the lines of sight to two high redshift quasars. We use cosmological simulations with gas dynamics to investigate HeII absorption in the cold dark matter (CDM) theory of structure formation. We consider two Omega=1 CDM models with different normalizations and one Omega_0=0.4 CDM model, all incorporating the photoionizing UV background spectrum computed by Haardt & Madau (1996). The simulated gas distribution, combined with the H&M spectral shape, accounts for the relative observed values of taubar_HI and taubar_HeII, the effective mean optical depths for HI and HeII absorption. If the background intensity is as high as H&M predict, then matching the absolute values of taubar_HI and taubar_HeII requires a baryon abundance larger (by factors between 1.5 and 3 for the various CDM models) than our assumed value of Omega_b h^2=0.0125. The simulations reproduce the evolution of taubar_heII over the observed redshift range, 2.2 < z < 3.3, if the HeII photoionization rate remains roughly constant. HeII absorption in the CDM simulations is produced by a diffuse, fluctuating, intergalactic medium, which also gives rise to the HI ly-alpha forest. Much of the HeII opacity arises in underdense regions where the HI optical depth is very low. We compute statistical properties of the HeII and HI absorption that can be used to test the CDM models and distinguish them from an alternative scenario in which the HeII absorption is caused by discrete, compact clouds. The CDM scenario predicts that a substantial amount of baryonic material resides in underdense regions at high redshift. HeII absorption is the only sensitive probe of such extremely diffuse, intergalactic gas, so it can provide a vital test of this fundamental prediction.

In gravitational instability models, \lya absorption arises from a continuous fluctuating medium, so that spectra provide a non-linear one-dimensional ``map'' of the underlying density field. We characterise this continuous absorption using statistical measures applied to the distribution of absorbed flux. We describe two simple members of a family of statistics which we apply to simulated spectra in order to show their sensitivity as probes of cosmological parameters (H$_{0}$, $\Omega$, the initial power spectrum of matter fluctuations) and the physical state of the IGM. We make use of SPH simulation results to test the flux statistics, as well as presenting a preliminary application to Keck HIRES data.

One way of recovering information about the initial conditions of the Universe is by measuring features of the cosmological density field which are preserved during gravitational evolution and galaxy formation. In this paper we study the total number density of peaks in a (galaxy) point distribution smoothed with a filter, evaluating its usefulness as a means of inferring the shape of the initial (matter) power spectrum. We find that in numerical simulations which start from Gaussian initial conditions, the peak density follows well that predicted by the theory of Gaussian density fields, even on scales where the clustering is mildly non-linear. For smaller filter scales, $r \simlt 4-6 \hmpc$, we see evidence of merging as the peak density decreases with time. On larger scales, the peak density is independent of time. One might also expect it to be fairly robust with respect to variations in biasing, i.e. the way galaxies trace mass fluctuations. We find that this is the case when we apply various biasing prescriptions to the matter distribution in simulations. If the initial conditions are Gaussian, it is possible to use the peak density measured from the evolved field to reconstruct the shape of the initial power spectrum. We describe a stable method for doing this and apply it to several biased and unbiased non-linear simulations. We are able to recover the slope of the linear matter power spectrum on scales $k \simlt 0.4 \hmpc^{-1}$. The reconstruction has the advantage of being independent of the cosmological parameters ($\Omega$, $\Lambda$, $H_0$) and of the clustering normalisation ($\sigma_8$). The peak density and reconstructed power spectrum slope therefore promise to be powerful discriminators between popular cosmological scenarios.

We analyse the spatial clustering properties of a new catalogue of very rich galaxy clusters selected from the APM Galaxy Survey. These clusters are of comparable richness and space density to Abell Richness Class $\geq 1$ clusters, but selected using an objective algorithm from a catalogue demonstrably free of artificial inhomogeneities. Evaluation of the two-point correlation function $\xi_{cc}(r)$ for the full sample and for richer subsamples reveals that the correlation amplitude is consistent with that measured for lower richness APM clusters and X-ray selected clusters. We apply a maxmimum likelihood estimator to find the best fitting slope and amplitude of a power law fit to $\xi_{cc}(r)$, and to estimate the correlation length $r_{0}$ (the value of $r$ at which $\xi_{cc}(r)$ is equal to unity). For clusters with a mean space density of $1.6\times 10^{-6}\hmpccc$ (equivalent to the space density of Abell Richness $\geq 2$ clusters), we find $r_{0}=21.3^{+11.1}_{-9.3} \hmpc$ (95% confidence limits). This is consistent with the weak richness dependence of $\xi_{cc}(r)$ expected in Gaussian models of structure formation. In particular, the amplitude of $\xi_{cc}(r)$ at all richnesses matches that of $\xi_{cc}(r)$ for clusters selected in N-Body simulations of a low density Cold Dark Matter model.

We present a method for reconstructing cosmological densityn for and velocity fields using the Lagrangian Zel'dovich formalism. . The method involves finding the least action solution for straight line particle paths in an evolving density field. Our starting point is the final, evolved density , so that we are in effect carrying out the standard Zel'dovich Approximation based process in reverse. Using a simple numerical algorithm we are able to minimise the action for the trajectories of several million particles. We apply our method to the evolved density taken from N-body simulations of different cold dark matter dominated universes, testing both the prediction for the present day velocity field and for the initial density field. The method is easy to apply, reproduces the accuracy of the forward Zel'dovich Approximation, and also works directly in redshift space with minimal modification.

We estimate the variance $\xibar_2$, the skewness $\xibar_3$ and the kurtosis $\xibar_4$ in the distribution of density fluctuations in a complete sample from the APM Cluster Redshift Survey with 339 clusters and a mean depth $ \sim 250\Mpc$. We are able to measure the statistics of fluctuations in spheres of radius $R \simeq 5-80 \Mpc$, with reasonable errorbars. The statistics in the cluster distribution follow the hierarchical pattern $\xibar_J=S_J~\xibar_2^{J-1}$ with $S_J$ roughly constant, $S_3 \simeq 2$ and $S_4 \sim 8$. We analyse the distribution of clusters taken from N-body simulations of different dark matter models. The results are compared with an alternative method of simulating clusters which uses the truncated Zel'dovich approximation. We argue that this alternative method is not reliable enough for making quantitative predictions of $\xibar$. The N-body simulation results follow similar hierarchical relations to the observations, with $S_J$ almost unaffected by redshift distortions from peculiar motions. The standard $\Omega=1$ Cold Dark Matter (CDM) model is inconsistent with either the second, third or fourth order statistics at all scales. However both a hybrid Mixed Dark Matter model and a low density CDM variant agree with the $\xibar_J$ observations.

We investigate the feasibility of carrying out likelihood analysis on the velocities of galaxy clusters to determine power spectrum parameters. Using N-body simulations of cosmological density fields we show that the velocity field traced by clusters is highly non-linear on small scales (r < 10 h^-1 Mpc), mostly due to strong infall into superclusters. The transverse and line of sight velocity correlation functions relevant to likelihood analysis deviate significantly from the linear theory values. Unfortunately, the sparseness of cluster samples means that they are not amenable to any rigorous smoothing procedure to rectify this. We are forced instead to remove close pairs of clusters which would contribute disproportionately to the likelihood. This reduces the size still further of the small samples of observed clusters available to us, forcing us into the conclusion that using straightfoward linear theory covariance analysis on clusters is unpromising. Nevertheless we apply the technique to two sets of observational data and find marginal evidence for more power in the mass density field than standard CDM. The normalisation is consistent with \Omega^{0.6}\sigma_{8} \sim 0.7 for both data sets, within the (large) errors.

We present new estimates of the spatial two-point correlation function of rich clusters of galaxies selected from the APM Galaxy Survey. We have measured redshifts for a sample of $364$ clusters out to a depth of $\sim 450\hmpc$. The clusters have a mean space density of $\bar{n} = 3.4\times 10^{-5}\hmpccc$. The two-point correlation function, $\xi_{cc}$, for this sample is equal to unity at a pair-separation of $r_0 = 14.3\pm1.75\hmpc$ (2$\sigma$ errors), consistent with our earlier results from a smaller sample. The new observations provide an accurate determination of the shape of $\xi_{cc}$ to pair-separations of about $50\hmpc$. Our results show that $\xi_{cc}$ has a higher amplitude than expected according to the standard $\Omega=1$ cold dark matter (CDM) model on spatial scales $2\simlt s \simlt 50\hmpc$, but are in good agreement with scale-invariant fluctuations in either a low density CDM model or a critical density universe made up of a mixture of hot and cold dark matter. Our results provide strong constraints on so called `co-operative' models of galaxy formation in which the galaxy formation process introduces large-scale structure in the galaxy distribution.

We use ensembles of high-resolution CDM simulations to investigate the shape and amplitude of the two point correlation function of rich clusters. The standard scale-invariant CDM model with $\Omega=1$ provides a poor description of the clustering measured from the APM rich cluster redshift survey, which is better fitted by models with more power at large scales. The amplitudes of the rich cluster correlation functions measured from our models depend weakly on cluster richness. Analytic calculations of the clustering of peaks in a Gaussian density field overestimate the amplitude of the N-body cluster correlation functions, but reproduce qualitatively the weak trend with cluster richness. Our results suggest that the high amplitude measured for the correlation function of richness class $R \geq 2$ Abell clusters is either an artefact arising from incompleteness in the Abell catalogue, or an indication that the density perturbations in the early universe were very non-Gaussian.

We use high resolution dissipationless N-body simulations to examine the spatial distribution of galaxy clusters on large scales. The Standard CDM model and two of its main competitors, Low density CDM and Mixed Dark Matter are compared.The two-point correlation function of simulated clusters is compared with an extended survey of APM clusters , and it is found that Standard CDM exhibits a lack of power on all scales, whereas the two alternative scenarios are able to match the spatial correlations well. Of the remaining two models,the velocities in the MDM universe have a higher amplitude and their distribution is much broader. We compare these peculiar velocities with observations and find that both models have difficulty in reproducing the observed numbers of very high peculiar velocity clusters. The reliable detection of several more clusters with velocities in excess of 1000 km/s would render the LCDM scenario in particular very unlikely.