We report the direct detection of the kinetic Sunyaev-Zel'dovich effect in galaxy clusters with a 3.8 sigma significance level. The measurement is performed by stacking the Planck map at 217 GHz at the positions of galaxy clusters from the Wen-Han-Liu (WHL) catalog. To avoid the cancellation of positive and negative kSZ signals, we use the large scale distribution of the SDSS galaxies to estimate the peculiar velocities of the galaxy clusters along the line-of-sight and incorporate the sign in the (velocity weighted) stacking of the kSZ signals. Using this technique, we are able to measure the kSZ signal around galaxy clusters beyond 3R500. Assuming a standard beta-model, we also find that the gas fraction within R500 is f(gas,500) = 0.13 +- 0.03 for the clusters with the mass of M500 ~ 0.9 * 10^14 Msun/h, and is slightly lower than the universal baryon fraction. We compare this result to predictions from the Magneticum cosmological hydrodynamic simulations as well as other kSZ and X-ray measurements, where most of them show a lower gas fraction than the universal value for the same mass of clusters.

Using the latest upper limits on the 21-cm power spectrum at $z\approx9.1$ from the Low Frequency Array (LOFAR), we explore regions of parameter space which are inconsistent with the data. We use 21CMMC, a Monte Carlo Markov Chain sampler of 21cmFAST which directly forward models the 3D cosmic 21-cm signal in a fully Bayesian framework. We use the astrophysical parameterisation from 21cmFAST, which includes mass-dependent star formation rates and ionising escape fractions as well as soft-band X-ray luminosities to place limits on the properties of the high-$z$ galaxies. Further, we connect the disfavoured regions of parameter space with existing observational constraints on the Epoch of Reionisation such as ultra-violet (UV) luminosity functions, background UV photoionisation rate, intergalactic medium (IGM) neutral fraction and the electron scattering optical depth. We find that all models exceeding the 21-cm signal limits set by LOFAR at $z\approx9.1$ are excluded at $\gtrsim2\sigma$ by other probes. Finally, we place limits on the IGM spin temperature from LOFAR, disfavouring at 95 per cent confidence spin temperatures below $\sim2.6$ K across an IGM neutral fraction range of $0.15 \lesssim \bar{x}_{H{\scriptscriptstyle I}} \lesssim 0.6$. Note, these limits are only obtained from 141 hrs of data in a single redshift bin. With tighter upper limits, across multiple redshift bins expected in the near future from LOFAR, more viable models will be ruled out. Our approach demonstrates the potential of forward modelling tools such as 21CMMC in combining 21-cm observations with other high-$z$ probes to constrain the astrophysics of galaxies.

The ARCADE2 and LWA1 experiments have claimed an excess over the Cosmic Microwave Background (CMB) at low radio frequencies. If the cosmological high-redshift contribution to this radio background is between 0.1% and 22% of the CMB at $1.42\,$GHz, it could explain the tentative EDGES Low-Band detection of the anomalously deep absorption in the 21-cm signal of neutral hydrogen (Fialkov & Barkana 2019). We use the upper limit on the 21-cm signal from the Epoch of Reionization ($z=9.1$) based on $141\,$hours of observations with LOFAR (Mertens et al. 2020) to evaluate the contribution of the high redshift Universe to the detected radio background. Marginalizing over astrophysical properties of star-forming halos, we find (at 68% C.L.) that the cosmological radio background can be at most 0.7% of the CMB at $1.42\,$GHz. This limit rules out strong contribution of the high-redshift Universe to the ARCADE2 and LWA1 measurements. Even though LOFAR places limit on the extra radio background, excess of $0.1-0.7$% over the CMB (at $1.42\,$GHz) is still allowed and could explain the EDGES Low-Band detection. We also constrain the thermal and ionization state of the gas at $z = 9.1$, and put limits on the properties of the first star-forming objects. We find that, in agreement with the limits from EDGES High-Band data, LOFAR data disfavour scenarios with inefficient X-ray sources and cases where the Universe was ionized by massive halos only.

A new upper limit on the 21-cm signal power spectrum at a redshift of $z \approx 9.1$ is presented, based on 141 hours of data obtained with the Low-Frequency Array (LOFAR). The analysis includes significant improvements in spectrally-smooth gain-calibration, Gaussian Process Regression (GPR) foreground mitigation and optimally-weighted power spectrum inference. Previously seen `excess power' due to spectral structure in the gain solutions has markedly reduced but some excess power still remains with a spectral correlation distinct from thermal noise. This excess has a spectral coherence scale of $0.25 - 0.45$\,MHz and is partially correlated between nights, especially in the foreground wedge region. The correlation is stronger between nights covering similar local sidereal times. A best 2-$\sigma$ upper limit of $\Delta^2_{21} < (73)^2\,\mathrm{mK^2}$ at $k = 0.075\,\mathrm{h\,cMpc^{-1}}$ is found, an improvement by a factor $\approx 8$ in power compared to the previously reported upper limit. The remaining excess power could be due to residual foreground emission from sources or diffuse emission far away from the phase centre, polarization leakage, chromatic calibration errors, ionosphere, or low-level radio-frequency interference. We discuss future improvements to the signal processing chain that can further reduce or even eliminate these causes of excess power.

We derive constraints on the thermal and ionization states of the intergalactic medium (IGM) at redshift $\approx$ 9.1 using new upper limits on the 21-cm power spectrum measured by the LOFAR radio-telescope and a prior on the ionized fraction at that redshift estimated from recent cosmic microwave background (CMB) observations. We have used results from the reionization simulation code GRIZZLY and a Bayesian inference framework to constrain the parameters which describe the physical state of the IGM. We find that, if the gas heating remains negligible, an IGM with ionized fraction $\gtrsim 0.13$ and a distribution of the ionized regions with a characteristic size $\gtrsim 8 ~h^{-1}$ comoving megaparsec (Mpc) and a full width at the half maximum (FWHM) $\gtrsim 16 ~h^{-1}$ Mpc is ruled out. For an IGM with a uniform spin temperature $T_{\rm S} \gtrsim 3$ K, no constraints on the ionized component can be computed. If the large-scale fluctuations of the signal are driven by spin temperature fluctuations, an IGM with a volume fraction $\lesssim 0.34$ of heated regions with a temperature larger than CMB, average gas temperature 7-160 K and a distribution of the heated regions with characteristic size 3.5-70 $h^{-1}$ Mpc and FWHM of $\lesssim 110$ $h^{-1}$ Mpc is ruled out. These constraints are within the 95 per cent credible intervals. With more stringent future upper limits from LOFAR at multiple redshifts, the constraints will become tighter and will exclude an increasingly large region of the parameter space.

In this paper we use radiative transfer + N-body simulations to explore the feasibility of measurements of cross-correlations between the 21cm field observed by the Square Kilometer Array (SKA) and high-z Lyman Alpha Emitters (LAEs) detected in galaxy surveys with the Subaru Hyper Supreme Cam (HSC), Subaru Prime Focus Spectrograph (PFS) and Wide Field Infrared Survey Telescope (WFIRST). 21cm-LAE cross-correlations are in fact a powerful probe of the epoch of reionization as they are expected to provide precious information on the progress of reionization and the typical scale of ionized regions at different redshifts. The next generation observations with SKA will have a noise level much lower than those with its precursor radio facilities, introducing a significant improvement in the measurement of the cross-correlations. We find that an SKA-HSC/PFS observation will allow to investigate scales below ~10 Mpc/h and ~60 Mpc/h at z=7.3 and 6.6, respectively. WFIRST will allow to access also higher redshifts, as it is expected to observe spectroscopically ~900 LAEs per square degree and unit redshift in the range 7.5<z<8.5. Because of the reduction of the shot noise compared to HSC and PFS, observations with WFIRST will result in more precise cross-correlations and increased observable scales.

We use hydrodynamics and radiative transfer simulations to study the 21~cm signal around a bright QSO at $z \sim 10$. Due to its powerful UV and X-ray radiation, the QSO quickly increases the extent of the fully ionized bubble produced by the pre-existing stellar type sources, in addition to partially ionize and heat the surrounding gas. As expected, a longer QSO lifetime, $t_{\rm QSO}$, results in a 21~cm signal in emission located at increasingly larger angular radii, $\theta$, and covering a wider range of $\theta$. Similar features can be obtained with a higher galactic emissivity efficiency, $f_{\rm UV}$, so that determining the origin of a large ionized bubble (i.e. QSO vs stars) is not straightforward. Such degeneracy could be reduced by taking advantage of the finite light traveltime effect, which is expected to affect an HII region produced by a QSO differently from one created by stellar type sources. From an observational point of view, we find that the 21 cm signal around a QSO at various $t_{\rm QSO}$ could be detected by SKA1-low with a high signal-noise ratio (S/N). As a reference, for $t_{\rm QSO} = 10\,\rm Myr$, a S/N $\sim 8$ is expected assuming that no pre-heating of the IGM has taken place due to high-$z$ energetic sources, while it can reach value above 10 in case of pre-heating. Observations of the 21~cm signal from the environment of a high-$z$ bright QSO could then be used to set constraints on its lifetime, as well as to reduce the degeneracy between $f_{\rm UV}$ and $t_{\rm QSO}$.

The intergalactic medium (IGM) plays an important role in the formation and evolution of galaxies. Recent developments in upcoming radio telescopes are starting to open up the possibility of making a first direct detection of the 21 cm signal of neutral hydrogen (HI) from the warm gas of the IGM in large-scale filaments. The cosmological hydrodynamical EAGLE simulation is used to estimate the typical IGM filament signal. Assuming the same average signal for all filaments, a prediction is made for the detectability of such a signal with the upcoming mid-frequency array of the Square Kilometer Array (SKA1-mid) or the future upgrade to SKA2. The signal-to-noise (S/N) then only depends on the size and orientation of each filament. With filament spines inferred from existing galaxy surveys as a proxy for typical real filaments, we find hundreds of filaments in the region of the sky accessible to the SKA that can be detected. Once the various phases of the SKA telescope become operational, their own surveys will be able to find the galaxies required to infer the position of even more filaments within the survey area. We find that in 120 h, SKA1-mid/SKA2 will detect HI emission from the strongest filaments in the field with a S/N of the order of 10 to $\sim$150 for the most pessimistic model considered here. Some of the brighter filaments can be detected with an integration time of a few minutes with SKA1-mid and a few seconds with SKA2. Therefore, SKA2 will be capable of not only detecting but also mapping a large part of the IGM in these filaments.

We use analytic computations to predict the power spectrum as well as the bispectrum of Cosmic Infrared Background (CIB) anisotropies. Our approach is based on the halo model and takes into account the mean luminosity-mass relation. The model is used to forecast the possibility to simultaneously constrain cosmological, CIB and halo occupation distribution (HOD) parameters in the presence of foregrounds. For the analysis we use wavelengths in eight frequency channels between 200 and 900$\;\mathrm{GHz}$ with survey specifications given by Planck and LiteBird. We explore the sensitivity to the model parameters up to multipoles of $\ell =1000$ using auto- and cross-correlations between the different frequency bands. With this setting, cosmological, HOD and CIB parameters can be constrained to a few percent. Galactic dust is modeled by a power law and the shot noise contribution as a frequency dependent amplitude which are marginalized over. We find that dust residuals in the CIB maps only marginally influence constraints on standard cosmological parameters. Furthermore, the bispectrum yields tighter constraints (by a factor four in $1\sigma$ errors) on almost all model parameters while the degeneracy directions are very similar to the ones of the power spectrum. The increase in sensitivity is most pronounced for the sum of the neutrino masses. Due to the similarity of degeneracies a combination of both analysis is not needed for most parameters. This, however, might be due to the simplified bias description generally adopted in such halo model approaches.

Neutral hydrogen pervades the infant Universe, and its redshifted 21-cm signal allows one to chart the Universe. This signal allows one to probe astrophysical processes such as the formation of the first stars, galaxies, (super)massive black holes and enrichment of the pristine gas from z~6 to z~30, as well as fundamental physics related to gravity, dark matter, dark energy and particle physics at redshifts beyond that. As one enters the Dark Ages (z>30), the Universe becomes pristine. Ground-based low-frequency radio telescopes aim to detect the spatial fluctuations of the 21-cm signal. Complementary, global 21-cm experiments aim to measure the sky-averaged 21-cm signal. Escaping RFI and the ionosphere has motivated space-based missions, such as the Dutch-Chinese NCLE instrument (currently in lunar L2), the proposed US-driven lunar or space-based instruments DAPPER and FARSIDE, the lunar-orbit interferometer DSL (China), and PRATUSH (India). To push beyond the current z~25 frontier, though, and measure both the global and spatial fluctuations (power-spectra/tomography) of the 21-cm signal, low-frequency (1-100MHz; BW~50MHz; z>13) space-based interferometers with vast scalable collecting areas (1-10-100 km2), large filling factors (~1) and large fields-of-view (4pi sr.) are needed over a mission lifetime of >5 years. In this ESA White Paper, we argue for the development of new technologies enabling interferometers to be deployed, in space (e.g. Earth-Sun L2) or in the lunar vicinity (e.g. surface, orbit or Earth-Moon L2), to target this 21-cm signal. This places them in a stable environment beyond the reach of most RFI from Earth and its ionospheric corruptions, enabling them to probe the Dark Ages as well as the Cosmic Dawn, and allowing one to investigate new (astro)physics that is inaccessible in any other way in the coming decades. [Abridged]

Observations of the redshifted 21-cm hyperfine line of neutral hydrogen from early phases of the Universe such as Cosmic Dawn and the Epoch of Reionization promise to open a new window onto the early formation of stars and galaxies. We present the first upper limits on the power spectrum of redshifted 21-cm brightness temperature fluctuations in the redshift range $z = 19.8 - 25.2$ ($54-68$ MHz frequency range) using 14 hours of data obtained with the LOFAR-Low Band Antenna (LBA) array. We also demonstrate the application of a multiple pointing calibration technique to calibrate the LOFAR-LBA dual-pointing observations centred on the North Celestial Pole and the radio galaxy 3C220.3. We observe an unexplained excess of $\sim 30-50\%$ in Stokes $I$ noise compared to Stokes $V$ for the two observed fields, which decorrelates on $\gtrsim 12$ seconds and might have a physical origin. We show that enforcing smoothness of gain errors along frequency direction during calibration reduces the additional variance in Stokes $I$ compared Stokes $V$ introduced by the calibration on sub-band level. After subtraction of smooth foregrounds, we achieve a $2\sigma$ upper limit on the 21-cm power spectrum of $\Delta_{21}^2 < (14561\,\text{mK})^2$ at $k\sim 0.038\,h\,\text{cMpc}^{-1}$ and $\Delta_{21}^2 < (14886\,\text{mK})^2$ at $k\sim 0.038 \,h\,\text{cMpc}^{-1}$ for the 3C220 and NCP fields respectively and both upper limits are consistent with each other. The upper limits for the two fields are still dominated by systematics on most $k$ modes.

Large scale filaments, with lengths that can reach tens of Mpc, are the most prominent features in the cosmic web. These filaments have only been observed indirectly through the positions of galaxies in large galaxy surveys or through absorption features in the spectra of high redshift sources. In this study we propose to go one step further and directly detect intergalactic medium filaments through their emission in the HI 21cm line. We make use of high resolution cosmological simulations to estimate the intensity of this emission in low redshift filaments and use it to make predictions for the direct detectability of specific filaments previously inferred from galaxy surveys, in particular the Sloan Digital Sky Survey. Given the expected signal of these filaments our study shows that HI emission from large filaments can be observed by current and next generation radio telescopes. We estimate that gas in filaments of length $l \gtrsim$ 15 $h^{-1}$Mpc with relatively small inclinations to the line of sight ($\lesssim 10^\circ$) can be observed in $\sim40-100$ hours with telescopes such as GMRT or EVLA, potentially providing large improvements over our knowledge of the astrophysical properties of these filaments. Due to their large field of view and sufficiently long integration times, upcoming HI surveys with the Apertif and ASKAP instruments will be able to detect large filaments independently of their orientation and curvature. Furthermore, our estimates indicate that a more powerful future radio telescope like SKA-2 can be used to map most of these filaments, which will allow them to be used as a strong cosmological probe.

Contamination due to foregrounds (Galactic and Extra-galactic), calibration errors and ionospheric effects pose major challenges in detection of the cosmic 21 cm signal in various Epoch of Reionization (EoR) experiments. We present the results of a pilot study of a field centered on 3C196 using LOFAR Low Band (56-70 MHz) observations, where we quantify various wide field and calibration effects such as gain errors, polarized foregrounds, and ionospheric effects. We observe a `pitchfork' structure in the 2D power spectrum of the polarized intensity in delay-baseline space, which leaks into the modes beyond the instrumental horizon (EoR/CD window). We show that this structure largely arises due to strong instrumental polarization leakage ($\sim30\%$) towards {Cas\,A} ($\sim21$ kJy at 81 MHz, brightest source in northern sky), which is far away from primary field of view. We measure an extremely small ionospheric diffractive scale ($r_{\text{diff}} \approx 430$ m at 60 MHz) towards {Cas\,A} resembling pure Kolmogorov turbulence compared to $r_{\text{diff}} \sim3 - 20$ km towards zenith at 150 MHz for typical ionospheric conditions. This is one of the smallest diffractive scales ever measured at these frequencies. Our work provides insights in understanding the nature of aforementioned effects and mitigating them in future Cosmic Dawn observations (e.g. with SKA-low and HERA) in the same frequency window.

The H-alpha line emission is an important probe for a number of fundamental quantities in galaxies, including their number density, star formation rate (SFR) and overall gas content. A new generation of low-resolution intensity mapping probes, e.g. SPHEREx and CDIM, will observe galaxies in H-alpha emission over a large fraction of the sky from the local Universe till a redshift of z ~ 6 to 10, respectively. This will also be the target line for observations by the high-resolution Euclid and WFIRST instruments in the z ~ 0.7 - 2 redshift range. In this paper, we estimate the intensity and power spectra of the H-alpha line in the z ~ 0 - 5 redshift range using observed line luminosity functions (LFs), when possible, and simulations, otherwise. We estimate the significance of our predictions by accounting for the modelling uncertainties (e.g. SFR, extinction, etc.) and observational contamination. We find that Intensity Mapping (IM) surveys can make a statistical detection of the full H-alpha emission between z ~ 0.8 - 5. Moreover, we find that the high-frequency resolution and the sensitivity of the planned CDIM surveys allow for the separation of H-alpha emission from several interloping lines. We explore ways to use the combination of these line intensities to probe galaxy properties. As expected, our study indicates that galaxy surveys will only detect bright galaxies that contribute up to a few percent of the overall H-alpha intensity. However, these surveys will provide important constraints on the high end of the H-alpha LF and put strong constraints on the AGN LF.

We introduce a novel technique, called "granulometry", to characterize and recover the mean size and the size distribution of HII regions from 21-cm tomography. The technique is easy to implement, but places the previously not very well defined concept of morphology on a firm mathematical foundation. The size distribution of the cold spots in 21-cm tomography can be used as a direct tracer of the underlying probability distribution of HII region sizes. We explore the capability of the method using large-scale reionization simulations and mock observational data cubes while considering capabilities of SKA1-low and a future extension to SKA2. We show that the technique allows the recovery of the HII region size distribution with a moderate signal-to-noise ratio from wide-field imaging ($\rm SNR\lesssim3$), for which the statistical uncertainty is sample variance dominated. We address the observational requirements on the angular resolution, the field-of-view, and the thermal noise limit for a successful measurement. To achieve a full scientific return from 21-cm tomography and to exploit a synergy with 21-cm power spectra, we suggest an observing strategy using wide-field imaging (several tens of square degrees) by an interferometric mosaicking/multi-beam observation with additional intermediate baselines (~2-4 km).

We present the first limits on the Epoch of Reionization (EoR) 21-cm HI power spectra, in the redshift range $z=7.9-10.6$, using the Low-Frequency Array (LOFAR) High-Band Antenna (HBA). In total 13\,h of data were used from observations centred on the North Celestial Pole (NCP). After subtraction of the sky model and the noise bias, we detect a non-zero $\Delta^2_{\rm I} = (56 \pm 13 {\rm mK})^2$ (1-$\sigma$) excess variance and a best 2-$\sigma$ upper limit of $\Delta^2_{\rm 21} < (79.6 {\rm mK})^2$ at $k=0.053$$h$cMpc$^{-1}$ in the range $z=$9.6-10.6. The excess variance decreases when optimizing the smoothness of the direction- and frequency-dependent gain calibration, and with increasing the completeness of the sky model. It is likely caused by (i) residual side-lobe noise on calibration baselines, (ii) leverage due to non-linear effects, (iii) noise and ionosphere-induced gain errors, or a combination thereof. Further analyses of the excess variance will be discussed in forthcoming publications.

Leakage of diffuse polarized emission into Stokes I caused by the polarized primary beam of the instrument might mimic the spectral structure of the 21-cm signal coming from the epoch of reionization (EoR) making their separation difficult. Therefore, understanding polarimetric performance of the antenna is crucial for a successful detection of the EoR signal. Here, we have calculated the accuracy of the nominal model beam of LOFAR in predicting the leakage from Stokes I to Q, U by comparing them with the corresponding leakage of compact sources actually observed in the 3C295 field. We have found that the model beam has errors of less than or equal to 10% on the predicted levels of leakage of ~1% within the field of view, i. e. if the leakage is taken out perfectly using this model the leakage will reduce to $10^{-3}$ of the Stokes I flux. If similar levels of accuracy can be obtained in removing leakage from Stokes Q, U to I, we can say, based on the results of our previous paper, that the removal of this leakage using this beam model would ensure that the leakage is well below the expected EoR signal in almost the whole instrumental k-space of the cylindrical power spectrum. We have also shown here that direction dependent calibration can remove instrumentally polarized compact sources, given an unpolarized sky model, very close to the local noise level.

We present a detailed analysis of the ionization and thermal structure of the intergalactic medium (IGM) around a high-redshift QSO using a large suite of cosmological, multi-frequency radiative transfer (RT) simulations, exploring the contribution from galaxies as well as the QSO, and the effect of X-rays and secondary ionization. We show that in high-z QSO environments both the central QSO and the surrounding galaxies concertedly control the reionization morphology of hydrogen and helium and have a non-linear impact on the thermal structure of the IGM. A QSO imprints a distinctive morphology on H II regions if its total ionizing photon budget exceeds that of the surrounding galaxies since the onset of hydrogen reionization; otherwise, the morphology shows little difference from that of H II regions produced only by galaxies. In addition, the spectral shape of the collective radiation field from galaxies and QSOs controls the thickness of the I-fronts. While a UV-obscured QSO can broaden the I-front, the contribution from other UV sources, either galaxies or unobscured QSO, is sufficient to maintain a sharp I-front. X-rays photons from the QSO are responsible for a prominent extended tail of partial ionization ahead of the I-front. QSOs leave a unique imprint on the morphology of He II / He III regions. We suggest that, while the physical state of the IGM is modified by QSOs, the most direct test to understand the role of galaxies and QSOs during reionization is to perform galaxy surveys in a region of sky imaged by 21 cm tomography.

A large fraction of the baryonic matter in the Universe is located in filaments in the intergalactic medium. However, the low surface brightness of these filaments has not yet allowed their direct detection except in very special regions in the circum-galactic medium (CGM). Here we simulate the intensity and spatial fluctuations in Lyman Alpha ${\rm (Ly\alpha)}$ emission from filaments in the intergalactic medium (IGM) and discuss the prospects for the next generation of space based instruments to detect the low surface brightness universe at UV wavelengths. Starting with a high resolution N-body simulation we obtain the dark matter density fluctuations and associate baryons with the dark matter particles assuming that they follow the same spatial distribution. The IGM thermal and ionization state is set by a model of the UV background and by the relevant cooling processes for a hydrogen and helium gas. The ${\rm Ly\alpha}$ emissivity is then estimated, taking into account recombination and collisional excitation processes. We find that the detection of these filaments through their ${\rm Ly\alpha}$ emission is well in the reach of the next generation of UV space based instruments and so it should be achieved in the next decade. The density field is populated with halos and galaxies and their ${\rm Ly\alpha}$ emission is estimated. Galaxies are treated as foregrounds and so we discuss methods to reduce their contamination from observational maps. Finally, we estimate the UV continuum background as a function of the redshift of the ${\rm Ly\alpha}$ emission line and discuss how this continuum can affect observations.

LOFAR is the LOw Frequency Radio interferometer ARray located at mid-latitude ($52^{\circ} 53'N$). Here, we present results on ionospheric structures derived from 29 LOFAR nighttime observations during the winters of 2012/2013 and 2013/2014. We show that LOFAR is able to determine differential ionospheric TEC values with an accuracy better than 1 mTECU over distances ranging between 1 and 100 km. For all observations the power law behavior of the phase structure function is confirmed over a long range of baseline lengths, between $1$ and $80$ km, with a slope that is in general larger than the $5/3$ expected for pure Kolmogorov turbulence. The measured average slope is $1.89$ with a one standard deviation spread of $0.1$. The diffractive scale, i.e. the length scale where the phase variance is $1\, \mathrm{rad^2}$, is shown to be an easily obtained single number that represents the ionospheric quality of a radio interferometric observation. A small diffractive scale is equivalent to high phase variability over the field of view as well as a short time coherence of the signal, which limits calibration and imaging quality. For the studied observations the diffractive scales at $150$ MHz vary between $3.5$ and $30\,$ km. A diffractive scale above $5$ km, pertinent to about $90 \%$ of the observations, is considered sufficient for the high dynamic range imaging needed for the LOFAR Epoch of Reionization project. For most nights the ionospheric irregularities were anisotropic, with the structures being aligned with the Earth magnetic field in about $60\%$ of the observations.

The redshifted 21 cm line of neutral hydrogen is a promising probe of the Epoch of Reionization (EoR). However, its detection requires a thorough understanding and control of the systematic errors. We study two systematic biases observed in the LOFAR EoR residual data after calibration and subtraction of bright discrete foreground sources. The first effect is a suppression in the diffuse foregrounds, which could potentially mean a suppression of the 21 cm signal. The second effect is an excess of noise beyond the thermal noise. The excess noise shows fluctuations on small frequency scales, and hence it can not be easily removed by foreground removal or avoidance methods. Our analysis suggests that sidelobes of residual sources due to the chromatic point spread function and ionospheric scintillation can not be the dominant causes of the excess noise. Rather, both the suppression of diffuse foregrounds and the excess noise can occur due to calibration with an incomplete sky model containing predominantly bright discrete sources. We show that calibrating only on bright sources can cause suppression of other signals and introduce an excess noise in the data. The levels of the suppression and excess noise depend on the relative flux of sources which are not included in the model with respect to the flux of modeled sources. We discuss possible solutions such as using only long baselines to calibrate the interferometric gain solutions as well as simultaneous multi-frequency calibration along with their benefits and shortcomings.

The removal of the Galactic and extragalactic foregrounds remains a major challenge for those wishing to make a detection of the Epoch of Reionization 21-cm signal. Multiple methods of modelling these foregrounds with varying levels of assumption have been trialled and shown promising recoveries on simulated data. Recently however there has been increased discussion of using the expected shape of the foregrounds in Fourier space to define an EoR window free of foreground contamination. By carrying out analysis within this window only, one can avoid the foregrounds and any statistical bias they might introduce by instead removing these foregrounds. In this paper we discuss the advantages and disadvantages of both foreground removal and foreground avoidance. We create a series of simulations with noise levels in line with both current and future experiments and compare the recovered statistical cosmological signal from foreground avoidance and a simplified, frequency independent foreground removal model. We find that while, for current generation experiments, foreground avoidance enables a better recovery at $k_{perp} > 0.6 \mathrm{Mpc}^{-1}$, foreground removal is able to recover significantly more signal at small $k_{los}$ for both current and future experiments. We also relax the assumption that the foregrounds are smooth by introducing a Gaussian random factor along the line-of-sight and then also spatially. We find that both methods perform well for foreground models with line-of-sight and spatial variations around $0.1\%$ however at levels larger than this foregrounds removal shows a greater signal recovery.

The observed 21-cm signal from the epoch of reionization will be distorted along the line-of-sight by the peculiar velocities of matter particles. These redshift-space distortions will affect the contrast in the signal and will also make it anisotropic. This anisotropy contains information about the cross-correlation between the matter density field and the neutral hydrogen field, and could thus potentially be used to extract information about the sources of reionization. In this paper, we study a collection of simulated reionization scenarios assuming different models for the sources of reionization. We show that the 21-cm anisotropy is best measured by the quadrupole moment of the power spectrum. We find that, unless the properties of the reionization sources are extreme in some way, the quadrupole moment evolves very predictably as a function of global neutral fraction. This predictability implies that redshift-space distortions are not a very sensitive tool for distinguishing between reionization sources. However, the quadrupole moment can be used as a model-independent probe for constraining the reionization history. We show that such measurements can be done to some extent by first-generation instruments such as LOFAR, while the SKA should be able to measure the reionization history using the quadrupole moment of the power spectrum to great accuracy.

This study aims to characterize linear polarization structures in LOFAR observations of the interstellar medium (ISM) in the 3C196 field, one of the primary fields of the LOFAR-Epoch of Reionization key science project. We have used the high band antennas (HBA) of LOFAR to image this region and Rotation Measure (RM) synthesis to unravel the distribution of polarized structures in Faraday depth. The brightness temperature of the detected Galactic emission is $5-15~{\rm K}$ in polarized intensity and covers the range from -3 to +8 ${\rm rad~m^{-2}}$ in Faraday depth. The most interesting morphological feature is a strikingly straight filament at a Faraday depth of $+0.5~{\rm rad~m^{-2}}$ running from north to south, right through the centre of the field and parallel to the Galactic plane. There is also an interesting system of linear depolarization canals conspicuous in an image showing the peaks of Faraday spectra. We used the Westerbork Synthesis Radio Telescope (WSRT) at 350 MHz to image the same region. For the first time, we see some common morphology in the RM cubes made at 150 and 350~{\rm MHz}. There is no indication of diffuse emission in total intensity in the interferometric data, in line with results at higher frequencies and previous LOFAR observations. Based on our results, we determined physical parameters of the ISM and proposed a simple model that may explain the observed distribution of the intervening magneto-ionic medium. The mean line-of-sight magnetic field component, $B_\parallel$, is determined to be $0.3\pm0.1~{\rm \mu G}$ and its spatial variation across the 3C196 field is $0.1~{\rm \mu G}$. The filamentary structure is probably an ionized filament in the ISM, located somewhere within the Local Bubble. This filamentary structure shows an excess in thermal electron density ($n_e B_\parallel>6.2~{\rm cm^{-3}\mu G}$) compared to its surroundings.

The 21cm-galaxy cross-power spectrum is expected to be one of the promising probes of the Epoch of Reionization (EoR), as it could offer information about the progress of reionization and the typical scale of ionized regions at different redshifts. With upcoming observations of 21cm emission from the EoR with the Low Frequency Array (LOFAR), and of high redshift Lyalpha emitters (LAEs) with Subaru's Hyper Suprime Cam (HSC), we investigate the observability of such cross-power spectrum with these two instruments, which are both planning to observe the ELAIS-N1 field at z=6.6. In this paper we use N-body + radiative transfer (both for continuum and Lyalpha photons) simulations at redshift 6.68, 7.06 and 7.3 to compute the 3D theoretical 21cm-galaxy cross-power spectrum, as well as to predict the 2D 21cm-galaxy cross-power spectrum expected to be observed by LOFAR and HSC. Once noise and projection effects are accounted for, our predictions of the 21cm-galaxy cross-power spectrum show clear anti-correlation on scales larger than ~ 60 h$^{-1}$ Mpc (corresponding to k ~ 0.1 h Mpc$^{-1}$), with levels of significance p=0.04 at z=6.6 and p=0.048 at z=7.3. On smaller scales, instead, the signal is completely contaminated.

Recent Low Frequency Array (LOFAR) observations at 115-175 MHz of a field at medium Galactic latitudes (centered at the bright quasar 3C196) have shown striking filamentary structures in polarization that extend over more than 4 degrees across the sky. In addition, the Planck satellite has released full sky maps of the dust emission in polarization at 353GHz. The LOFAR data resolve Faraday structures along the line of sight, whereas the Planck dust polarization maps probe the orientation of the sky projected magnetic field component. Hence, no apparent correlation between the two is expected. Here we report a surprising, yet clear, correlation between the filamentary structures, detected with LOFAR, and the magnetic field orientation, probed by the Planck satellite. This finding points to a common, yet unclear, physical origin of the two measurements in this specific area in the sky. A number of follow-up multi- frequency studies are proposed to shed light on this unexpected finding.

We investigate the feasibility of detecting 21cm absorption features in the afterglow spectra of high redshift long Gamma Ray Bursts (GRBs). This is done employing simulations of cosmic reionization, together with the instrumental characteristics of the LOw Frequency ARray (LOFAR). We find that absorption features could be marginally (with a S/N larger than a few) detected by LOFAR at z>7 if the GRB originated from PopIII stars, while the detection would be easier if the noise were reduced by one order of magnitude, i.e. similar to what is expected for the first phase of the Square Kilometer Array (SKA1-low). On the other hand, more standard GRBs are too dim to be detected even with ten times the sensitivity of SKA1-low, and only in the most optimistic case can a S/N larger than a few be reached at z>9.

Experiments designed to measure the redshifted 21~cm line from the Epoch of Reionization (EoR) are challenged by strong astrophysical foreground contamination, ionospheric distortions, complex instrumental response and other different types of noise (e.g. radio frequency interference). The astrophysical foregrounds are dominated by diffuse synchrotron emission from our Galaxy. Here we present a simulation of the Galactic emission used as a foreground module for the LOFAR- EoR key science project end-to-end simulations. The simulation produces total and polarized intensity over $10^\circ \times 10^\circ$ maps of the Galactic synchrotron and free-free emission, including all observed characteristics of the emission: spatial fluctuations of amplitude and spectral index of the synchrotron emission, together with Faraday rotation effects. The importance of these simulations arise from the fact that the Galactic polarized emission could behave in a manner similar to the EoR signal along the frequency direction. As a consequence, an improper instrumental calibration will give rise to leakages of the polarized to the total signal and mask the desired EoR signal. In this paper we address this for the first time through realistic simulations.

Detection of the 21-cm signal coming from the epoch of reionization (EoR) is challenging especially because, even after removing the foregrounds, the residual Stokes $I$ maps contain leakage from polarized emission that can mimic the signal. Here, we discuss the instrumental polarization of LOFAR and present realistic simulations of the leakages between Stokes parameters. From the LOFAR observations of polarized emission in the 3C196 field, we have quantified the level of polarization leakage caused by the nominal model beam of LOFAR, and compared it with the EoR signal using power spectrum analysis. We found that at 134--166 MHz, within the central 4$^\circ$ of the field the $(Q,U)\rightarrow I$ leakage power is lower than the EoR signal at $k<0.3$ Mpc$^{-1}$. The leakage was found to be localized around a Faraday depth of 0, and the rms of the leakage as a fraction of the rms of the polarized emission was shown to vary between 0.2-0.3\%, both of which could be utilized in the removal of leakage. Moreover, we could define an `EoR window' in terms of the polarization leakage in the cylindrical power spectrum above the PSF-induced wedge and below $k_\parallel\sim 0.5$ Mpc$^{-1}$, and the window extended up to $k_\parallel\sim 1$ Mpc$^{-1}$ at all $k_\perp$ when 70\% of the leakage had been removed. These LOFAR results show that even a modest polarimetric calibration over a field of view of $\lesssim 4^\circ$ in the future arrays like SKA will ensure that the polarization leakage remains well below the expected EoR signal at the scales of 0.02-1 Mpc$^{-1}$.

To obtain a physically well-motivated definition of superclusters, we proposed in our previous work to select superclusters with an overdensity criterion that selects only those objects that will collapse in the future, including those that are at a turn-around in the present epoch. In this paper we present numerical values for these criteria for a range of standard cosmological models. We express these criteria in terms of a density ratio or, alternatively, as an infall velocity and show that these two criteria give almost identical results. To better illustrate the implications of this definition, we applied our criteria to some prominent structures in the local Universe, the Local supercluster, Shapley supercluster, and the recently reported Laniakea supercluster to understand their future evolution. We find that for the Local and Shapley superclusters, only the central regions will collapse in the future, while Laniakea does not constitute a significant overdensity and will disperse in the future. Finally, we suggest that those superclusters that will survive the accelerating cosmic expansion and collapse in the future be called "superstes-clusters", where "superstes" means survivor in Latin, to distinguish them from traditional superclusters.

The SKA will build upon early detections of the EoR by precursor instruments, such as MWA, PAPER, and LOFAR, and planned instruments, such as HERA, to make the first high signal-to-noise measurements of fluctuations in the 21 cm brightness temperature from both reionization and the cosmic dawn. This will allow both imaging and statistical maps of the 21cm signal at redshifts z = 6 - 27 and constrain the underlying cosmology and evolution of the density field. This era includes nearly 60% of the (in principle) observable volume of the Universe and many more linear modes than the CMB, presenting an opportunity for SKA to usher in a new level of precision cosmology. This optimistic picture is complicated by the need to understand and remove the effect of astrophysics, so that systematics rather than statistics will limit constraints. This chapter describes the cosmological, as opposed to astrophysical, information available to SKA. Key areas for discussion include: cosmological parameters constraints using 21cm fluctuations as a tracer of the density field; lensing of the 21cm signal, constraints on heating via exotic physics such as decaying or annihilating dark matter; impact of fundamental physics such as non-Gaussianity or warm dark matter on the source population; and constraints on the bulk flows arising from the decoupling of baryons and photons at z = 1000. The chapter explores the path to separating cosmology from astrophysics, for example via velocity space distortions and separation in redshift. We discuss new opportunities for extracting cosmology made possible by the sensitivity of SKA Phase 1 and explores the advances achievable with SKA2.

We explore the observability of the neutral hydrogen (HI) and the singly-ionized isotope helium-3 ($^3$HeII) in the intergalactic medium (IGM) from the Epoch of Reionization down to the local Universe. The hyperfine transition of $^3$HeII, which is not as well known as the HI transition, has energy splitting corresponding to 8 cm. It also has a larger spontaneous decay rate than that of neutral hydrogen, whereas its primordial abundance is much smaller. Although both species are mostly ionized in the IGM, the balance between ionization and recombination in moderately high density regions renders them abundant enough to be observed. We estimate the emission signal of both hyperfine transitions from large scale filamentary structures and discuss the prospects for observing them with current and future radio telescopes. We conclude that HI in filaments is possibly observable even with current telescopes after 100 hours of observation. On the other hand, $^3$HeII is only detectable with future telescopes, such as SKA, after the same amount of time.

We present radio observations of the Moon between $35$ and $80$ MHz to demonstrate a novel technique of interferometrically measuring large-scale diffuse emission extending far beyond the primary beam (global signal) for the first time. In particular, we show that (i) the Moon appears as a negative-flux source at frequencies $35<\nu<80$ MHz since it is `colder' than the diffuse Galactic background it occults, (ii) using the (negative) flux of the lunar disc, we can reconstruct the spectrum of the diffuse Galactic emission with the lunar thermal emission as a reference, and (iii) that reflected RFI (radio-frequency interference) is concentrated at the center of the lunar disc due to specular nature of reflection, and can be independently measured. Our RFI measurements show that (i) Moon-based Cosmic Dawn experiments must design for an Earth-isolation of better than $80$ dB to achieve an RFI temperature $<1$ mK, (ii) Moon-reflected RFI contributes to a dipole temperature less than $20$ mK for Earth-based Cosmic Dawn experiments, (iii) man-made satellite-reflected RFI temperature exceeds $20$ mK if the aggregate cross section of visible satellites exceeds $80$ m$^2$ at $800$ km height, or $5$ m$^2$ at $400$ km height. Currently, our diffuse background spectrum is limited by sidelobe confusion on short baselines (10-15% level). Further refinement of our technique may yield constraints on the redshifted global $21$-cm signal from Cosmic Dawn ($40>z>12$) and the Epoch of Reionization ($12>z>5$).

This study aims to characterise the polarized foreground emission in the ELAIS-N1 field and to address its possible implications for the extraction of the cosmological 21-cm signal from the Low-Frequency Array - Epoch of Reionization (LOFAR-EoR) data. We use the high band antennas of LOFAR to image this region and RM-synthesis to unravel structures of polarized emission at high Galactic latitudes. The brightness temperature of the detected Galactic emission is on average 4 K in polarized intensity and covers the range from -10 to +13rad m^-2 in Faraday depth. The total polarized intensity and polarization angle show a wide range of morphological features. We have also used the Westerbork Synthesis Radio Telescope (WSRT) at 350 MHz to image the same region. The LOFAR and WSRT images show a similar complex morphology, at comparable brightness levels, but their spatial correlation is very low. The fractional polarization at 150 MHz, expressed as a percentage of the total intensity, amounts to 1.5%. There is no indication of diffuse emission in total intensity in the interferometric data, in line with results at higher frequencies. The wide frequency range, good angular resolution and good sensitivity make LOFAR an exquisite instrument for studying Galactic polarized emission at a resolution of 1-2 rad m^-2 in Faraday depth. The different polarised patterns observed at 150 MHz and 350 MHz are consistent with different source distributions along the line of sight wring in a variety of Faraday thin regions of emission. The presence of polarised foregrounds is a serious complication for Epoch of Reionization experiments. To avoid the leakage of polarized emission into total intensity, which can depend on frequency, we need to calibrate the instrumental polarization across the field of view to a small fraction of 1%.

Several experiments are underway to detect the cosmic redshifted 21-cm signal from neutral hydrogen from the Epoch of Reionization (EoR). Due to their very low signal-to-noise ratio, these observations aim for a statistical detection of the signal by measuring its power spectrum. We investigate the extraction of the variance of the signal as a first step towards detecting and constraining the global history of the EoR. Signal variance is the integral of the signal's power spectrum, and it is expected to be measured with a high significance. We demonstrate this through results from a simulation and parameter estimation pipeline developed for the Low Frequency Array (LOFAR)-EoR experiment. We show that LOFAR should be able to detect the EoR in 600 hours of integration using the variance statistic. Additionally, the redshift ($z_r$) and duration ($\Delta z$) of reionization can be constrained assuming a parametrization. We use an EoR simulation of $z_r = 7.68$ and $\Delta z = 0.43$ to test the pipeline. We are able to detect the simulated signal with a significance of 4 standard deviations and extract the EoR parameters as $z_r = 7.72^{+0.37}_{-0.18}$ and $\Delta z = 0.53^{+0.12}_{-0.23}$ in 600 hours, assuming that systematic errors can be adequately controlled. We further show that the significance of detection and constraints on EoR parameters can be improved by measuring the cross-variance of the signal by cross-correlating consecutive redshift bins.

With improving telescopes, it may now be possible to observe the Epoch of Reionization in multiple ways. We examine two of these observables - the excess light in the near-infrared background that may be due to high redshift stars and ionized HII bubbles, and the 21 cm emission from neutral hydrogen. Because these two forms of emission should result from different, mutually exclusive regions, an anticorrelation should exist between them. We discuss the strengths of using cross-correlations between these observations to learn more about high redshift star formation and reionization history. In particular, we create simulated maps of emission from both the near-infrared background and 21 cm emission. We find that these observations are anticorrelated, with the strongest anticorrelation originating from times when the universe is half ionized. This result is robust and does not depend on the properties of the stars themselves. Rather, it depends on the ionization history. Cross-correlations can provide redshift information, which the near-infrared background cannot provide alone. In addition, cross-correlations can help separate foreground emission from the true high redshift component, making it possible to say with greater certainty that we are indeed witnessing the Epoch of Reionization.

We estimate the potential contribution of M < 10^9 Msun dwarf galaxies to the reionization and early metal-enrichment of the Milky Way environment, or circum-Galactic Medium. Our approach is to use the observed properties of ancient stars (> 12 Gyr old) measured in nearby dwarf galaxies to characterize the star-formation at high-z. We use a merger-tree model for the build-up of the Milky Way, which self-consistently accounts for feedback processes, and which is calibrated to match the present-day properties of the Galaxy and its dwarf satellites. We show that the high-z analogues of nearby dwarf galaxies can produce the bulk of ionizing radiation (>80%) required to reionize the Milky Way environment. Our fiducial model shows that the gaseous environment can be 50% reionized at z ~ 8 by galaxies with 10^7 Msun < M < 10^8 Msun. At later times, radiative feedback stops the star-formation in these small systems, and reionization is completed by more massive dwarf galaxies by z_rei = 6.4\pm 0.5. The metals ejected by supernova-driven outflows from M < 10^9 Msun dwarf galaxies almost uniformly fill the Milky Way environment by z ~ 5, enriching it to Z ~ 2 10^-2 Zsun. At z ~ 2 these early metals are still found to represent ~ 50% of the total mass of heavy elements in the circum-Galactic Medium.

Faint undetected sources of radio-frequency interference (RFI) might become visible in long radio observations when they are consistently present over time. Thereby, they might obstruct the detection of the weak astronomical signals of interest. This issue is especially important for Epoch of Reionisation (EoR) projects that try to detect the faint redshifted HI signals from the time of the earliest structures in the Universe. We explore the RFI situation at 30-163 MHz by studying brightness histograms of visibility data observed with LOFAR, similar to radio-source-count analyses that are used in cosmology. An empirical RFI distribution model is derived that allows the simulation of RFI in radio observations. The brightness histograms show an RFI distribution that follows a power-law distribution with an estimated exponent around -1.5. With several assumptions, this can be explained with a uniform distribution of terrestrial radio sources whose radiation follows existing propagation models. Extrapolation of the power law implies that the current LOFAR EoR observations should be severely RFI limited if the strength of RFI sources remains strong after time integration. This is in contrast with actual observations, which almost reach the thermal noise and are thought not to be limited by RFI. Therefore, we conclude that it is unlikely that there are undetected RFI sources that will become visible in long observations. Consequently, there is no indication that RFI will prevent an EoR detection with LOFAR.

One of the most promising ways to study the epoch of reionization (EoR) is through radio observations of the redshifted 21-cm line emission from neutral hydrogen. These observations are complicated by the fact that the mapping of redshifts to line-of-sight positions is distorted by the peculiar velocities of the gas. Such distortions can be a source of error if they are not properly understood, but they also encode information about cosmology and astrophysics. We study the effects of redshift space distortions on the power spectrum of 21-cm radiation from the EoR using large scale $N$-body and radiative transfer simulations. We quantify the anisotropy introduced in the 21-cm power spectrum by redshift space distortions and show how it evolves as reionization progresses and how it relates to the underlying physics. We go on to study the effects of redshift space distortions on LOFAR observations, taking instrument noise and foreground subtraction into account. We find that LOFAR should be able to directly observe the power spectrum anisotropy due to redshift space distortions at spatial scales around $k \sim 0.1$ Mpc$^{-1}$ after $\gtrsim$ 1000 hours of integration time. At larger scales, sample errors become a limiting factor, while at smaller scales detector noise and foregrounds make the extraction of the signal problematic. Finally, we show how the astrophysical information contained in the evolution of the anisotropy of the 21-cm power spectrum can be extracted from LOFAR observations, and how it can be used to distinguish between different reionization scenarios.

The main objective of this work is to accelerate the Maximum-Likelihood (ML) estimation procedure in radio interferometric calibration. We introduce the OS-LS and the OS-SAGE radio interferometric calibration methods, as a combination of the Ordered-Subsets (OS) method with the Least-Squares (LS) and Space Alternating Generalized Expectation maximization (SAGE) calibration techniques, respectively. The OS algorithm speeds up the ML estimation and achieves nearly the same level of accuracy of solutions as the one obtained by the non-OS methods. We apply the OS-LS and OS-SAGE calibration methods to simulated observations and show that these methods have a much higher convergence rate relative to the conventional LS and SAGE techniques. Moreover, the obtained results show that the OS-SAGE calibration technique has a superior performance compared to the OS-LS calibration method in the sense of achieving more accurate results while having significantly less computational cost.

There are only a few ways to constrain the Era of Reionization and the properties of high redshift (z>6) stars through observations. Here, we discuss one of these observables - the spectrum of the Near Infrared Background - and how it is potentially affected by the transition from Population III to Population II stars. The stronger Lyman-alpha emission expected from massive Population III stars could result in a 'bump' in the spectrum of the Near Infrared Background (referred to in this work as the Lyman-alpha bump). The strength and shape of this bump can reveal properties of Population III stars. The Lyman-alpha bump is predicted to be higher if Population III stars are more massive and present at lower redshifts. The shape of the bump is governed by the star formation rate and the time it takes Population III stars to transition to Population II stars. If Population III stars are indeed massive, a bump is predicted as long as Population III stars exist at z < 15, even if their star formation rate is as low as 10^-7 M_sun yr^-1 Mpc^-3. This means that there may be some observational signature in the Near Infrared Background of small pockets of metal-free gas forming Population III stars at z ~ 6, even if they are quite rare.

LOFAR, the LOw-Frequency ARray, is a new-generation radio interferometer constructed in the north of the Netherlands and across europe. Utilizing a novel phased-array design, LOFAR covers the largely unexplored low-frequency range from 10-240 MHz and provides a number of unique observing capabilities. Spreading out from a core located near the village of Exloo in the northeast of the Netherlands, a total of 40 LOFAR stations are nearing completion. A further five stations have been deployed throughout Germany, and one station has been built in each of France, Sweden, and the UK. Digital beam-forming techniques make the LOFAR system agile and allow for rapid repointing of the telescope as well as the potential for multiple simultaneous observations. With its dense core array and long interferometric baselines, LOFAR achieves unparalleled sensitivity and angular resolution in the low-frequency radio regime. The LOFAR facilities are jointly operated by the International LOFAR Telescope (ILT) foundation, as an observatory open to the global astronomical community. LOFAR is one of the first radio observatories to feature automated processing pipelines to deliver fully calibrated science products to its user community. LOFAR's new capabilities, techniques and modus operandi make it an important pathfinder for the Square Kilometre Array (SKA). We give an overview of the LOFAR instrument, its major hardware and software components, and the core science objectives that have driven its design. In addition, we present a selection of new results from the commissioning phase of this new radio observatory.

Using a combination of N-body simulations, semi-analytic models and radiative transfer calculations, we have estimated the theoretical cross power spectrum between galaxies and the 21cm emission from neutral hydrogen during the epoch of reionization. In accordance with previous studies, we find that the 21cm emission is initially correlated with halos on large scales (> 30 Mpc), anti-correlated on intermediate (~ 5 Mpc), and uncorrelated on small (< 3 Mpc) scales. This picture quickly changes as reionization proceeds and the two fields become anti-correlated on large scales. The normalization of the cross power spectrum can be used to set constraints on the average neutral fraction in the intergalactic medium and its shape can be a tool to study the topology of reionization. When we apply a drop-out technique to select galaxies and add to the 21cm signal the noise expected from the LOFAR telescope, we find that while the normalization of the cross power spectrum remains a useful tool for probing reionization, its shape becomes too noisy to be informative. On the other hand, for a Lyalpha Emitter (LAE) survey both the normalization and the shape of the cross power spectrum are suitable probes of reionization. A closer look at a specific planned LAE observing program using Subaru Hyper-Suprime Cam reveals concerns about the strength of the 21cm signal at the planned redshifts. If the ionized fraction at z ~ 7 is lower that the one estimated here, then using the cross power spectrum may be a useful exercise given that at higher redshifts and neutral fractions it is able to distinguish between two toy models with different topologies.

The Square Kilometre Array (SKA) will have a low frequency component (SKA-low) which has as one of its main science goals the study of the redshifted 21cm line from the earliest phases of star and galaxy formation in the Universe. This 21cm signal provides a new and unique window on both the formation of the first stars and accreting black holes and the later period of substantial ionization of the intergalactic medium. The signal will teach us fundamental new things about the earliest phases of structure formation, cosmology and even has the potential to lead to the discovery of new physical phenomena. Here we present a white paper with an overview of the science questions that SKA-low can address, how we plan to tackle these questions and what this implies for the basic design of the telescope.

One of the key science projects of the Low-Frequency Array (LOFAR) is the detection of the cosmological signal coming from the Epoch of Reionization (EoR). Here we present the LOFAR EoR Diagnostic Database (LEDDB) that is used in the storage, management, processing and analysis of the LOFAR EoR observations. It stores referencing information of the observations and diagnostic parameters extracted from their calibration. This stored data is used to ease the pipeline processing, monitor the performance of the telescope and visualize the diagnostic parameters which facilitates the analysis of the several contamination effects on the signals. It is implemented with PostgreSQL and accessed through the psycopg2 python module. We have developed a very flexible query engine, which is used by a web user interface to access the database, and a very extensive set of tools for the visualization of the diagnostic parameters through all their multiple dimensions.

The aim of the LOFAR Epoch of Reionization (EoR) project is to detect the spectral fluctuations of the redshifted HI 21cm signal. This signal is weaker by several orders of magnitude than the astrophysical foreground signals and hence, in order to achieve this, very long integrations, accurate calibration for stations and ionosphere and reliable foreground removal are essential. One of the prospective observing windows for the LOFAR EoR project will be centered at the North Celestial Pole (NCP). We present results from observations of the NCP window using the LOFAR highband antenna (HBA) array in the frequency range 115 MHz to 163 MHz. The data were obtained in April 2011 during the commissioning phase of LOFAR. We used baselines up to about 30 km. With about 3 nights, of 6 hours each, effective integration we have achieved a noise level of about 100 microJy/PSF in the NCP window. Close to the NCP, the noise level increases to about 180 microJy/PSF, mainly due to additional contamination from unsubtracted nearby sources. We estimate that in our best night, we have reached a noise level only a factor of 1.4 above the thermal limit set by the noise from our Galaxy and the receivers. Our continuum images are several times deeper than have been achieved previously using the WSRT and GMRT arrays. We derive an analytical explanation for the excess noise that we believe to be mainly due to sources at large angular separation from the NCP.

The accurate and precise removal of 21-cm foregrounds from Epoch of Reionization redshifted 21-cm emission data is essential if we are to gain insight into an unexplored cosmological era. We apply a non-parametric technique, Generalized Morphological Component Analysis or GMCA, to simulated LOFAR-EoR data and show that it has the ability to clean the foregrounds with high accuracy. We recover the 21-cm 1D, 2D and 3D power spectra with high accuracy across an impressive range of frequencies and scales. We show that GMCA preserves the 21-cm phase information, especially when the smallest spatial scale data is discarded. While it has been shown that LOFAR-EoR image recovery is theoretically possible using image smoothing, we add that wavelet decomposition is an efficient way of recovering 21-cm signal maps to the same or greater order of accuracy with more flexibility. By comparing the GMCA output residual maps (equal to the noise, 21-cm signal and any foreground fitting errors) with the 21-cm maps at one frequency and discarding the smaller wavelet scale information, we find a correlation coefficient of 0.689, compared to 0.588 for the equivalently smoothed image. Considering only the central 50% of the maps, these coefficients improve to 0.905 and 0.605 respectively and we conclude that wavelet decomposition is a significantly more powerful method to denoise reconstructed 21-cm maps than smoothing.

The new generation of radio synthesis arrays, such as LOFAR and SKA, have been designed to surpass existing arrays in terms of sensitivity, angular resolution and frequency coverage. This evolution has led to the development of advanced calibration techniques that ensure the delivery of accurate results at the lowest possible computational cost. However, the performance of such calibration techniques is still limited by the compact, bright sources in the sky, used as calibrators. It is important to have a bright enough source that is well distinguished from the background noise level in order to achieve satisfactory results in calibration. We present "clustered calibration" as a modification to traditional radio interferometric calibration, in order to accommodate faint sources that are almost below the background noise level into the calibration process. The main idea is to employ the information of the bright sources' measured signals as an aid to calibrate fainter sources that are nearby the bright sources. In the case where we do not have bright enough sources, a source cluster could act as a bright source that can be distinguished from background noise. We construct a number of source clusters assuming that the signals of the sources belonging to a single cluster are corrupted by almost the same errors, and each cluster is calibrated as a single source, using the combined coherencies of its sources simultaneously. This upgrades the power of an individual faint source by the effective power of its cluster. We give performance analysis of clustered calibration to show the superiority of this approach compared to the traditional unclustered calibration. We also provide analytical criteria to choose the optimum number of clusters for a given observation in an efficient manner.

We present an analysis of Planck satellite data on the Coma Cluster observed via the Sunyaev-Zeldovich effect. Planck is able, for the first time, to detect SZ emission up to r ~ 3 X R_500. We test previously proposed models for the pressure distribution in clusters against the azimuthally averaged data. We find that the Arnaud et al. universal pressure profile does not fit Coma, and that their pressure profile for merging systems provides a good fit of the data only at r<R_500: by r=2XR_500 it underestimates the observed y profile by a factor of ~2. This may indicate that at these larger radii either i) the cluster SZ emission is contaminated by unresolved SZ sources along the line of sight or ii) the pressure profile of Coma is higher at r>R_500 than the mean pressure profile predicted by the simulations. The Planck image shows significant local steepening of the y profile in two regions about half a degree to the west and to the south-east of the cluster centre. These features are consistent with the presence of shock fronts at these radii, and indeed the western feature was previously noticed in the ROSAT PSPC mosaic as well as in the radio. Using Planck y profiles extracted from corresponding sectors we find pressure jumps of 4.5+0.4-0.2 and 5.0+1.3-0.1 in the west and southeast, respectively. Assuming Rankine-Hugoniot pressure jump conditions, we deduce that the shock waves should propagate with Mach number M_w=2.03+0.09-0.04 and M_se=2.05+0.25-0.02 in the West and Southeast, respectively. Finally, we find that the y and radio-synchrotron signals are quasi-linearly correlated on Mpc scales with small intrinsic scatter. This implies either that the energy density of cosmic-ray electrons is relatively constant throughout the cluster, or that the magnetic fields fall off much more slowly with radius than previously thought.

We discuss the feasibility of the detection of the 21cm forest in the diffuse IGM with the radio telescope LOFAR. The optical depth to the 21cm line has been derived using simulations of reionization which include detailed radiative transfer of ionizing photons. We find that the spectra from reionization models with similar total comoving hydrogen ionizing emissivity but different frequency distribution look remarkably similar. Thus, unless the reionization histories are very different from each other (e.g. a predominance of UV vs. x-ray heating) we do not expect to distinguish them by means of observations of the 21cm forest. Because the presence of a strong x-ray background would make the detection of 21cm line absorption impossible, the lack of absorption could be used as a probe of the presence/intensity of the x-ray background and the thermal history of the universe. Along a random line of sight LOFAR could detect a global suppression of the spectrum from z>12, when the IGM is still mostly neutral and cold, in contrast with the more well-defined, albeit broad, absorption features visible at lower redshift. Sharp, strong absorption features associated with rare, high density pockets of gas could be detected also at z~7 along preferential lines of sight.

Aims: This paper discusses the spectral occupancy for performing radio astronomy with the Low-Frequency Array (LOFAR), with a focus on imaging observations. Methods: We have analysed the radio-frequency interference (RFI) situation in two 24-h surveys with Dutch LOFAR stations, covering 30-78 MHz with low-band antennas and 115-163 MHz with high-band antennas. This is a subset of the full frequency range of LOFAR. The surveys have been observed with a 0.76 kHz / 1 s resolution. Results: We measured the RFI occupancy in the low and high frequency sets to be 1.8% and 3.2% respectively. These values are found to be representative values for the LOFAR radio environment. Between day and night, there is no significant difference in the radio environment. We find that lowering the current observational time and frequency resolutions of LOFAR results in a slight loss of flagging accuracy. At LOFAR's nominal resolution of 0.76 kHz and 1 s, the false-positives rate is about 0.5%. This rate increases approximately linearly when decreasing the data frequency resolution. Conclusions: Currently, by using an automated RFI detection strategy, the LOFAR radio environment poses no perceivable problems for sensitive observing. It remains to be seen if this is still true for very deep observations that integrate over tens of nights, but the situation looks promising. Reasons for the low impact of RFI are the high spectral and time resolution of LOFAR; accurate detection methods; strong filters and high receiver linearity; and the proximity of the antennas to the ground. We discuss some strategies that can be used once low-level RFI starts to become apparent. It is important that the frequency range of LOFAR remains free of broadband interference, such as DAB stations and windmills.

The survey of galaxy clusters performed by Planck through the Sunyaev-Zeldovich effect has already discovered many interesting objects, thanks to the whole coverage of the sky. One of the SZ candidates detected in the early months of the mission near to the signal to noise threshold, PLCKG214.6+37.0, was later revealed by XMM-Newton to be a triple system of galaxy clusters. We have further investigated this puzzling system with a multi-wavelength approach and we present here the results from a deep XMM-Newton re-observation. The characterisation of the physical properties of the three components has allowed us to build a template model to extract the total SZ signal of this system with Planck data. We partly reconciled the discrepancy between the expected SZ signal from X-rays and the observed one, which are now consistent at less than 1.2 sigma. We measured the redshift of the three components with the iron lines in the X-ray spectrum, and confirmed that the three clumps are likely part of the same supercluster structure. The analysis of the dynamical state of the three components, as well as the absence of detectable excess X-ray emission, suggest that we are witnessing the formation of a massive cluster at an early phase of interaction.

The first generation of redshifted 21 cm detection experiments, carried out with arrays like LOFAR, MWA and GMRT, will have a very low signal-to-noise ratio per resolution element (\sim 0.2). In addition, whereas the variance of the cosmological signal decreases on scales larger than the typical size of ionization bubbles, the variance of the formidable galactic foregrounds increases, making it hard to disentangle the two on such large scales. The poor sensitivity on small scales on the one hand, and the foregrounds effect on large scales on the other hand, make direct imaging of the Epoch of Reionization of the Universe very difficult, and detection of the signal therefore is expected to be statistical.Despite these hurdles, in this paper we argue that for many reionization scenarios low resolution images could be obtained from the expected data. This is because at the later stages of the process one still finds very large pockets of neutral regions in the IGM, reflecting the clustering of the large-scale structure, which stays strong up to scales of \sim 120 comoving Mpc/h (\sim 1 degree). The coherence of the emission on those scales allows us to reach sufficient S/N (\sim 3) so as to obtain reionization 21 cm images. Such images will be extremely valuable for answering many cosmological questions but above all they will be a very powerful tool to test our control of the systematics in the data. The existence of this typical scale (\sim 120 comoving Mpc/h) also argues for designing future EoR experiments, e.g., with SKA, with a field of view of at least 4 degree.

The Universe's dark ages end with the formation of the first generation of galaxies. These objects start emitting ultraviolet radiation that carves out ionized regions around them. After a sufficient number of ionizing sources have formed, the ionized fraction of the gas in the Universe rapidly increases until hydrogen becomes fully ionized. This period, during which the cosmic gas went from neutral to ionized, is known as the Universe's Epoch of Reionization . The Epoch of Reionization is related to many fundamental questions in cosmology, such as properties of the first galaxies, physics of (mini-)quasars, formation of very metal-poor stars and a slew of other important research topics in astrophysics. Hence uncovering it will have far reaching implications on the study of structure formation in the early Universe. This chapter reviews the current observational evidence for the occurrence of this epoch, its key theoretical aspects and main characteristics, and finally the various observational probes that promise to uncover it. A special emphasis is put on the redshifted 21 cm probe, the various experiments that are currently being either built or designed, and what we can learn from them about the Epoch of Reionization.

We introduce a new implementation of the FastICA algorithm on simulated LOFAR EoR data with the aim of accurately removing the foregrounds and extracting the 21-cm reionization signal. We find that the method successfully removes the foregrounds with an average fitting error of 0.5 per cent and that the 2D and 3D power spectra are recovered across the frequency range. We find that for scales above several PSF scales the 21-cm variance is successfully recovered though there is evidence of noise leakage into the reconstructed foreground components. We find that this blind independent component analysis technique provides encouraging results without the danger of prior foreground assumptions.

Techniques to improve the data quality of interferometric radio observations are considered. Fundaments of fringe frequencies in the uv-plane are discussed and filters are used to attenuate radio-frequency interference (RFI) and off-axis sources. Several new applications of filters are introduced and tested. A low-pass filter in time and frequency direction on single baseline data is successfully used to lower the noise in the area of interest and to remove sidelobes coming from unmodelled off-axis sources and RFI. Related side effects of data integration, averaging and gridding are analysed, and shown to be able to cause ghosts and an increase in noise, especially when using long baselines or interferometric elements that have a large field of view. A novel projected fringe low-pass filter is shown to be potentially useful for first order source separation. Initial tests show that the filters can be several factors faster compared to common source separation techniques such as peeling and a variant of peeling that is currently being tested on LOFAR observations called "demixed peeling". Further testing is required to support the performance of the filters.

We study the halo mass function in the presence of the kurtosis type of primordial non-Gaussianity. The kurtosis corresponds to the trispectrum as defined in Fourier space. The primordial trispectrum is commonly characterized by two parameters, $\tau_{\rm NL}$ and $g_{\rm NL}$. As applications of the derived non-Gaussian mass function, we consider the effect on the abundance of void structure, the effect on early star formation and on formation of the most massive object at high redshift. We show that by comparing the effects of primordial non-Gaussianity on cluster abundance with that on void abundance, we can distinguish between the skewness and the kurtosis types of primordial non-Gaussianity. As for early star formation, we show that the kurtosis type of primordial non-Gaussianity seems not to affect the reionization history of the Universe on average. However, at high redshifts (up to $z\simeq 20$) such non-Gaussianity does somewhat affect the early stages of reionization.

The Lyman alpha line is a robust tracer of high redshift galaxies. We present estimates of Lyman alpha emission from a protogalactic halo illuminated by UV background radiation fields with various intensities. For this purpose, we performed cosmological hydrodynamics simulations with the adaptive mesh refinement code FLASH, including a detailed network for primordial chemistry,comprising the formation of primordial molecules, a multi-level model for the hydrogen atom as well as the photo-ionization and photo-dissociation processes in a UV background. We find that the presence of a background radiation field J_21 excites the emission of Lyman alpha photons, increasing the Lyman alpha luminosity up to two orders of magnitude. For a halo of \sim 10^10 M_sun, we find that a maximum flux of 5 \times 10^-15 erg cm^-2 s^-1 is obtained for J21 \times f_esc = 0.1, where f_esc is the escape fraction of the ionizing radiation. Depending on the environmental conditions, the flux may vary by three orders of magnitude. For J_21 \times f_esc > 0.1 the Lyman alpha luminosity decreases as the atomic hydrogen abundance becomes rather small. The fluxes derived here can be probed using Subaru and the upcoming James Webb Space Telescope. The emission of Lyman alpha photons is extended and comes from the envelope of the halo rather than its core. In the center of the halo, line trapping becomes effective above columns of 10^22 cm^-2 and suppresses the emission of Lyman alpha. In addition, cooling by primordial molecules may decrease the gas temperature in the central region, which further reduces Lyman alpha emission. In the central core, H_2 is photo-dissociated for a background flux of J_21 \geq 1000. For weaker radiation fields, i.e. J_21 < 0.1, H_2 and HD cooling are particularly strong in the center of the halo, leading to gas temperatures as low as \sim 100 K.

This paper introduces an amendment to radio interferometric calibration of sources below the noise level. The main idea is to employ the information of the stronger sources' measured signals as a plug-in criterion to solve for the weaker ones. For this purpose, we construct a number of source clusters, with centroids mainly near the strongest sources, assuming that the signals of the sources belonging to a single cluster are corrupted by almost the same errors. Due to this characteristic of clusters, each cluster is calibrated as a single source, using all the coherencies of its sources simultaneously. The obtained solutions for every cluster are assigned to all the cluster's sources. An illustrative example reveals the superiority of this calibration compared to the un-clustered calibration.

The measurement of the brightness temperature fluctuations of neutral hydrogen 21 cm lines from the Epoch of Reionisation (EoR) is expected to be a powerful tool for revealing the reionisation process. We study the 21 cm cross-correlation with Cosmic Microwave Background (CMB) temperature anisotropies, focusing on the effect of the patchy reionisation. We calculate, up to second order, the angular power spectrum of the cross-correlation between 21 cm fluctuations and the CMB kinetic Sunyaev-Zel'dovich effect (kSZ) from the EoR, using an analytical reionisation model. We show that the kSZ and the 21 cm fluctuations are anti-correlated on the scale corresponding to the typical size of an ionised bubble at the observed redshift of the 21 cm fluctuations. The amplitude of the angular power spectrum of the cross-correlation depends on the fluctuations of the ionised fraction. Especially, in a highly inhomogeneous reionisation model, the amplitude reaches the order of $100 \mu K^2$ at $\ell \sim 3000$. We also show that second order terms may help in distinguishing between reionisation histories.

The aim of the new generation of radio synthesis arrays such as LOFAR and SKA is to achieve much higher sensitivity, resolution and frequency coverage than what is available now, especially at low frequencies. To accomplish this goal, the accuracy of the calibration techniques used is of considerable importance. Moreover, since these telescopes produce huge amounts of data, speed of convergence of calibration is a major bottleneck. The errors in calibration are due to system noise (sky and instrumental) as well as the estimation errors introduced by the calibration technique itself, which we call solver noise. We define solver noise as the distance between the optimal solution (the true value of the unknowns, uncorrupted by the system noise) and the solution obtained by calibration. We present the Space Alternating Generalized Expectation Maximization (SAGE) calibration technique, which is a modification of the Expectation Maximization algorithm, and compare its performance with the traditional Least Squares calibration based on the level of solver noise introduced by each technique. For this purpose, we develop statistical methods that use the calibrated solutions to estimate the level of solver noise. The SAGE calibration algorithm yields very promising results both in terms of accuracy and speed of convergence. The comparison approaches we adopt introduce a new framework for assessing the performance of different calibration schemes.

The formation of the first galaxies is accompanied by large accretion flows and virialization shocks, during which the gas is shock-heated to temperatures of $\sim10^4$ K, leading to potentially strong fluxes in the Lyman alpha line. Indeed, a number of Lyman alpha blobs has been detected at high redshift. In this letter, we explore the origin of such Lyman alpha emission using cosmological hydrodynamical simulations that include a detailed model of atomic hydrogen as a multi-level atom and the effects of line trapping with the adaptive mesh refinement code FLASH. We see that baryons fall into the center of a halo through cold streams of gas, giving rise to a Lyman alpha luminosity of at least $\rm 10^{44} erg s^{-1}$ at $\rm z=4.7$, similar to observed Lyman alpha blobs. We find that a Lyman alpha flux of $\rm 5.0\times 10^{-17} erg cm^{-2} s^{-1}$ emerges from the envelope of the halo rather than its center, where the photons are efficiently trapped. Such emission can be probed in detail with the upcoming James Webb Space Telescope (JWST) and will constitute an important probe of gas infall and accretion.

Simulations estimating the differential brightness temperature of the redshifted 21-cm from the epoch of reionization (EoR) often assume that the spin temperature is decoupled from the background CMB temperature and is much larger than it. Although a valid assumption towards the latter stages of the reionization process, it does not necessarily hold at the earlier epochs. Violation of this assumption will lead to fluctuations in differential brightness temperature that are neither driven by density fluctuations nor by HII regions. Therefore, it is vital to calculate the spin temperature self-consistently by treating the Lyman-alpha and collisional coupling of spin temperature to the kinetic temperature. In this paper we develop an extension to the BEARS algorithm, originally developed to model reionization history, to include these coupling effects. Here we simulate the effect in ionization and heating for three models in which the reionization is driven by stars, miniqsos or a mixture of both.We also perform a number of statistical tests to quantify the imprint of the self-consistent inclusion of the spin temperature decoupling from the CMB. We find that the evolution of the spin temperature has an impact on the measured signal specially at redshifts higher than 10 and such evolution should be taken into account when one attempts to interpret the observational data.

Numerous cosmological simulations have been performed to study the formation of the first objects. We present the results of high resolution 3-D cosmological simulations of primordial objects formation using the adaptive mesh refinement code FLASH by including in an approximate manner the radiative transfer effects of Lyman alpha photons. We compare the results of a Lyman alpha trapping case inside gas clouds with atomic and molecular hydrogen cooling cases.The principal objective of this research is to follow the collapse of a zero metallicity halo with an effective equation of state (that accounts for the trapping) and to explore the fate of a halo in each of the three cases, specifically, the impact of thermodynamics on fragmentation of halos.Our results show that in the case of Lyman alpha trapping, fragmentation is halted and a massive object is formed at the center of a halo. The temperature of the gas remains well above $10^{4}$ K and the halo is not able to fragment to stellar masses. In the atomic cooling case, gas collapses into one or two massive clumps in contrast to the Lyman alpha trapping case. For the molecular hydrogen cooling case, gas cools efficiently and fragments.The formation of massive primordial objects is thus strongly dependent on the thermodynamics of the gas. A salient feature of our results is that for the formation of massive objects, e.g. intermediate mass black holes, feedback effects are not required to suppress $H_{2}$ cooling, as molecular hydrogen is collisionally dissociated at temperatures higher than $10^{4}$ K as a consequence of Lyman alpha trapping.

In the coming years a new insight into galaxy formation and the thermal history of the Universe is expected to come from the detection of the highly redshifted cosmological 21 cm line. The cosmological 21 cm line signal is buried under Galactic and extragalactic foregrounds which are likely to be a few orders of magnitude brighter. Strategies and techniques for effective subtraction of these foreground sources require a detailed knowledge of their structure in both intensity and polarization on the relevant angular scales of 1-30 arcmin. We present results from observations conducted with the Westerbork telescope in the 140-160 MHz range with 2 arcmin resolution in two fields located at intermediate Galactic latitude, centred around the bright quasar 3C196 and the North Celestial Pole. They were observed with the purpose of characterizing the foreground properties in sky areas where actual observations of the cosmological 21 cm line could be carried out. The polarization data were analysed through the rotation measure synthesis technique. We have computed total intensity and polarization angular power spectra. Total intensity maps were carefully calibrated, reaching a high dynamic range, 150000:1 in the case of the 3C196 field. [abridged]

Radio astronomy is entering a new era with new and future radio observatories such as the Low Frequency Array and the Square Kilometer Array. We describe in detail an automated flagging pipeline and evaluate its performance. With only a fraction of the computational cost of correlation and its use of the previously introduced SumThreshold method, it is found to be both fast and unrivalled in its high accuracy. The LOFAR radio environment is analysed with the help of this pipeline. The high time and spectral resolution of LOFAR have resulted in an observatory where only a few percent of the data is lost due to RFI.

The 21-cm line fluctuations and the cosmic microwave background (CMB) are powerful probes of the epoch of reionisation of the universe. We study the potential of the cross-correlation between 21-cm line fluctuations and CMB anisotropy to obtain further constraints on the reionisation history. We compute analytically the 21-cm cross-correlation with the CMB temperature anisotropy and polarisation, and we calculate the signal-to-noise (SN) ratio for its detection with Planck together with LOFAR, MWA and SKA. We find, on the one hand, that the 21-cm cross-correlation signal with CMB polarisation from the instant reionisation can be detected with an SN ratio of $\sim 1$ for LOFAR and $\sim 10$ for SKA. On the other hand, we confirm that the detection of the 21-cm cross-correlation with CMB polarisation is practically infeasible.

One of the aims of the Low Frequency Array (LOFAR) Epoch of Reionization (EoR) project is to measure the power spectrum of variations in the intensity of redshifted 21-cm radiation from the EoR. The sensitivity with which this power spectrum can be estimated depends on the level of thermal noise and sample variance, and also on the systematic errors arising from the extraction process, in particular from the subtraction of foreground contamination. We model the extraction process using realistic simulations of the cosmological signal, the foregrounds and noise, and so estimate the sensitivity of the LOFAR EoR experiment to the redshifted 21-cm power spectrum. Detection of emission from the EoR should be possible within 360 hours of observation with a single station beam. Integrating for longer, and synthesizing multiple station beams within the primary (tile) beam, then enables us to extract progressively more accurate estimates of the power at a greater range of scales and redshifts. We discuss different observational strategies which compromise between depth of observation, sky coverage and frequency coverage. A plan in which lower frequencies receive a larger fraction of the time appears to be promising. We also study the nature of the bias which foreground fitting errors induce on the inferred power spectrum, and discuss how to reduce and correct for this bias. The angular and line-of-sight power spectra have different merits in this respect, and we suggest considering them separately in the analysis of LOFAR data.

The Epoch of Reionization (EoR) is the epoch in which hydrogen in the Universe reionize after the "Dark Ages". This is the second of two major phase transitions that hydrogen in the Universe underwent, the first phase being the recombination era in which hydrogen became neutral at redshift about 1100. The EoR, occurs around z of 10 and is probably caused by the first radiation emitting astrophysical sources, hence it is crucial to our understanding of when and how the Universe "decided" to start forming astrophysical objects and how that influenced subsequent structure formation in the Universe. As such, the EoR is related to many fundamental questions in cosmology, galaxy formation, quasars and very metal poor stars; all are foremost research issues in modern astrophysics. The redshifted 21 cm hyperfine line is widely considered as the most promising probe for studying the EoR in detail. In the near future a number of low frequency radio telescopes (LOFAR, MWA, GMRT and SKA) will be able to observe the 21 cm radiation arriving from the high redshift Universe. In this paper I present our current picture of the ionization process, review the 21 cm line physics and discuss the challenges that the current generation experiments are expected to face. Finally, I discuss the potential of SKA in exploring the EoR and the Universe's Dark Ages.

We describe and compare several post-correlation radio frequency interference classification methods. As data sizes of observations grow with new and improved telescopes, the need for completely automated, robust methods for radio frequency interference mitigation is pressing. We investigated several classification methods and find that, for the data sets we used, the most accurate among them is the SumThreshold method. This is a new method formed from a combination of existing techniques, including a new way of thresholding. This iterative method estimates the astronomical signal by carrying out a surface fit in the time-frequency plane. With a theoretical accuracy of 95% recognition and an approximately 0.1% false probability rate in simple simulated cases, the method is in practice as good as the human eye in finding RFI. In addition it is fast, robust, does not need a data model before it can be executed and works in almost all configurations with its default parameters. The method has been compared using simulated data with several other mitigation techniques, including one based upon the singular value decomposition of the time-frequency matrix, and has shown better results than the rest.

The Universe's Epoch of Reionization (EoR) can be studied using a number of observational probes that provide complementary or corroborating information. Each of these probes suffers from its own systematic and statistical uncertainties. It is therefore useful to consider the mutual information that these data sets contain. In this paper we present a cross-correlation study between the kinetic Sunyaev-Zel'dovich effect (kSZ) -- produced by the scattering of CMB photons off free electrons produced during the reionization process -- and the cosmological 21cm signal -- which reflects the neutral hydrogen content of the Universe, as a function of redshift. The study is carried out using a simulated reionization history in 100Mpc/h scale N-body simulations with radiative transfer. In essence we find that the two probes anti-correlate. The significance of the anti-correlation signal depends on the extent of the reionization process, wherein extended histories result in a much stronger signal compared to instantaneous cases. Unfortunately however, once the primary CMB fluctuations are included into our simulation they serve as a source of large correlated noise that renders the cross-correlation signal insignificant, regardless of the reionization scenario.

We present the first results from a series of observations conducted with the Westerbork telescope in the 140--160 MHz range with a 2 arcmin resolution aimed at characterizing the properties of the foregrounds for epoch of reionization experiments. For the first time we have detected fluctuations in the Galactic diffuse emission on scales greater than 13 arcmin at 150 MHz, in the low Galactic latitude area known as Fan region. Those fluctuations have an $rms$ of 14 K. The total intensity power spectrum shows a power--law behaviour down to $\ell \sim 900$ with slope $\beta^I_\ell = -2.2 \pm 0.3$. The detection of diffuse emission at smaller angular scales is limited by residual point sources. We measured an $rms$ confusion noise of $\sim$3 mJy beam$^{-1}$. Diffuse polarized emission was also detected for the first time at this frequency. The polarized signal shows complex structure both spatially and along the line of sight. The polarization power spectrum shows a power--law behaviour down to $\ell \sim 2700$ with slope $\beta^P_\ell = -1.65 \pm 0.15$. The $rms$ of polarization fluctuations is 7.2 K on 4 arcmin scales. By extrapolating the measured spectrum of total intensity emission, we find a contamination on the cosmological signal of $\delta T= \sqrt{\ell (\ell+1) C^I_\ell / 2\pi} \sim 5.7$ K on 5 arcmin scales and a corresponding $rms$ value of $\sim$18.3 K at the same angular scale. The level of the polarization power spectrum is $\delta T \sim 3.3$ K on 5 arcmin scales. Given its exceptionally bright polarized signal, the Fan region is likely to represent an upper limit on the sky brightness at moderate and high Galactic latitude.

An obstacle to the detection of redshifted 21cm emission from the epoch of reionization (EoR) is the presence of foregrounds which exceed the cosmological signal in intensity by orders of magnitude. We argue that in principle it would be better to fit the foregrounds non-parametrically - allowing the data to determine their shape - rather than selecting some functional form in advance and then fitting its parameters. Non-parametric fits often suffer from other problems, however. We discuss these before suggesting a non-parametric method, Wp smoothing, which seems to avoid some of them. After outlining the principles of Wp smoothing we describe an algorithm used to implement it. We then apply Wp smoothing to a synthetic data cube for the LOFAR EoR experiment. The performance of Wp smoothing, measured by the extent to which it is able to recover the variance of the cosmological signal and to which it avoids leakage of power from the foregrounds, is compared to that of a parametric fit, and to another non-parametric method (smoothing splines). We find that Wp smoothing is superior to smoothing splines for our application, and is competitive with parametric methods even though in the latter case we may choose the functional form of the fit with advance knowledge of the simulated foregrounds. Finally, we discuss how the quality of the fit is affected by the frequency resolution and range, by the characteristics of the cosmological signal and by edge effects.

Detecting redshifted 21cm emission from neutral hydrogen in the early Universe promises to give direct constraints on the epoch of reionization (EoR). It will, though, be very challenging to extract the cosmological signal (CS) from foregrounds and noise which are orders of magnitude larger. Fortunately, the signal has some characteristics which differentiate it from the foregrounds and noise, and we suggest that using the correct statistics may tease out signatures of reionization. We generate mock datacubes simulating the output of the Low Frequency Array (LOFAR) EoR experiment. These cubes combine realistic models for Galactic and extragalactic foregrounds and the noise with three different simulations of the CS. We fit out the foregrounds, which are smooth in the frequency direction, to produce residual images in each frequency band. We denoise these images and study the skewness of the one-point distribution in the images as a function of frequency. We find that, under sufficiently optimistic assumptions, we can recover the main features of the redshift evolution of the skewness in the 21cm signal. We argue that some of these features - such as a dip at the onset of reionization, followed by a rise towards its later stages - may be generic, and give us a promising route to a statistical detection of reionization.

A number of experiments are set to measure the 21-cm signal of neutral hydrogen from the Epoch of Reionization (EoR). The common denominator of these experiments are the large data sets produced, contaminated by various instrumental effects, ionospheric distortions, RFI and strong Galactic and extragalactic foregrounds. In this paper, the first in a series, we present the Data Model that will be the basis of the signal analysis for the LOFAR (Low Frequency Array) EoR Key Science Project (LOFAR EoR KSP). Using this data model we simulate realistic visibility data sets over a wide frequency band, taking properly into account all currently known instrumental corruptions (e.g. direction-dependent gains, complex gains, polarization effects, noise, etc). We then apply primary calibration errors to the data in a statistical sense, assuming that the calibration errors are random Gaussian variates at a level consistent with our current knowledge based on observations with the LOFAR Core Station 1. Our aim is to demonstrate how the systematics of an interferometric measurement affect the quality of the calibrated data, how errors correlate and propagate, and in the long run how this can lead to new calibration strategies. We present results of these simulations and the inversion process and extraction procedure. We also discuss some general properties of the coherency matrix and Jones formalism that might prove useful in solving the calibration problem of aperture synthesis arrays. We conclude that even in the presence of realistic noise and instrumental errors, the statistical signature of the EoR signal can be detected by LOFAR with relatively small errors. A detailed study of the statistical properties of our data model and more complex instrumental models will be considered in the future.

The cosmic microwave background (CMB) polarisation and the 21 cm line fluctuations are powerful probes of cosmological reionisation. We study how the cross-correlation between the CMB polarisation (E-modes) and the 21 cm line fluctuations can be used to gain further understanding of the reionisation history, within the framework of inhomogeneous reionisation. Since the E-mode polarisation reflects the amplitude of the quadrupole component of the CMB temperature fluctuations, the angular power spectrum of the cross-correlation exhibits oscillations at all multipoles. The first peak of the power spectrum appears at the scale corresponding to the quadrupole at the redshift that is probed by the 21 cm line fluctuations. The peak reaches its maximum value in redshift when the average ionisation fraction of the universe is about half. On the other hand, on small scales, there is a damping that depends on the duration of reionisation. Thus, the cross-correlation between the CMB polarisation and the 21 cm line fluctuations has the potential to constrain accurately the epoch and the duration of reionisation.

Radio Interferometry is an essential method for astronomical observations. Self-calibration techniques have increased the quality of the radio astronomical observations (and hence the science) by orders of magnitude. Recently, there is a drive towards sensor arrays built using inexpensive hardware and distributed over a wide area acting as radio interferometers. Calibration of such arrays poses new problems in terms of computational cost as well as in performance of existing calibration algorithms. We consider the application of the Space Alternating Generalized Expectation Maximization (SAGE) \cite{Fess94} algorithm for calibration of radio interferometric arrays. Application to real data shows that this is an improvement over existing calibration algorithms that are based on direct, deterministic non linear optimization. As presented in this paper, we can improve the computational cost as well as the quality of the calibration using this algorithm.

We present an efficient method to generate large simulations of the Epoch of Reionization (EoR) without the need for a full 3-dimensional radiative transfer code. Large dark-matter-only simulations are post-processed to produce maps of the redshifted 21cm emission from neutral hydrogen. Dark matter haloes are embedded with sources of radiation whose properties are either based on semi-analytical prescriptions or derived from hydrodynamical simulations. These sources could either be stars or power-law sources with varying spectral indices. Assuming spherical symmetry, ionized bubbles are created around these sources, whose radial ionized fraction and temperature profiles are derived from a catalogue of 1-D radiative transfer experiments. In case of overlap of these spheres, photons are conserved by redistributing them around the connected ionized regions corresponding to the spheres. The efficiency with which these maps are created allows us to span the large parameter space typically encountered in reionization simulations. We compare our results with other, more accurate, 3-D radiative transfer simulations and find excellent agreement for the redshifts and the spatial scales of interest to upcoming 21cm experiments. We generate a contiguous observational cube spanning redshift 6 to 12 and use these simulations to study the differences in the reionization histories between stars and quasars. Finally, the signal is convolved with the LOFAR beam response and its effects are analyzed and quantified. Statistics performed on this mock data set shed light on possible observational strategies for LOFAR.

Future high redshift 21-cm experiments will suffer from a high degree of contamination, due both to astrophysical foregrounds and to non-astrophysical and instrumental effects. In order to reliably extract the cosmological signal from the observed data, it is essential to understand very well all data components and their influence on the extracted signal. Here we present simulated astrophysical foregrounds datacubes and discuss their possible statistical effects on the data. The foreground maps are produced assuming 5 deg x 5 deg windows that match those expected to be observed by the LOFAR Epoch-of-Reionization (EoR) key science project. We show that with the expected LOFAR-EoR sky and receiver noise levels, which amount to ~52 mK at 150 MHz after 300 hours of total observing time, a simple polynomial fit allows a statistical reconstruction of the signal. We also show that the polynomial fitting will work for maps with realistic yet idealised instrument response, i.e., a response that includes only a uniform uv coverage as a function of frequency and ignores many other uncertainties. Polarized galactic synchrotron maps that include internal polarization and a number of Faraday screens along the line of sight are also simulated. The importance of these stems from the fact that the LOFAR instrument, in common with all current interferometric EoR experiments has an instrumentally polarized response.

A one dimensional radiative transfer code is developed to track the ionization and heating pattern around the first miniquasars and Population III stars. The code follows the evolution of the ionization of the species of hydrogen and helium and the intergalactic medium temperature profiles as a function of redshift. The radiative transfer calculations show that the ionization signature of the first miniquasars and stars is very similar yet the heating pattern around the two is very different. Furthermore, the first massive miniquasars (~>10^5 M_{sun}) do produce large ionized bubbles around them, which can potentially be imaged directly using future radio telescopes. It is also shown that the ionized bubbles not only stay ionized for considerable time after the switching off of the source, but continue to expand for a short while due to secondary collisions prompted by the X-ray part of their spectra. Varying spectral shapes also produced sizable variations in ionized fraction and temperature profile. We also compare the radiative transfer results with the analytical approximation usually adopted for heating by miniquasars and find that, because of the inadequate treatment of the He species, the analytical approach leads to an underestimation of the temperature in the outer radii by a factor ~5. Population III stars - with masses in the range of 10 - 1000 M_{sun} and modelled as blackbodies at a temperature of 50000 K - are found to be efficient in ionizing their surroundings. Observational effects on the 21 cm brightness temperature, the thermal and kinetic Sunyaev-Ze'ldovich effects, are also studied in the context of the upcoming radio and microwave telescopes like LOFAR and SPT.

In the pre-reionization Universe, the regions of the inter-galactic medium (IGM) which are far from luminous sources are the last to undergo reionization. Until then, they should be scarcely affected by stellar radiation; instead, the X-ray emission from an early black hole (BH) population can have much larger influence. We investigate the effects of such emission, looking at a number of BH model populations (differing for the cosmological density evolution of BHs, the BH properties, and the spectral energy distribution of the BH emission). We find that BH radiation can easily heat the IGM to 10^3-10^4 K, while achieving partial ionization. The most interesting consequence of this heating is that BHs are expected to induce a 21-cm signal (delta T_b ~ 20-30 mK at z<~12) which should be observable with forthcoming experiments (e.g. LOFAR). We also find that at z<~10 BH emission strongly increases the critical mass separating star-forming and non-star-forming halos.

A simple algorithm is employed to deproject the two dimensional images of a pilot sample of 12 high-quality images of edge-on disk galaxies and to study their intrinsic 3 dimensional stellar distribution. We examine the radial profiles of the stars as a function of height above the plane and report a general trend within our sample of an increasing radial scalelength with height outside of the dustlane. This could be explained by the widespread presence of a thick disk component in these galaxies. In addition, the 3 dimensional view allows the study of the vertical distribution of the outer disk, beyond the break region, where we detect a significant increase in scalelength with vertical distance from the major axis for the truncated disks. This could be regarded as a weakening of the "truncation" with increasing distance from the plane. Furthermore, we conclude that the recently revised classification of the radial surface brightness profiles found for face-on galaxies is indeed independent of geometry. In particular, we find at least one example of each of the three main profile classes as defined in complete samples of intermediate to face-on galaxies: not-truncated, truncated and antitruncated. The position and surface brightness that mark the break location in the radial light distribution are found to be consistent with those of face-on galaxies.

A simple analytical model is used to calculate the X-ray heating of the IGM for a range of black hole masses. This process is efficient enough to decouple the spin temperature of the intergalactic medium from the cosmic microwave background (CMB) temperature and produce a differential brightness temperature of the order of $\sim 5-20 \mathrm{mK}$ out to distances as large as a few co-moving Mpc, depending on the redshift, black hole mass and lifetime. We explore the influence of two types of black holes, those with and without ionising UV radiation. The results of the simple analytical model are compared to those of a full spherically symmetric radiative transfer code. Two simple scenarios are proposed for the formation and evolution of black hole mass density in the Universe. The first considers an intermediate mass black hole that form as an end-product of Population III stars, whereas the second considers super-massive black holes that form directly through the collapse of massive halos with low spin parameter. These scenarios are shown not to violate any of the observational constraints, yet produce enough X-ray photons to decouple the spin-temperature from that of the CMB. This is an important issue for future high redshift 21 cm observations.

We measure the matter power spectrum from the Lyman-alpha forest of 31 quasar spectra spanning the redshift range of 1.6-3.6. The optical depth, $\tau$, for absorption of the intergalactic medium is obtained from the flux using the inversion method of Nusser & Haehnelt (1999). The optical depth is converted to density by using a simple power law relation, $\tau \propto (1+\delta)^\alpha$. The non-linear 1D power spectrum of the gas density is then inferred with a method that makes simultaneous use of the 1 and 2 point statistics of the flux and compared against theoretical models with a likelihood analysis. A Cold Dark Matter (CDM) model with standard cosmological parameters fits the data well. The power spectrum amplitude is measured to be (assuming a flat Universe), $\sigma_8 = (0.9 \pm 0.09)\times (\Omega_{\rm m}/0.3)^{-0.3}$, with $\alpha$ varying in the range of $1.56-1.8$ with redshift. Enforcing the same cosmological parameters in all four redshift bins, the likelihood analysis suggests some evolution in the density-temperature relation and the thermal smoothing length of the gas. The inferred evolution is consistent with that expected if reionization of \hep occurred at $z\sim 3.2$. A joint analysis with the WMAP results together with a prior on the Hubble constant as suggested by the HST key project data, yields values of $\Omega_{\rm m}$ and $\sigma_8$ that are consistent with the cosmological concordance model. We also perform a further inversion to obtain the linear 3D power spectrum of the matter density fluctuations.

We propose use of the thickness of the ionisation front as a discriminant between alternative modes of reionisation in the early universe, by stars or by miniquasars. Assuming a photoionisation-recombination balance, we find that for miniquasar sources the transition from neutral to ionised intergalactic medium is extended and has two features. The first is a sudden steep increase in the neutral fraction with a typical width of 5-10 comoving megaparsecs, depending on the miniquasar power. The second feature is a long wing that represents a much slower transition from neutral fraction of $\approx 0.8$ to 1. The angular resolution of LOFAR is expected to resolve these scales and will, therefore, play an important role in discriminating the hard sources of ionising photons from the stellar ones.

We introduce a simple linear equation relating the line-of-sight peculiar velocity and density contrast correlation functions. The relation, which we call the "Gaussian cell two-point energy-like equation", is valid in the distant-observer-limit and requires Gaussian smoothed fields. In the variance case, i.e., at zero lag, the equation is similar in its mathematical form to the Layzer-Irvine cosmic energy equation. \beta estimation with this equation from the PSC$z$ redshift galaxy survey and the SEcat catalogue of peculiar velocities is carried out, returning a value of \beta = 0.44+-0.08. The applicability of the method for the 6dF galaxy redshift and peculiar motions survey is demonstrated with mock data where it is shown that beta could be determined with ~5% accuracy. The prospects for constraining the dark energy equation of state with this method from the kinematic and thermal Sunyaev-Zel'dovich cluster surveys are discussed. The equation is also used to construct a nonparametric mass density power spectrum estimator from peculiar velocity data.

We reconstruct the underlying density field of the 2 degree Field Galaxy Redshift Survey (2dFGRS) for the redshift range 0.035<z<0.200 using the Wiener Filtering method. The Wiener Filter suppresses shot noise and accounts for selection and incompleteness effects. The method relies on prior knowledge of the 2dF power spectrum of fluctuations and the combination of matter density and bias parameters however the results are only slightly affected by changes to these parameters. We present maps of the density field in two different resolutions: 5 Mpc/h and 10 Mpc/h. We identify all major superclusters and voids in the survey. In particular, we find two large superclusters and two large local voids. A version of this paper with full set of colour maps can be found at http://www.ast.cam.ac.uk/~pirin.

A method for estimating redshifts of galaxy clusters based solely on resolved Sunyaev-Zel'dovich (SZ) images is proposed. Given a high resolution SZ cluster image (with FWHM of approx. 1 arcmin), the method indirectly measures its structure related parameters (amplitude, size, etc.) by fitting a model function to the higher order wavelet momenents of the cluster's SZ morphology. The applicability and accuracy of the wavelet method is assessed by applying it to maps of a set of clusters extracted from hydrodynamical simulations of cosmic structure formation. The parameters, derived by a fit to the spectrum of wavelet moments as a function of scale, are found to show a dependence on redshift z that is of the type x(z) = x_1 exp(-z/x_2) + x_3, where the monotony of this functional behaviour and the non-degeneracy of those parameters allow inversion and estimation of the redshift z. The average attainable accuracy in the z-estimation relative to 1+z is approx. 4-5% out to z = 1.2, which is comparable to photometric redshifts. For single-frequency SZ interferometers, where the ambient fluctuating CMB is the main noise source, the accuracy of the method drops slightly to <Delta z/(1+z)> = 6-7%.

The early reionisation of the Universe inferred from the WMAP polarisation results, if confirmed, poses a problem for the hypothesis that scale-invariant adiabatic density fluctuations account for large-scale structure and galaxy formation. One can only generate the required amount of early star formation if extreme assumptions are made about the efficiency and nature of early reionisation. We develop an alternative hypothesis that invokes an additional component of a non-scale-free isocurvature power spectrum together with the scale-free adiabatic power spectrum for inflation-motivated primordial density fluctuations. Such a component is constrained by the Lyman alpha forest observations, can account for the small-scale power required by spectroscopic gravitational lensing, and yields a source of early star formation that can reionise the universe at z~20 yet becomes an inefficient source of ionizing photons by z~10, thereby allowing the conventional adiabatic fluctuation component to reproduce the late thermal history of the intergalactic medium.

We apply a new algorithm, called the Unbiased Minimal Variance (hereafter UMV) estimator, to reconstruct the cosmic density and peculiar velocity fields in our local universe from the SEcat catalog of peculiar velocities comprising both early (ENEAR) and late type (SFI) galaxies. The reconstructed fields are compared with those predicted from the IRAS PSCz galaxy redshift survey to constrain the value of beta = Omega_m^{0.6}/b, where Omega_m and b are the mass density and the bias parameters. The comparison of the density and velocity fields is carried out within the same methodological framework, and leads, for the first time, to consistent values of beta, yielding beta = 0.57_{-0.13}^{+0.11} and beta = 0.51 +/- {0.06}, respectively. We find that the distribution of the density and velocity residuals, relative to their respective errors, is consistent with a Gaussian distribution with sigma approximately 1, indicating that the density field predicted from the PSCz is an acceptable fit to that deduced from the peculiar velocities of the SEcat galaxies.

I review the recent developments in the analysis of cosmic flow data, in particular, latest results of bulk flow measurements, comparison between redshift and peculiar velocity catalogs with emphasis on the measured value of the $\beta (=\Omega_m^{0.6}/b)$ parameter, and matter power spectrum estimates from galaxy peculiar velocity catalogs. Based on these developments, one can argue that most of the previous discrepancies in the interpretation of cosmic flow data, {\it maybe} with the exception of bulk flow measurements on scales $\gsim 100 h{^{-1}}{\rm Mpc}$, have either been resolved or fairly understood.

The temperature of the low-density intergalactic medium is set by the balance between adiabatic cooling resulting from the expansion of the universe, and photo-heating by the UV-background. We have analysed the Lyman-alpha forest of eleven high-resolution quasar spectra using wavelets, and find strong evidence of a marked jump in the temperature at the mean density, T_0, of 60 per cent around a redshift z=3.3, which we attribute to reionization of HeliumII. The jump can be seen in all three of our spectra that straddle redshift 3.3, at a significance of 99 per cent. Below z=3.1, our results are consistent with a smooth cooling down of the universe, as expected when adiabatic expansion dominates over photo-heating by a UV-background from QSOs and galaxies. We find no evidence of thermal fluctuations on scales greater than 5000 km/s larger than 50 per cent, which could be detected by our method, suggesting that the IGM follows a reasonably well-defined temperature-density relation. We demonstrate that the mean wavelet amplitude scales inversely with T_0 and calibrate the relation with hydrodynamical simulations. We find T_0= 12000K at z>3.6. Such high temperature suggest that Hydrogen reionization occured relatively recent.

The temperature of the diffuse, photo-heated intergalactic medium (IGM) depends on its reionization history because the thermal time scales are long. The widths of the hydrogen Lyman-alpha absorption lines seen in the spectra of distant quasars that arise in the IGM can be used to determine its temperature. We use a wavelet analysis of the Lyman-alpha forest region of quasar spectra to demonstrate that there is a relatively sudden increase in the line widths between redshifts z=3.5 and 3.0, which we associate with entropy injection resulting from the reionization of HeII. The subsequent fall-off in temperature after z~3.5, is consistent with a thermal evolution dominated by adiabatic expansion. If, as expected, the temperature also drops rapidly after hydrogen reionization, then the high temperatures inferred from the line widths before HeII reionization imply that hydrogen reionization occurred below redshift z=9.

A general method of deprojecting two-dimensional images to reconstruct the three dimensional structure of the projected object --specifically X-ray, Sunyaev-Zel'dovich (SZ) and gravitational lensing maps of rich clusters of galaxies -- assuming axial symmetry (Zaroubi et. al. 98), is considered. Here we test the applicability of the method for realistic, numerically simulated galaxy clusters, viewed from three orthogonal projections at four redshift outputs. We demonstrate that the assumption of axial symmetry is a good approximation for the 3D structure in this ensemble of galaxy clusters. Applying the method, we demonstrate that a unique determination of the cluster inclination angle is possible from comparison between the SZ and X-ray images and, independently, between SZ and surface density maps. Moreover, the results from these comparisons are found to be consistent with each other and with the full 3D structure inclination angle determination. The radial dark matter and gas density profiles as calculated from the actual and reconstructed 3D distributions show a very good agreement. The method is also shown to provide a direct determination of the baryon fraction in clusters, independent of the cluster inclination angle.

We present a new Unbiased Minimal Variance (UMV) estimator for the purpose of reconstructing the large--scale structure of the universe from noisy, sparse and incomplete data. Similar to the Wiener Filter (WF), the UMV estimator is derived by requiring the linear minimal variance solution given the data and an assumed prior model specifying the underlying field covariance matrix. However, unlike the WF, the minimization is carried out with the added constraint of an unbiased reconstructed mean field. The new estimator does not necessitate a noise model to estimate the underlying field; however, such a model is required for evaluating the errors at each point in space. The general application of the UMV estimator is to predict the values of the reconstructed field in un-sampled regions of space (e.g., interpolation in the unobserved Zone of Avoidance), and to dynamically transform from one measured field to another (e.g., inversion of radial peculiar velocities to over-densities). Here, we provide two very simple applications of the method. The first, is to recover a 1D signal from noisy, convolved data with gaps, e.g., CMB time-ordered data. The second application is a reconstruction of the density and 3D peculiar velocity fields from mock SEcat galaxy peculiar velocity catalogs.

The temperature of the low-density intergalactic medium is set by the balance between adiabatic cooling resulting from the expansion of the universe, and photo-heating by the UV-background. A sudden injection of entropy from the reionization will increase the temperature of the gas, leading to a broadening of the hydrogen Lyman-alpha absorption lines produced in the IGM, and observed in the spectra of background quasars. We present a method based on wavelets to characterise objectively the line widths of such absorption lines. We use high resolution hydrodynamical simulations to demonstrate that the algorithm can detect changes in temperature of order of 50 per cent on scales > 5000 km/s. We apply the method to a UVES/VLT spectrum of quasar 0055--269 (emission redshift=3.7) and detect at the 99 per cent confidence level a sudden increase in temperature below redshift z=3.3, which we interpret as evidence for HeliumII reionization.

We present a method for decomposing the cosmological velocity field in a given volume into its divergent component due to the density fluctuations inside the volume, and its tidal component due to the matter distribution outside the volume. The input consists of the density and velocity fields that are reconstructed either by POTENT or by Wiener Filter from a survey of peculiar velocities. The tidal field is further decomposed into a bulk velocity and a shear field. The method is applied here to the Mark III data within a sphere of radius 60 Mpc/h about the Local Group, and to the SFI data for comparison. We find that the tidal field contributes about half of the Local-Group velocity with respect to the CMB, with the tidal bulk velocity pointing to within ~ 30 degrees of the CMB dipole. The eigenvector with the largest eigenvalue of the shear tensor is aligned with the tidal bulk velocity to within ~ 40 degrees. The tidal field thus indicates the important dynamical role of a super attractor of mass (2-5) x 10^17 M_sun/h Omega^0.4 at ~ 150 Mpc/h, coinciding with the Shapley Concentration. There is also a hint for the dynamical role of two big voids in the Supergalactic Plane. The results are consistent for the two data sets and the two methods of reconstruction.

With the recent publication of the measurements of the radiation angular power spectrum from the BOOMERanG Antarctic flight (de Bernardis et al. 2000), it has become apparent that the currently favoured spatially-flat cold dark matter model (matter density parameter $\Omega_{\rm m}=0.3$, flatness being restored by a cosmological constant $\Omega_{\Lambda}=0.7$, Hubble parameter $h=0.65$, baryon density parameter $\Omega_{\rm b}h^2=0.02$) no longer provides a good fit to the data. We describe a phenomenological approach to resurrecting this paradigm. We consider a primordial power spectrum which incorporates a bump, arbitrarily placed at $k_{\rm b}$, and characterized by a Gaussian in log $k$ of standard deviation $\sigma_{\rm b}$ and amplitude ${\rm A}_{\rm b}$, that is superimposed onto a scale-invariant power spectrum. We generate a range of theoretical models that include a bump at scales consistent with cosmic microwave background and large-scale structure observations, and perform a simple $\chi^2$ test to compare our models with the $COBE$ DMR data and the recently published BOOMERanG and MAXIMA data. Unlike models that include a high baryon content, our models predict a low third acoustic peak. We find that low $\ell$ observations (20 $< \ell <$ 200) are a critical discriminant of the bumps because the transfer function has a sharp cutoff on the high $\ell$ side of the first acoustic peak...

We use the recently completed redshift-distance survey of nearby early-type galaxies (ENEAR) to measure the dipole component of the peculiar velocity field to a depth of cz ~ 6000 km/s. The sample consists of 1145 galaxies brighter than m_B=14.5 and cz < 7000 km/s, uniformly distributed over the whole sky, and 129 fainter cluster galaxies within the same volume. Most of the Dn-sigma distances were obtained from new spectroscopic and photometric observations conducted by this project, ensuring the homogeneity of the data over the whole sky. These 1274 galaxies are objectively assigned to 696 objects -- 282 groups/clusters and 414 isolated galaxies. We find that within a volume of radius ~ 6000 km/s, the best-fitting bulk flow has an amplitude of |vbulk| =220 +/- 42 km/s in the CMB restframe, pointing towards l=304 +/- 16 degrees, b=25 +/- 11 degrees. The error in the amplitude includes statistical, sampling and possible systematic errors. This solution is in excellent agreement with that obtained by the SFI Tully-Fisher survey. Our results suggest that most of the motion of the Local Group is due to fluctuations within 6000 km/s, in contrast to recent claims of large amplitude bulk motions on larger scales.

The temperature of the intergalactic medium (IGM) is an important factor in determining the line-widths of the absorption lines in the Lyman-alpha forest. We present a method to characterise the line-widths distribution using a decomposition of a Lyman-alpha spectrum in terms of discrete wavelets. Such wavelets form an orthogonal basis so the decomposition is unique. We demonstrate using hydrodynamic simulations that the mean and dispersion of the wavelet amplitudes is strongly correlated with both the temperature of the absorbing gas and its dependence on the gas density. Since wavelets are also localised in space, we are able to analyse the temperature distribution as a function of position along the spectrum. We illustrate how this method could be used to identify fluctuations in the IGM temperature which might result from late reionization or local effects.

Galaxy peculiar velocity data provide important dynamical clues to the structures obscured by the Zone of Avoidance (hereafter, ZOA) with resolution >~ 500km/s. This indirect probe complements the very challenging approach of directly mapping of the distribution of galaxies behind the Milky Way. In this work, the Wiener filter method is applied to reconstruct the 3D density and velocity distributions of the universe, within ~8000km/s, using SEcat the largest peculiar velocity catalog yet. This catalog is a combination of the SFI spiral galaxy peculiar velocity catalog and the newly completed nearby early-type galaxy peculiar velocity catalog, ENEAR. The recovered density is smoothed with 900km/s Gaussian. The main reconstructed structures are consistent with those extracted from the IRAS 1.2-Jy redshift galaxy catalogs. The revealed structures are identified and their robustness and significance are discussed.

We estimate the mass density fluctuations power spectrum (PS) on large scales by applying a maximum likelihood technique to the peculiarvelocity data of the recently completed redshift-distance survey of early-type galaxies (ENEAR). The general results are in agreement with the high amplitude power spectra found from similar analysis of other independent all-sky catalogs of peculiar velocity data such as MARK III and SFI. For Lambda & Open CDM COBE normalized PS models, the best-fit parameters are confined by a contour approximately defined by Omega h^{1.3}=0.377+-0.08 and Omega h^{0.88}=0.517+-0.083, respectively. Gamma-shape models, free of COBE normalization, resultsin the weak constraint of $\Gamma \geq 0.17$ and in the rather stringent constraint of sigma_8 Omega^{0.6}=1.0+-0.25. All quoted uncertainties refer to 3-sigma confidence-level. The calculated PS is used as a prior for Wiener reconstruction of the density field at different resolutions and the three-dimensional velocity field within a volume of radius ~80 Mpc/h. All major structures in the nearby universe are recovered and are well matched to those predicted from all-sky redshift surveys.

Using eigenmode expansion of the Mark-3 and SFI surveys of cosmological radial velocities a goodness-of-fit analysis is applied on a mode-by-mode basis. This differential analysis complements theBayesian maximum likelihood analysis that finds the most probable model given the data. Analyzing the surveys with their corresponding most likely models from the CMB-like family of models, as well as with the currently popular Lambda-CDM model, reveals a systematic inconsistency of the data with these `best' models. There is a systematic trend of the cumulative chi^2 to increase with the mode number (where the modes are sorted by decreasing order of the eigenvalues). This corresponds to a decrease of the chi^2 with the variance associated with a mode, and hence with its effective scale. It follows that the differential analysis finds that on small (large) scales the global analysis of all the modes `puts' less (more) power than actually required by the data. This observed trend might indicate one of the followings: a. The theoretical model (i.e. power spectrum) or the error model (or both) have an excess of power on large scales; b. Velocity bias; c. The velocity data suffers from still uncorrected systematic errors.

In this paper we pursue the origin of the non-Gaussianity determined by a bispectrum analysis of the COBE-DMR 4-year sky maps. The robustness of the statistic is demonstrated by the rebinning of the data into 12 coordinate systems. By computing the bispectrum statistic as a function of various data partitions - by channel, frequency, and time interval, we show that the observed non-Gaussian signal is driven by the 53 GHz data. This frequency dependence strongly rejects the hypothesis that the signal is cosmological in origin. A jack-knife analysis of the coadded 53 and 90 GHz sky maps reveals those sky pixels to which the bispectrum statistic is particularly sensitive. We find that by removing data from the 53 GHz sky maps for periods of time during which a known systematic effect perturbs the 31 GHz channels, the amplitudes of the bispectrum coefficients become completely consistent with that expected for a Gaussian sky. We conclude that the non-Gaussian signal detected by the normalised bispectrum statistic in the publicly available DMR sky maps is due to a systematic artifact. The impact of removing the affected data on estimates of the normalisation of simple models of cosmological anisotropy is negligible.

We estimate the power spectrum of mass density fluctuations from peculiar velocities of galaxies by applying an improved maximum-likelihood technique to the new all-sky SFI catalog. Parametric models are used for the power spectrum and the errors, and the free parameters are determined by assuming Gaussian velocity fields and errors and maximizing the probability of the data given the model. It has been applied to generalized CDM models with and without COBE normalization. The method has been carefully tested using artificial SFI catalogs. The most likely distance errors are found to be similar to the original error estimates in the SFI data. The general result that is not very sensitive to the prior model used is a relatively high amplitude of the power spectrum. For example, at k=0.1 h/Mpc we find P(k)\Omega^{1.2}=(4.4+/-1.7)X10^3 (Mpc/h)^3. An integral over the power spectrum yields \sigma_8\Omega^{0.6}=0.82+/-0.12. Model-dependent constraints on the cosmological parameters are obtained for families of CDM models. For example, for COBE-normalized \Lambda CDM models (scalar fluctuations only), the maximum-likelihood result can be approximated by \Omega n^{2} h_{60}^{1.3} =0.58+/-0.11. The formal random errors quoted correspond to the 90% confidence level. The total uncertainty, including systematic errors associated with nonlinear effects, may be larger by a factor of ~2. These results are in agreement with an application of a similar method to other data (Mark III).

We present an alternative, Bayesian method for large-scale reconstruction from observed peculiar velocity data. The method stresses a rigorous treatment of the random errors and it allows extrapolation into poorly sampled regions in real space or in k-space. A likelihood analysis is used to determine the fluctuation power spectrum, followed by a Wiener Filter (WF) analysis to obtain the minimum-variance mean fields of velocity and mass density. Constrained Realizations (CR) are then used to sample the statistical scatter about the WF mean field. The WF/CR method is applied as a demonstration to the Mark III data with 1200 km/s, 900 km/s, and 500 km/s resolutions. The main reconstructed structures are consistent with those extracted by the POTENT method. A comparison with the structures in the distribution of IRAS 1.2Jy galaxies yields a general agreement. The reconstructed velocity field is decomposed into its divergent and tidal components relative to a cube of +/-8000 km/s centered on the Local Group. The divergent component is very similar to the velocity field predicted from the distribution of IRAS galaxies. The tidal component is dominated by a bulk flow of 194 +/- 32 km/s towards the general direction of the Shapley concentration, and it also indicates a significant quadrupole.

We consider a general method of deprojecting 2D images to reconstruct the 3D structure of the projected object, assuming axial symmetry. The method consists of the application of the Fourier Slice Theorem to the general case where the axis of symmetry is not necessarily perpendicular to the line of sight, and is based on an extrapolation of the image Fourier transform into the so-called cone of ignorance. The method is specifically designed for the deprojection of X-ray, Sunyaev-Zeldovich (SZ) and gravitational lensing maps of rich clusters of galaxies. For known values of the Hubble constant, H0, and inclination angle, the quality of the projection depends on how exact is the extrapolation in the cone of ignorance. In the case where the axis of symmetry is perpendicular to the line of sight and the image is noise-free, the deprojection is exact. Given an assumed value of H0, the inclination angle can be found by matching the deprojected structure out of two different images of a given cluster, e.g., SZ and X-ray maps. However, this solution is degenerate with respect to its dependence on the assumed H0, and a third independent image of the given cluster is needed to determine H0 as well. The application of the deprojection algorithm to upcoming SZ, X-ray and weak lensing projected mass images of clusters will serve to determine the structure of rich clusters, the value of H0, and place constraints on the physics of the intra-cluster gas and its relation to the total mass distribution.

The power spectrum (PS) of mass density fluctuations, independent of `biasing', is estimated from the Mark III catalog of peculiar velocities using Bayesian statistics. A parametric model is assumed for the PS, and the free parameters are determined by maximizing the probability of the model given the data. The method has been tested using detailed mock catalogs. It has been applied to generalized CDM models with and without COBE normalization. The robust result for all the models is a relatively high PS, with $P(k) \Omega^{1.2} = (4.8 \pm 1.5) \times 10^3 (Mpc/h)^3$ at $k=0.1 h/Mpc$. An extrapolation to smaller scales using the different CDM models yields $\sigma_8 \Omega^{0.6} = 0.88 \pm 0.15$. The peak is weakly constrained to the range $0.02 \leq k \leq 0.06 h/Mpc$. These results are consistent with a direct computation of the PS (Kolatt & Dekel 1996). When compared to galaxy-density surveys, the implied values for $\beta$ ($\equiv \Omega^{0.6}/b$) are of order unity to within 25%. The parameters of the COBE-normalized, flat CDM model are confined by a 90% likelihood contour of the sort $\Omega h_{50}^\mu n^\nu = 0.8 \pm 0.2$, where $\mu = 1.3$ and $\nu = 3.4, 2.0$ for models with and without tensor fluctuations respectively. For open CDM the powers are $\mu = 0.95$ and $\nu = 1.4$ (no tensor fluctuations). A $\Gamma$-shape model free of COBE normalization yields only a weak constraint: $\Gamma = 0.4 \pm 0.2$.

The mass density field as extracted from peculiar velocities in our cosmological neighborhood is mapped back in time to the CMB in two ways. First, the density power spectrum ($P_k$) is translated into a temperature angular power spectrum of sub-degree resolution ($C_l$) and compared to observations. Second, the local density field is translated into a temperature map in a patch on the last-scattering surface of a distant observer. A likelihood analysis of the Mark III peculiar velocity data have constrained the range of parameters for $P_k$ within the family of COBE-normalized CDM models (Zaroubi et al 1996), favoring a slight tilt in the initial spectrum, $n<1$. The corresponding range of $C_l$'s is plotted against current observations, indicating that the CMB data can tighten the constraints further: only models with ``small'' tilt ($n\sim 0.9$) and ``high'' baryonic content ($\Omega_b \sim 0.1$) could survive the two data sets simultaneously. The local mass density field that has been recovered from the velocities via a Wiener method is convolved with a Boltzmann calculation to recover $10'$ resolution temperature maps as viewed from different directions. The extent of the CMB patch and the amplitude of fluctuations depend on the choice of cosmological parameters, e.g., the local $100\hmpc$ sphere corresponds to $90'$ to $30'$ at the CMB for $\Omega$ between 1 and 0 respectively. The phases of the temperature map are correlated with those of the density field, contrary to the contribution of the Sachs-Wolfe effect alone. This correlation suggests the possibility of an inverse reconstruction of the underlying density field from CMB data with interesting theoretical implications.

The power spectrum (PS) of {\it mass} density fluctuations, independent of ``biasing", is estimated from the Mark3 Catalog of Peculiar Velocities of galaxies using Bayesian statistics. A parametric model is assumed for the PS, and the free parameters are determined by maximizing the posterior probability of the model given the data. The method has been tested using detailed mock catalogs. We use generalized CDM models with and without COBE normalization. The robust result for all the models tested is $P(k) \Omega^{1.2} = (4.1 \pm 0.7)x10^3 (Mpc h^{-1})^3$ at $k=0.1 h Mpc^{-1}$, with the peak constrained to the range $0.03 < k < 0.06 h Mpc^{-1}$. It is consistent with a direct computation of the PS (Kolatt \& Dekel 1996). When compared to galaxy-density surveys, the implied values for $\beta$ ($\equiv \Omega^{0.6}/b$) are of order unity to within 25\%. A $\Gamma$-shape model, free of COBE normalization, is constrained by the velocity data to $\Gamma=0.5\pm 0.15$ and $\sigma_8 \Omega^{0.6}=0.85\pm 0.1$. Within the family of COBE-normalized CDM models, the best tilted model ($\Omega=1$, $n \approx 0.84 h_{50}^{-0.65}$) and the best open model ($n=1$, $\Omega \approx 0.75 h_{50}^{-1.0}$) are more likely than the best $\Lambda$ model ($n=1$, $\Lambda=1-\Omega$, $\Omega\approx 0.70 h_{50}^{-1.2}$). The most likely CDM model with $\Omega\leq 1$ is found to be of $\Omega=1$, $h\approx 0.75$, and a {\it tilted} spectrum of $n= 0.8\pm 0.02$ with tensor fluctuations.The tightest constraint for the tilted-$\Lambda$ models is of the sort $\Omega h_{50}^{1.2} n^\nu = 0.7 \pm 0.08$, with $\nu=3.8$ and $1.85$ with and without tensor fluctuations respectively.

The applicability of the highly idealized secondary infall model to `realistic' initial conditions is investigated. The collapse of proto-halos seeded by $3\sigma$ density perturbations to an Einstein--de Sitter universe is studied here for a variety of scale-free power spectra with spectral indices ranging from $n=1$ to $-2$. Initial conditions are set by the constrained realization algorithm and the dynamical evolution is calculated both analytically and numerically. The analytic calculation is based on the simple secondary infall model where spherical symmetry is assumed. A full numerical simulation is performed by a Tree N-body code where no symmetry is assumed. A hybrid calculation has been performed by using a monopole term code, where no symmetry is imposed on the particles but the force is approximated by the monopole term only. The main purpose of using such code is to suppress off-center mergers. In all cases studied here the rotation curves calculated by the two numerical codes are in agreement over most of the mass of the halos, excluding the very inner region, and these are compared with the analytically calculated ones. The main result obtained here, reinforces the foundings of many N-body experements, is that the collapse proceeds 'gently' and not {\it via} violent relaxation. There is a strong correlation of the final energy of individual particles with the initial one. In particular we find a preservation of the ranking of particles according to their binding energy. In cases where the analytic model predicts non-increasing rotation curves its predictions are confirmed by the simulations. Otherwise, sensitive dependence on initial conditions is found and the analytic model fails completely.

The formalism of Wiener filtering is applied to reconstruct, in terms of spherical harmonics, the projected $4\pi$ galaxy distribution of a mock IRAS 1.2 Jy catalog with a `Zone of Avoidance' $|b| = 15^\circ$. The Singular Value Decomposition al gorithm is utilized as an extra-regularizer in order to stabilize the inversion. This case sheds light on the amount of useful information contained in the data, and also suggests a method for obtaining an `optimal' functional basis, based on our \prior knowledge, for the purposes of likelihood analysis and Wiener reconstruction of cosmological data sets. The applicability of such a functional basis for redshift galaxy catalogs is also discussed.

The formalism of Wiener filtering is developed here for the purpose of reconstructing the large scale structure of the universe from noisy, sparse and incomplete data. The method is based on a linear minimum variance solution, given data and an assumed \prior model which specifies the covariance matrix of the field to be reconstructed. While earlier applications of the Wiener filter have focused on estimation, namely suppressing the noise in the measured quantities, we extend the method here to perform both prediction and dynamical reconstruction. The Wiener filter is used to predict the values of unmeasured quantities, such as the density field in un-sampled regions of space, or to deconvolve blurred data. The method is developed, within the context of linear gravitational instability theory, to perform dynamical reconstruction of one field which is dynamically related to some other observed field. This is the case, for example, in the reconstruction of the real space galaxy distribution from its redshift distribution When the field to be reconstructed is a Gaussian random field, such as the primordial perturbation field predicted by the canonical model of cosmology, the Wiener filter can be pushed to its fullest potential. In such a case the Wiener estimator coincides with the Bayesian estimator designed to maximize the {\it posterior} probability. The Wiener filter can be also derived by assuming a quadratic regularization function, in analogy with the `Maximum Entropy' method. The mean field obtained by the minimal variance solution can be supplemented with constrained realizations of the Gaussian field to

The clustering in redshift space is studied here to first order within the framework of gravitational instability. The distortion introduced by the peculiar velocities of galaxies results in anisotropy in the galaxy distribution and mode-mode coupling when analyzed in Fourier space. An exact linear calculation of the full covariance matrix in both the real and Fourier space is presented here. The explicit dependence on $\Omeg$ and the biasing parameter is calculated and its potential use as a probe of these parameters is analyzed. It is shown that Kaiser's formalism can be applied only to a data set that subtends a small solid angle on the sky, and therefore cannot be used in the case of all sky surveys. The covariance matrix in the real space is calculated explicitly for {\it CDM} model, where the behavior along and perpendicular to the line of sight is shown.

We present a new method for recovering the cosmological density, velocity, and potential fields from all-sky redshift catalogues. The method is based on an expansion of the fields in orthogonal radial (Bessel) and angular (spherical harmonic) functions. In this coordinate system, peculiar velocities introduce a coupling of the radial harmonics describing the density field in redshift space but leave the angular modes unaffected. In the harmonic transform space, this radial coupling is described by a distortion matrix which can be computed analytically within the context of linear theory; the redshift space harmonics can then be converted to their real space values by inversion of this matrix. Statistical noise is mitigated by regularizing the matrix inversion with a Wiener filter. The method yields a minimum variance estimate of the density field in real space. In this coordinate system, the minimum variance harmonics of the peculiar velocity and potential fields are related to those of the density field by simple linear transformations. Tests of the method with simulations of a CDM universe and comparison with previously proposed methods demonstrate it to be a very promising new reconstruction method for the local density and velocity field. A first application to the 1.2 Jy IRAS redshift survey is presented.

We derive an optimal linear filter to suppress the noise from the COBE DMR sky maps for a given power spectrum. We then apply the filter to the first-year DMR data, after removing pixels within $20^\circ$ of the Galactic plane from the data. The filtered data have uncertainties 12 times smaller than the noise level of the raw data. We use the formalism of constrained realizations of Gaussian random fields to assess the uncertainty in the filtered sky maps. In addition to improving the signal-to-noise ratio of the map as a whole, these techniques allow us to recover some information about the CMB anisotropy in the missing Galactic plane region. From these maps we are able to determine which hot and cold spots in the data are statistically significant, and which may have been produced by noise. In addition, the filtered maps can be used for comparison with other experiments on similar angular scales.

The analysis of whole-sky galaxy surveys commonly suffers from the problems of shot-noise and incomplete sky coverage (e.g. at the Zone of Avoidance). The orthogonal set of spherical harmonics is utilized here to expand the observed galaxy distribution. We show that in the framework of Bayesian statistics and Gaussian random fields the $4 \pi$ harmonics can be recovered and the shot-noise can be removed, giving the most probable picture of the underlying density field. The correction factor from observed to reconstructed harmonics turns out to be the well-known Wiener filter (the ratio of signal to signal+noise), which is also derived by requiring minimum variance. We apply the method to the projected 1.2 Jy IRAS survey. The reconstruction confirms the connectivity of the Supergalactic Plane across the Galactic Plane (at Galactic longitude $l \sim 135^o$ and $l \sim 315^o$) and the Puppis cluster behind the Galactic Plane ($ l \sim 240^o$). The method can be extended to 3-D in both real and redshift space, and applied to other cosmic phenomena such as the COBE Microwave Background maps.