We present SciServer, a science platform built and supported by the Institute for Data Intensive Engineering and Science at the Johns Hopkins University. SciServer builds upon and extends the SkyServer system of server-side tools that introduced the astronomical community to SQL (Structured Query Language) and has been serving the Sloan Digital Sky Survey catalog data to the public. SciServer uses a Docker/VM based architecture to provide interactive and batch mode server-side analysis with scripting languages like Python and R in various environments including Jupyter (notebooks), RStudio and command-line in addition to traditional SQL-based data analysis. Users have access to private file storage as well as personal SQL database space. A flexible resource access control system allows users to share their resources with collaborators, a feature that has also been very useful in classroom environments. All these services, wrapped in a layer of REST APIs, constitute a scalable collaborative data-driven science platform that is attractive to science disciplines beyond astronomy.

Cosmological N-body simulations are crucial for understanding how the Universe evolves. Studying large-scale distributions of matter in these simulations and comparing them to observations usually involves detecting dense clusters of particles called "halos,'' which are gravitationally bound and expected to form galaxies. However, traditional cluster finders are computationally expensive and use massive amounts of memory. Recent work by Liu et al (Liu et al. (2015)) showed the connection between cluster detection and memory-efficient streaming algorithms and presented a halo finder based on heavy hitter algorithm. Later, Ivkin et al. (Ivkin et al. (2018)) improved the scalability of suggested streaming halo finder with efficient GPU implementation. Both works map particles' positions onto a discrete grid, and therefore lose the rest of the information, such as their velocities. Therefore, two halos travelling through each other are indistinguishable in positional space, while the velocity distribution of those halos can help to identify this process which is worth further studying. In this project we analyze data from the Millennium Simulation Project (Springel et al. (2005)) to motivate the inclusion of the velocity into streaming method we introduce. We then demonstrate a use of suggested method, which allows one to find the same halos as before, while also detecting those which were indistinguishable in prior methods.

Cosmological $N$-body simulations play a vital role in studying models for the evolution of the Universe. To compare to observations and make a scientific inference, statistic analysis on large simulation datasets, e.g., finding halos, obtaining multi-point correlation functions, is crucial. However, traditional in-memory methods for these tasks do not scale to the datasets that are forbiddingly large in modern simulations. Our prior paper proposes memory-efficient streaming algorithms that can find the largest halos in a simulation with up to $10^9$ particles on a small server or desktop. However, this approach fails when directly scaling to larger datasets. This paper presents a robust streaming tool that leverages state-of-the-art techniques on GPU boosting, sampling, and parallel I/O, to significantly improve performance and scalability. Our rigorous analysis of the sketch parameters improves the previous results from finding the centers of the $10^3$ largest halos to $\sim 10^4-10^5$, and reveals the trade-offs between memory, running time and number of halos. Our experiments show that our tool can scale to datasets with up to $\sim 10^{12}$ particles while using less than an hour of running time on a single GPU Nvidia GTX 1080.

Probabilistic cross-identification has been successfully applied to a number of problems in astronomy from matching simple point sources to associating stars with unknown proper motions and even radio observations with realistic morphology. Here we study the Bayes factor for clustered objects and focus in particular on galaxies to assess the effect of typical angular correlations. Numerical calculations provide the modified relationship, which (as expected) suppresses the evidence for the associations at the shortest separations where the 2-point auto-correlation function is large. Ultimately this means that the matching probability drops at somewhat shorter scales than in previous models.

We study the quenching of star formation as a function of redshift, environment and stellar mass in the galaxy formation simulations of Henriques et al. (2015), which implement an updated version of the Munich semi-analytic model (L-GALAXIES) on the two Millennium Simulations after scaling to a Planck cosmology. In this model massive galaxies are quenched by AGN feedback depending on both black hole and hot gas mass, and hence indirectly on stellar mass. In addition, satellite galaxies of any mass can be quenched by ram-pressure or tidal stripping of gas and through the suppression of gaseous infall. This combination of processes produces quenching efficiencies which depend on stellar mass, host halo mass, environment density, distance to group centre and group central galaxy properties in ways which agree qualitatively with observation. Some discrepancies remain in dense regions and close to group centres, where quenching still seems too efficient. In addition, although the mean stellar age of massive galaxies agrees with observation, the assumed AGN feedback model allows too much ongoing star formation at late times. The fact that both AGN feedback and environmental effects are stronger in higher density environments leads to a correlation between the quenching of central and satellite galaxies which roughly reproduces observed conformity trends inside haloes.

We adapt the L-Galaxies semi-analytic model to follow the star-formation histories (SFH) of galaxies -- by which we mean a record of the formation time and metallicities of the stars that are present in each galaxy at a given time. We use these to construct stellar spectra in post-processing, which offers large efficiency savings and allows user-defined spectral bands and dust models to be applied to data stored in the Millennium data repository. We contrast model SFHs from the Millennium Simulation with observed ones from the VESPA algorithm as applied to the SDSS-7 catalogue. The overall agreement is good, with both simulated and SDSS galaxies showing a steeper SFH with increased stellar mass. The SFHs of blue and red galaxies, however, show poor agreement between data and simulations, which may indicate that the termination of star formation is too abrupt in the models. The mean star-formation rate (SFR) of model galaxies is well-defined and is accurately modelled by a double power law at all redshifts: SFR proportional to $1/(x^{-1.39}+x^{1.33})$, where $x=(t_a-t)/3.0\,$Gyr, $t$ is the age of the stars and $t_a$ is the loopback time to the onset of galaxy formation; above a redshift of unity, this is well approximated by a gamma function: SFR proportional to $x^{1.5}e^{-x}$, where $x=(t_a-t)/2.0\,$Gyr. Individual galaxies, however, show a wide dispersion about this mean. When split by mass, the SFR peaks earlier for high-mass galaxies than for lower-mass ones, and we interpret this downsizing as a mass-dependence in the evolution of the quenched fraction: the SFHs of star-forming galaxies show only a weak mass dependence.

We present the public data release of halo and galaxy catalogues extracted from the EAGLE suite of cosmological hydrodynamical simulations of galaxy formation. These simulations were performed with an enhanced version of the GADGET code that includes a modified hydrodynamics solver, time-step limiter and subgrid treatments of baryonic physics, such as stellar mass loss, element-by-element radiative cooling, star formation and feedback from star formation and black hole accretion. The simulation suite includes runs performed in volumes ranging from 25 to 100 comoving megaparsecs per side, with numerical resolution chosen to marginally resolve the Jeans mass of the gas at the star formation threshold. The free parameters of the subgrid models for feedback are calibrated to the redshift z=0 galaxy stellar mass function, galaxy sizes and black hole mass - stellar mass relation. The simulations have been shown to match a wide range of observations for present-day and higher-redshift galaxies. The raw particle data have been used to link galaxies across redshifts by creating merger trees. The indexing of the tree produces a simple way to connect a galaxy at one redshift to its progenitors at higher redshift and to identify its descendants at lower redshift. In this paper we present a relational database which we are making available for general use. A large number of properties of haloes and galaxies and their merger trees are stored in the database, including stellar masses, star formation rates, metallicities, photometric measurements and mock gri images. Complex queries can be created to explore the evolution of more than 10^5 galaxies, examples of which are provided in appendix. (abridged)

We have updated the Munich galaxy formation model to the Planck first-year cosmology, while modifying the treatment of baryonic processes to reproduce recent data on the abundance and passive fractions of galaxies from z= 3 down to z=0. Matching these more extensive and more precise observational results requires us to delay the reincorporation of wind ejecta, to lower the surface density threshold for turning cold gas into stars, to eliminate ram-pressure stripping in haloes less massive than ~10^14 Msun, and to modify our model for radio mode feedback. These changes cure the most obvious failings of our previous models, namely the overly early formation of low-mass galaxies and the overly large fraction of them that are passive at late times. The new model is calibrated to reproduce the observed evolution both of the stellar mass function and of the distribution of star formation rate at each stellar mass. Massive galaxies (M>10^11 [Msun]) assemble most of their mass before z=1 and are predominantly old and passive at z=0, while lower mass galaxies assemble later and, for M<10^9.5 (Msun), are still predominantly blue and star forming at z=0. This phenomenological but physically based model allows the observations to be interpreted in terms of the efficiency of the various processes that control the formation and evolution of galaxies as a function of their stellar mass, gas content, environment and time.

We present an algorithm using Principal Component Analysis (PCA) to subtract galaxies from imaging data, and also two algorithms to find strong, galaxy-scale gravitational lenses in the resulting residual image. The combined method is optimized to find full or partial Einstein rings. Starting from a pre-selection of potential massive galaxies, we first perform a PCA to build a set of basis vectors. The galaxy images are reconstructed using the PCA basis and subtracted from the data. We then filter the residual image with two different methods. The first uses a curvelet (curved wavelets) filter of the residual images to enhance any curved/ring feature. The resulting image is transformed in polar coordinates, centered on the lens galaxy center. In these coordinates, a ring is turned into a line, allowing us to detect very faint rings by taking advantage of the integrated signal-to-noise in the ring (a line in polar coordinates). The second way of analysing the PCA-subtracted images identifies structures in the residual images and assesses whether they are lensed images according to their orientation, multiplicity and elongation. We apply the two methods to a sample of simulated Einstein rings, as they would be observed with the ESA Euclid satellite in the VIS band. The polar coordinates transform allows us to reach a completeness of 90% and a purity of 86%, as soon as the signal-to-noise integrated in the ring is higher than 30, and almost independent of the size of the Einstein ring. Finally, we show with real data that our PCA-based galaxy subtraction scheme performs better than traditional subtraction based on model fitting to the data. Our algorithm can be developed and improved further using machine learning and dictionary learning methods, which would extend the capabilities of the method to more complex and diverse galaxy shapes.

In this document and the accompanying documents we describe a data model (Simulation Data Model) describing numerical computer simulations of astrophysical systems. The primary goal of this standard is to support discovery of simulations by describing those aspects of them that scientists might wish to query on, i.e. it is a model for meta-data describing simulations. This document does not propose a protocol for using this model. IVOA protocols are being developed and are supposed to use the model, either in its original form or in a form derived from the model proposed here, but more suited to the particular protocol. The SimDM has been developed in the IVOA Theory Interest Group with assistance of representatives of relevant working groups, in particular DM and Semantics.

The Photometry Data Model (PhotDM) standard describes photometry filters, photometric systems, magnitude systems, zero points and its interrelation with the other IVOA data models through a simple data model. Particular attention is given necessarily to optical photometry where specifications of magnitude systems and photometric zero points are required to convert photometric measurements into physical flux density units.

We identify 42 candidate groups lying between 1.8<z<3.0 from a sample of 3502 galaxies with spectroscopic redshifts in the zCOSMOS-deep redshift survey within the same redshift interval. These systems contain three to five spectroscopic galaxies that lie within 500kpc in projected distance (in physical space) and within 700km/s in velocity. Based on extensive analysis of mock catalogues that have been generated from the Millennium simulation, we examine the likely nature of these systems at the time of observation, and what they will evolve into down to the present epoch. Although few of the "member" galaxies are likely to reside in the same halo at the epoch we observe them, 50% of the systems will eventually bring them all into the same halo, and almost all (93%) will have at least part of the member galaxies in the same halo by the present epoch. Most of the candidate groups can therefore be described as "proto-groups". An estimate of the overdensities is also consistent with the idea that these systems are being seen at the start of the assembly process. We also examine present-day haloes and ask whether their progenitors would have been seen amongst our candidate groups. For present-day haloes between 10^14-10^15Msun/h, 35% should have appeared amongst our candidate groups, and this would have risen to 70% if our survey had been fully-sampled, so we can conclude that our sample can be taken as representative of a large fraction of such systems. There is a clear excess of massive galaxies above 10^10Msun around the locations of the candidate groups in a large independent COSMOS photo-z sample, but we see no evidence in this latter data for any colour differentiation with respect to the field. This is however consistent with the idea that such differentiation arises in satellite galaxies, as indicated at z<1, if the candidate groups are indeed only starting to be assembled.

We utilize the Millennium-II simulation databases to study the spin bias of dark subhalos in the Local Group-like systems which have two prominent satellites with comparable masses. Selecting the group-size halos with total mass similar to that of the Local Group (LG) from the friends-of-friends halo catalog and locating their subhalos from the substructure catalog, we determine the most massive (main) and second to the most massive (submain) ones among the subhalos hosted by each selected halo. When the dimensionless spin parameter (lambda) of each subhalo is derived from its specific angular momentum and circular velocity at virial radius, a signal of correlation is detected between the spin parameters of the subhalos and the main-to-submain mass ratios of their host halos at z=0: The higher main-to-submain mass ratio a host halo has, the higher mean spin parameter its subhalos have. It is also found that the correlations exist even for the subhalo progenitors at z=0.5 and z=1. Our interpretation of this result is that the subhalo spin bias is not a transient effect but an intrinsic property of a LG-like system with higher main-to- submain mass ratio, caused by stronger anisotropic stress in the region. A cosmological implication of our result is also discussed.

Using the technique of Angulo & White (2010) we scale the Millennium and Millennium-II simulations of structure growth in a LCDM universe from the cosmological parameters with which they were carried out (based on first-year results from the Wilkinson Microwave Anisotropy Probe, WMAP1) to parameters consistent with the seven-year WMAP data (WMAP7). We implement semi-analytic galaxy formation modelling on both simulations in both cosmologies to investigate how the formation, evolution and clustering of galaxies are predicted to vary with cosmological parameters. The increased matter density Omega_m and decreased linear fluctuation amplitude sigma8 in WMAP7 have compensating effects, so that the abundance and clustering of dark halos are predicted to be very similar to those in WMAP1 for z <= 3. As a result, local galaxy properties can be reproduced equally well in the two cosmologies by slightly altering galaxy formation parameters. The evolution of the galaxy populations is then also similar. In WMAP7, structure forms slightly later. This shifts the peak in cosmic star formation rate to lower redshift, resulting in slightly bluer galaxies at z=0. Nevertheless, the model still predicts more passive low-mass galaxies than are observed. For rp< 1Mpc, the z=0 clustering of low-mass galaxies is weaker for WMAP7 than for WMAP1 and closer to that observed, but the two cosmologies give very similar results for more massive galaxies and on large scales. At z>1 galaxies are predicted to be more strongly clustered for WMAP7. Differences in galaxy properties, including, clustering, in these two cosmologies are rather small up to redshift 3. Given that there are still considerable residual uncertainties in galaxy formation models, it is very difficult to distinguish WMAP1 from WMAP7 through observations of galaxy properties or their evolution.

[abridged] We construct lightcones for the semi-analytic galaxy formation simulation of Guo et al. (2011) and make mock catalogues for comparison with deep high-redshift surveys. Photometric properties are calculated with two different stellar population synthesis codes (Bruzual & Charlot 2003; Maraston 2005) in order to study sensitivity to this aspect of the modelling. The catalogues are publicly available and include photometry for a large number of observed bands from 4000{\deg}A to 6{\mu}m, as well as rest-frame photometry and intrinsic properties of the galaxies. Guo et al. (2011) tuned their model to fit the low-redshift galaxy population but noted that at z > 1 it overpredicts the abundance of galaxies below the "knee" of the stellar mass function. Here we extend the comparison to deep galaxy counts in the B, i, J, K and IRAC 3.6{\mu}m, 4.5{\mu}m and 5.8{\mu}m bands, to the redshift distributions of K and 5.8{\mu}m selected galaxies, and to the evolution of rest-frame luminosity functions in the B and K bands. The B, i and J counts are well reproduced, but at longer wavelengths the overabundant high-redshift galaxies produce excess faint counts. The predicted redshift distributions for K and 5.8{\mu}m selected samples highlight the effect of emission from thermally pulsing AGB stars. The full treatment of Maraston (2005) predicts three times as many z~2 galaxies in faint 5.8{\mu}m selected samples as the model of Bruzual & Charlot (2003), whereas the two models give similar predictions for K-band selected samples. Although luminosity functions are adequately reproduced out to z~3 in rest-frame B, the same is true at rest-frame K only if TP-AGB emission is included, and then only at high luminosity. Fainter than L* the two synthesis models agree but overpredict the number of galaxies, another reflection of the overabundance of ~10^10M\odot model galaxies at z > 1.

We apply Monte Carlo Markov Chain (MCMC) methods to large-scale simulations of galaxy formation in a LambdaCDM cosmology in order to explore how star formation and feedback are constrained by the observed luminosity and stellar mass functions of galaxies. We build models jointly on the Millennium and Millennium-II simulations, applying fast sampling techniques which allow observed galaxy abundances over the ranges 7<log(M*/Msun)<12 and z=0 to z=3 to be used simultaneously as constraints in the MCMC analysis. When z=0 constraints alone are imposed, we reproduce the results of previous modelling by Guo et al. (2012), but no single set of parameters can reproduce observed galaxy abundances at all redshifts simultaneously, reflecting the fact that low-mass galaxies form too early and thus are overabundant at high redshift in this model. The data require the efficiency with which galactic wind ejecta are reaccreted to vary with redshift and halo mass quite differently than previously assumed, but in a similar way as in some recent hydrodynamic simulations of galaxy formation. We propose a specific model in which reincorporation timescales vary inversely with halo mass and are independent of redshift. This produces an evolving galaxy population which fits observed abundances as a function of stellar mass, B- and K-band luminosity at all redshifts simultaneously. It also produces a significant improvement in two other areas where previous models were deficient. It leads to present day dwarf galaxy populations which are younger, bluer, more strongly star-forming and more weakly clustered on small scales than before, although the passive fraction of faint dwarfs remains too high.

Simulations of galaxy evolution aim to capture our current understanding as well as to make predictions for testing by future experiments. Simulations and observations are often compared in an indirect fashion: physical quantities are estimated from the data and compared to models. However, many applications can benefit from a more direct approach, where the observing process is also simulated and the models are seen fully from the observer's perspective. To facilitate this, we have developed the Millennium Run Observatory (MRObs), a theoretical virtual observatory which uses virtual telescopes to `observe' semi-analytic galaxy formation models based on the suite of Millennium Run dark matter simulations. The MRObs produces data that can be processed and analyzed using the standard software packages developed for real observations. At present, we produce images in forty filters from the rest-frame UV to IR for two stellar population synthesis models, three different models of IGM absorption, and two cosmologies (WMAP1/7). Galaxy distributions for a large number of mock lightcones can be `observed' using models of major ground- and space-based telescopes. The data include lightcone catalogues linked to structural properties of galaxies, pre-observation model images, mock telescope images, and Source Extractor products that can all be traced back to the higher level dark matter, semi-analytic galaxy, and lightcone catalogues available in the Millennium database. Here, we describe our methods and announce a first public release of simulated surveys (e.g., SDSS, CFHT-LS, GOODS, GOODS/ERS, CANDELS, and HUDF). The MRObs browser, an online tool, further facilitates exploration of the simulated data. We demonstrate the benefits of a direct approach through a number of example applications (galaxy number counts in CANDELS, clusters, morphologies, and dropout selections).

We present a novel, virtual X-ray observatory designed to obtain synthetic observations from hydro-numerical simulations, named PHOX. In particular, we provide a description of the code constituting the photon simulator and of the new approach implemented. We apply PHOX to simulated galaxy clusters in order to demonstrate its capabilities. In fact, X-ray observations of clusters of galaxies continue to provide us with an increasingly detailed picture of their structure and of the underlying physical phenomena governing the gaseous component, which dominates their baryonic content. Therefore, it is fundamental to find the most direct and faithful way to compare such observational data with hydrodynamical simulations of cluster-like objects, which can currently include various complex physical processes. Here, we present and analyse synthetic Suzaku observations of two cluster-size haloes obtained by processing with PHOX the hydrodynamical simulation of the large-scale, filament-like region in which they reside. Taking advantage of the simulated data, we test the results inferred from the X-ray analysis of the mock observations against the underlying, known solution. Remarkably, we are able to recover the theoretical temperature distribution of the two haloes by means of the multi-temperature fitting of the synthetic spectra. Moreover, the shapes of the reconstructed distributions allow us to trace the different thermal structure that distinguishes the dynamical state of the two haloes.

VODataService refers to an XML encoding standard for a specialized extension of the IVOA Resource Metadata that is useful for describing data collections and the services that access them. It is defined as an extension of the core resource metadata encoding standard known as VOResource [Plante et al. 2008] using XML Schema. The specialized resource types defined by the VODataService schema allow one to describe how the data underlying the resource cover the sky as well as cover frequency and time. This coverage description leverages heavily the Space-Time Coordinates (STC) standard schema [Rots 2007]. VODataService also enables detailed descriptions of tables that includes information useful to the discovery of tabular data. It is intended that the VODataService data types will be particularly useful in describing services that support standard IVOA service protocols.

This document describes an XML encoding standard for IVOA Resource Metadata, referred to as VOResource. This schema is primarily intended to support interoperable registries used for discovering resources; however, any application that needs to describe resources may use this schema. In this document, we define the types and elements that make up the schema as representations of metadata terms defined in the IVOA standard, Resource Metadata for the Virtual Observatory [Hanicsh et al. 2004]. We also describe the general model for the schema and explain how it may be extended to add new metadata terms and describe more specific types of resources.

We present the online MultiDark Database -- a Virtual Observatory-oriented, relational database for hosting various cosmological simulations. The data is accessible via an SQL (Structured Query Language) query interface, which also allows users to directly pose scientific questions, as shown in a number of examples in this paper. Further examples for the usage of the database are given in its extensive online documentation (www.multidark.org). The database is based on the same technology as the Millennium Database, a fact that will greatly facilitate the usage of both suites of cosmological simulations. The first release of the MultiDark Database hosts two 8.6 billion particle cosmological N-body simulations: the Bolshoi (250/h Mpc simulation box, 1/h kpc resolution) and MultiDark Run1 simulation (MDR1, or BigBolshoi, 1000/h Mpc simulation box, 7/h kpc resolution). The extraction methods for halos/subhalos from the raw simulation data, and how this data is structured in the database are explained in this paper. With the first data release, users get full access to halo/subhalo catalogs, various profiles of the halos at redshifts z=0-15, and raw dark matter data for one time-step of the Bolshoi and four time-steps of the MultiDark simulation. Later releases will also include galaxy mock catalogs and additional merging trees for both simulations as well as new large volume simulations with high resolution. This project is further proof of the viability to store and present complex data using relational database technology. We encourage other simulators to publish their results in a similar manner.

(abridged) We perform a reconstruction of the cosmological large scale flows in the nearby Universe using two complementary observational sets. The first, the SFI++ sample of Tully-Fisher (TF) measurements of galaxies, provides a direct probe of the flows. The second, the whole sky distribution of galaxies in the 2MASS redshift survey (2MRS), yields a prediction of the flows given the cosmological density parameter, $\Omega$, and a biasing relation between mass and galaxies. We aim at an unbiased comparison between the peculiar velocity fields extracted from the two data sets and its implication on the cosmological parameters and the biasing relation. We expand the fields in a set of orthonormal basis functions, each representing a plausible realization of a cosmological velocity field. Our analysis completely avoids the strong error covariance in the smoothed TF velocities by the use of orthonormal basis functions and employs elaborate realistic mock data sets to extensively calibrate the errors in 2MRS predicted velocities. We relate the 2MRS galaxy distribution to the mass density field by a linear bias factor, $b$, and include a luminosity dependent, $\propto L^\alpha$, galaxy weighting. We assess the agreement between the fields as a function of $\alpha$ and $\beta=f(\Omega)/b$, where $f$ is the growth factor of linear perturbations. The agreement is excellent with a reasonable $\chi^2$ per degree of freedom. For $\alpha=0$, we derive $0.28<\beta<0.37$ and $0.24<\beta<0.43$, respectively, at the 68.3% and 95.4% confidence levels (CLs). For $\beta=0.33$, we get $\alpha<0.25$ and $\alpha<0.5$, respectively, at the 68.3% and 95.4% CLs. We set a constraint on the fluctuation normalization, finding $\sigma_8 = 0.73 \pm 0.1$, in very good agreement with the latest WMAP results.

We apply updated semi-analytic galaxy formation models simultaneously to the stored halo/subhalo merger trees of the Millennium and Millennium-II simulations. These differ by a factor of 125 in mass resolution, allowing explicit testing of resolution effects on predicted galaxy properties. We have revised the treatments of the transition between the rapid infall and cooling flow regimes of gas accretion, of the sizes of bulges and of gaseous and stellar disks, of supernova feedback, of the transition between central and satellite status as galaxies fall into larger systems, and of gas and star stripping once they become satellites. Plausible values of efficiency and scaling parameters yield an excellent fit not only to the observed abundance of low-redshift galaxies over 5 orders of magnitude in stellar mass and 9 magnitudes in luminosity, but also to the observed abundance of Milky Way satellites. This suggests that reionisation effects may not be needed to solve the "missing satellite" problem except, perhaps, for the faintest objects. The same model matches the observed large-scale clustering of galaxies as a function of stellar mass and colour. The fit remains excellent down to ~30kpc for massive galaxies. For M* < 6 x 10^10Msun, however, the model overpredicts clustering at scales below 1 Mpc, suggesting that the sigma_8 adopted in the simulations (0.9) is too high. Galaxy distributions within rich clusters agree between the simulations and match those observed, but only if galaxies without dark matter subhalos (so-called orphans) are included. Our model predicts a larger passive fraction among low-mass galaxies than is observed, as well as an overabundance of ~10^10Msun galaxies beyond z~0.6, reflecting deficiencies in the way star-formation rates are modelled.

We present the largest Wiener reconstruction of the cosmic density field made to date. The reconstruction is based on the Sloan Digital Sky Survey data release 6 covering the northern Galactic cap. We use a novel supersampling algorithm to suppress aliasing effects and a Krylov-space inversion method to enable high performance with high resolution. These techniques are implemented in the ARGO computer code. We reconstruct the field over a 500 Mpc cube with Mpc grid-resolution while accounting both for the angular and radial selection functions of the SDSS, and the shot noise giving an effective resolution of the order of ~10 Mpc. In addition, we correct for the redshift distortions in the linear and nonlinear regimes in an approximate way. We show that the commonly used method of inverse weighting the galaxies by the corresponding selection function heads to excess noise in regions where the density of the observed galaxies is small. It is more accurate and conservative to adopt a Bayesian framework in which we model the galaxy selection/detection process to be Poisson-binomial. This results in heavier smoothing in regions of reduced sampling density. Our results show a complex cosmic web structure with huge void regions indicating that the recovered matter distribution is highly non-Gaussian. Filamentary structures are clearly visible on scales up to ~20 Mpc. We also calculate the statistical distribution of density after smoothing the reconstruction with Gaussian kernels of different radii r_S and find good agreement with a log-normal distribution for ~10 Mpc < r_S < ~30 Mpc.

We present the Millennium-II Simulation (MS-II), a very large N-body simulation of dark matter evolution in the concordance LCDM cosmology. The MS-II assumes the same cosmological parameters and uses the same particle number and output data structure as the original Millennium Simulation (MS), but was carried out in a periodic cube one-fifth the size (100 Mpc/h) with 5 times better spatial resolution (a Plummer equivalent softening of 1.0 kpc/h) and with 125 times better mass resolution (a particle mass of 6.9 \times 10^6 Msun/h). By comparing results at MS and MS-II resolution, we demonstrate excellent convergence in dark matter statistics such as the halo mass function, the subhalo abundance distribution, the mass dependence of halo formation times, the linear and nonlinear autocorrelations and power spectra, and halo assembly bias. Together, the two simulations provide precise results for such statistics over an unprecedented range of scales, from halos similar to those hosting Local Group dwarf spheroidal galaxies to halos corresponding to the richest galaxy clusters. The "Milky Way" halos of the Aquarius Project were selected from a lower resolution version of the MS-II and were then resimulated at much higher resolution. As a result, they are present in the MS-II along with thousands of other similar mass halos. A comparison of their assembly histories in the MS-II and in resimulations of 1000 times better resolution shows detailed agreement over a factor of 100 in mass growth. We publicly release halo catalogs and assembly trees for the MS-II in the same format within the same archive as those already released for the MS.

Until recently, it has been possible only for nearby galaxies to study the scaling relations between central black hole and host galaxy in detail. Because of the small number densities at low redshift, (luminous) AGN are underrepresented in such detailed studies. The advent of adaptive optics (AO) at large telescopes helps overcoming this hurdle, allowing to reach small linear scales over a wide range in redshift. Finding AO-suitable targets, i.e., AGN having a nearby reference star, and carrying out an initial multiwavelength classification is an excellent use case for the Virtual Observatory. We present our Virtual-Observatory approach to select an AO-suitable catalog of X-ray-emitting AGN at redshifts 0.1<z<1.

Motivated by recent observational studies of the environment of z~6 QSOs, we have used the Millennium Run (MR) simulations to construct a very large (~20 deg^2) mock redshift survey of star-forming galaxies at z~6. We use this simulated survey to study the relation between density enhancements in the distribution of i-dropouts and Lya-emitters, and their relation to the most massive halos and protocluster regions at z~6. Our simulation predicts significant variations in surface density across the sky with some voids and filaments extending over scales of 1 degree, much larger than probed by current surveys. Approximately one third of all z~6 halos hosting i-dropouts brighter than z=26.5 mag (~M* at z=6) become part of z=0 galaxy clusters. i-dropouts associated with protocluster regions are found in regions where the surface density is enhanced on scales ranging from a few to several tens of arcminutes on the sky. We analyze two structures of i-dropouts and Lya-emitters observed with the Subaru Telescope and show that these structures must be the seeds of massive clusters-in-formation. In striking contrast, six z~6 QSO fields observed with HST show no significant enhancements in their i-dropout number counts. With the present data, we cannot rule out the QSOs being hosted by the most massive halos. However, neither can we confirm this widely used assumption. We conclude by giving detailed recommendations for the interpretation and planning of observations by current and future ground- and space based instruments that will shed new light on questions related to the large-scale structure at z~6.

Despite the recent advances in graphics hardware capabilities, a brute force approach is incapable of interactively displaying terabytes of data. We have implemented a system that uses hierarchical level-of-detailing for the results of cosmological simulations, in order to display visually accurate results without loading in the full dataset (containing over 10 billion points). The guiding principle of the program is that the user should not be able to distinguish what they are seeing from a full rendering of the original data. Furthermore, by using a tree-based system for levels of detail, the size of the underlying data is limited only by the capacity of the IO system containing it.

In the last decade we have seen an enormous increase in the size and quality of spectroscopic galaxy surveys, both at low and high redshift. New statistical techniques to analyse large portions of galaxy spectra are now finding favour over traditional index based methods. Here we will review a new robust and iterative Principal Component Analysis (PCA) algorithm, which solves several common issues with classic PCA. Application to the 4000AA break region of galaxies in the VIMOS VLT Deep Survey (VVDS) and Sloan Digital Sky Survey (SDSS) gives new high signal-to-noise ratio spectral indices easily interpretable in terms of recent star formation history. In particular, we identify a sample of post-starburst galaxies at z~0.7 and z~0.07. We quantify for the first time the importance of post-starburst galaxies, consistent with being descendants of gas-rich major mergers, for building the red sequence. Finally, we present a comparison with new low and high redshift "mock spectroscopic surveys" derived from a Millennium Run semi-analytic model.

The formation of dark matter halos tends to occur anisotropically along the filaments of the Cosmic Web, which induces both ellipticity-ellipticity (EE) correlations between the shapes of halos, as well as ellipticity-direction (ED) cross-correlations between halo shapes and the directions to neighboring halos. We analyze the halo catalogue and the semi-analytic galaxy catalogue of the recent Millennium Run Simulation to measure the EE and ED correlations numerically at four different redshifts (z=0, 0.5, 1 and 2). For the EE correlations, we find that (i) the major-axis correlation is strongest while the intermediate-axis correlation is weakest; (ii) the signal is significant at distances out to 10 Mpc/h; (iii) the signal decreases as z decreases; (iv) and its behavior depends strongly on the halo mass scale, with larger masses showing stronger correlations at large distances. For the ED correlations, we find that (i) the correlations are much stronger than the EE correlations, and are significant even out to distances of 50 Mpc/h; (ii) the signal also decreases as z decreases; (iii) and it increases with halo mass at all distances. We also provide empirical fitting functions for the EE and ED correlations. The EE correlations are found to scale linearly with the linear density correlation function, xi(r). While the ED cross-correlation is found to scale as xi^{1/2}(r) at large distances beyond 10 Mpc/h. The best-fit values of the fitting parameters for the EE and the ED correlations are all determined through chi^{2}-statistics. Our results may be useful for quantifying the filamentary distribution of dark matter halos over a wide range of scales.

Star-forming galaxies constitute the majority of galaxies with stellar masses >10^10 M_Sun/h^2 at z~2 and dominate the star-formation rate density of the Universe at this early epoch. It is thus critical to understand their origins, evolution, and connection to the underlying dark matter distribution. To this end, we identify the dark matter halos (including subhalos) that are likely to contain star-forming galaxies at z~2 (z2SFGs) within a large dissipationless cosmological simulation and then use halo merger histories to follow the evolution of z2SFG descendants to z~1 and z~0. The evolved halos at these epochs are then confronted with an array of observational data in order to uncover the likely descendants of z2SFGs. Though the evolved halos have clustering strengths comparable to red galaxies at z~1 and z~0, we find that the bulk of z2SFGs do not evolve into red galaxies, at either epoch. This conclusion is based primarily on the fact that the space density of z2SFGs is much higher than that of lower redshift red galaxies, even when accounting for the merging of z2SFG descendants, which decreases the number density of z2SFG descendants by at most a factor of two by z~0. Of the ~50% of z2SFGs that survive to z~0, ~70% reside at the center of z~0 dark matter halos with M>10^12 M_Sun/h. Halo occupation modeling of z~0 galaxies suggests that such halos are occupied by galaxies with M_r<-20.5, implying that these z2SFGs evolve into ``typical'' ~ L* galaxies today, including our own Galaxy. The remaining ~30% become satellite galaxies by z~0, and comparison to halo occupation modeling suggests that they are rather faint, with M_r<-19.5. (ABRIDGED)

[Abridged] We investigate trends between the recent star formation history and black hole growth in galaxy bulges in the Sloan Digital Sky Survey (SDSS). The galaxies lie at 0.01<z<0.07 where the fibre aperture covers only the central 0.6-4.0kpc diameter of the galaxy. We find strong trends between black hole growth, as measured by dust-attenuation-corrected OIII luminosity, and the recent star formation history of the bulges. We conclude that our results support the popular hypothesis for black hole growth occurring through gas inflow into the central regions of galaxies, followed by a starburst and triggering of the AGN. However, while this is a significant pathway for the growth of black holes, it is not the dominant one in the present-day Universe. More unspectacular processes are apparently responsible for the majority of this growth. In order to arrive at these conclusions we have developed a set of new high signal-to-noise ratio (SNR) optical spectral indicators, designed to allow a detailed study of stellar populations which have undergone recent enhanced star formation. Working in the rest-frame wavelength range 3750-4150AA, ideally suited to many recent and ongoing spectroscopic surveys at low and high redshift, the first two indices are equivalent to the previously well studied 4000AA break strength and Hdelta equivalent width. The primary advantage of this new method is a greatly improved SNR for the latter index, allowing the present study to use spectra with SNR-per-pixel as low as 8.

The Millennium Run is the largest simulation of the formation of structure within the $\Lambda$CDM cosmogony so far carried out. It uses $10^{10}$ particles to follow the dark matter distribution in a cubic region 500$h^{-1}$Mpc on a side, and has a spatial resolution of 5 $h^{-1}$kpc. Application of simplified modelling techniques to the stored output of this calculation allows the formation and evolution of the $\sim 10^7$ galaxies more luminous than the Small Magellanic Cloud to be simulated for a variety of assumptions about the detailed physics involved. As part of the activities of the German Astrophysical Virtual Observatory we have used a relational database to store the detailed assembly histories both of all the haloes and subhaloes resolved by the simulation, and of all the galaxies that form within these structures for two independent models of the galaxy formation physics. We have created web applications that allow users to query these databases remotely using the standard Structured Query Language (SQL). This allows easy access to all properties of the galaxies and halos, as well as to the spatial and temporal relations between them and their environment. Information is output in table format compatible with standard Virtual Observatory tools and protocols. With this announcement we are making these structures fully accessible to all users. Interested scientists can learn SQL, gain familiarity with the database design and test queries on a small, openly accessible version of the Millennium Run (with volume 1/512 that of the full simulation). They can then request accounts to run similar queries on the databases for the full simulations.

We present the first systematic study of (non-radio-selected) radio-loud narrow-line Seyfert 1 (NLS1) galaxies. Cross-correlation of the `Catalogue of Quasars and Active Nuclei' with several radio and optical catalogues led to the identification of 11 radio-loud NLS1 candidates including 4 previously known ones. Most of the radio-loud NLS1s are compact, steep spectrum sources accreting close to, or above, the Eddington limit. The radio-loud NLS1s of our sample are remarkable in that they occupy a previously rarely populated regime in NLS1 multi-wavelength parameter space. While their [OIII]/H_beta and FeII/H_beta intensity ratios almost cover the whole range observed in NLS1 galaxies, their radio properties extend the range of radio-loud objects to those with small widths of the broad Balmer lines. Among the radio-detected NLS1 galaxies, the radio index R distributes quite smoothly up to the critical value of R ~ 10 and covers about 4 orders of magnitude in total. Statistics show that ~7% of the NLS1 galaxies are formally radio-loud while only 2.5% exceed a radio index R > 100. Several mechanisms are considered as explanations for the radio loudness of the NLS1 galaxies and for the lower frequency of radio-louds among NLS1s than quasars. While properties of most sources (with 2-3 exceptions) generally do not favor relativistic beaming, the combination of accretion mode and spin may explain the observations. (abbreviated)

We simulate the formation and evolution of the local galaxy population starting from initial conditions with a smoothed linear density field which matches that derived from the IRAS 1.2 Jy galaxy survey. Our simulations track the formation and evolution of all dark matter haloes more massive than 10e+11 solar masses out to a distance of 8000 km/s from the Milky Way. We implement prescriptions similar to those of Kauffmann et al. (1999) to follow the assembly and evolution of the galaxies within these haloes. We focus on two variants of the CDM cosmology: an LCDM and a tCDM model. Galaxy formation in each is adjusted to reproduce the I-band Tully-Fisher relation of Giovanelli et al. (1997). We compare the present-day luminosity functions, colours, morphology and spatial distribution of our simulated galaxies with those of the real local population, in particular with the Updated Zwicky Catalog, with the IRAS PSCz redshift survey, and with individual local clusters such as Coma, Virgo and Perseus. We also use the simulations to study the clustering bias between the dark matter and galaxies of differing type. Although some significant discrepancies remain, our simulations recover the observed intrinsic properties and the observed spatial distribution of local galaxies reasonably well. They can thus be used to calibrate methods which use the observed local galaxy population to estimate the cosmic density parameter or to draw conclusions about the mechanisms of galaxy formation. To facilitate such work, we publically release our z=0 galaxy catalogues, together with the underlying mass distribution.

We study the biasing relation between dark-matter halos or galaxies and the underlying mass distribution, using cosmological $N$-body simulations in which galaxies are modelled via semi-analytic recipes. The nonlinear, stochastic biasing is quantified in terms of the mean biasing function and the scatter about it as a function of time, scale and object properties. The biasing of galaxies and halos shows a general similarity and a characteristic shape, with no galaxies in deep voids and a steep slope in moderately underdense regions. At $\sim 8\hmpc$, the nonlinearity is typically $\lsim 10$ percent and the stochasticity is a few tens of percent, corresponding to $\sim 30$ percent variations in the cosmological parameter $\beta=\Omega^{0.6}/b$. Biasing depends weakly on halo mass, galaxy luminosity, and scale. The time evolution is rapid, with the mean biasing larger by a factor of a few at $z\sim 3$ compared to $z=0$, and with a minimum for the nonlinearity and stochasticity at an intermediate redshift. Biasing today is a weak function of the cosmological model, reflecting the weak dependence on the power-spectrum shape, but the time evolution is more cosmology-dependent, relecting the effect of the growth rate. We provide predictions for the relative biasing of galaxies of different type and color, to be compared with upcoming large redshift surveys. Analytic models in which the number of objects is conserved underestimate the evolution of biasing, while models that explicitly account for merging provide a good description of the biasing of halos and its evolution, suggesting that merging is a crucial element in the evolution of biasing.

In hierarchical models of gravitational clustering, virialized haloes are biased tracers of the matter distribution. As discussed by Mo & White (1996), this bias is nonlinear and stochastic. They developed a model which allows one to write down analytic expressions for the mean of the bias relation, in the initial Lagrangian, and the evolved, Eulerian spaces. We provide analytic expressions for the higher order moments as well. We also show how the Mo-White approach can be extended to compute the evolution, not just of the haloes, but of the dark matter distribution itself. The model predictions for the mean and scatter in the Eulerian bias relation, as well as for the Eulerian halo-mass and halo-halo correlation functions, are in reasonable agreement with numerical simulations of hierarchical gravitational clustering for haloes of a wide range of masses, whereas the predictions for the corresponding Lagrangian quantities are accurate only for massive haloes.

We evaluate the accuracy of semi-analytic merger-trees by comparing them with the merging histories of dark-matter halos in N-body simulations, focusing on the joint distribution of the number of progenitors and their masses. We first confirm that the halo mass function as predicted directly by the Press-Schechter (PS) model deviates from the simulations by up to 50% depending on the mass scale and redshift, while the means of the projected distributions of progenitor number and mass for a halo of a given mass are more accurately predicted by the Extended PS model. We then use the full merger trees to study the joint distribution as a function of redshift and parent-halo mass. We find that while the deviation of the mean quantities due to the inaccuracy of the Extended PS model partly propagates into the higher moments of the distribution, the merger-tree procedure does not introduce a significant additional source of error. In particular, certain properties of the merging history such as the mass ratio of the progenitors and the total accretion rate are reproduced quite accurately for galaxy sized halos ($\sim 10^{12}\msun$), and less so for larger masses. We conclude that although there could be $\sim 50%$ deviations in the absolute numbers and masses of progenitors and in the higher order moment of these distributions, the relative properties of progenitors for a given halo are reproduced fairly well by the merger trees. They can thus provide a useful framework for modelling galaxy formation once the above-mentioned limitations are taken into account.

We describe a simple efficient algorithm that allows one to construct Monte-Carlo realizations of merger histories of dark matter halos. The algorithm is motivated by the excursion set model (Bond et al. 1991) for the conditional and unconditional halo mass functions. The forest of trees constructed using this algorithm depends on the underlying power spectrum. For Poisson or white-noise initial power-spectra, the forest has exactly the same properties as the ensemble of trees described by Sheth (1996) and Sheth & Pitman (1997). In this case, many ensemble averaged higher order statistics of the tree distribution can be computed analytically. For Gaussian initial conditions with more general power-spectra, mean properties of the ensemble closely resemble the mean properties expected from the excursion set approach. Various statistical quantities associated with the trees constructed using our algorithm are in good agreement with what is measured in numerical simulations of hierarchical gravitational clustering.

We use large N-body simulations of dissipationless gravitational clustering in cold dark matter (CDM) cosmologies to study whether the properties of dark matter halos are affected by their environment. We look for correlations between the masses, formation redshifts, concentrations, shapes and spins of halos and the overdensity of their local environment. We also look for correlations of these quantities with the local tidal field. Our conclusion is extremely simple. Only the mass distribution varies as a function of environment. This variation is well described by a simple analytic formula based on the conditional Press-Schechter theory. We find no significant dependence of any other halo property on environment. Our results do not depend on our choice of cosmology. According to current hierarchical models, the structure and evolutionary history of a galaxy is fully determined by the structure and evolutionary history of the dark halo in which it is embedded. If these models are correct, clustering variations between galaxies of differing morphological types, luminosities, colours and surface brightnesses, must all arise because the halo mass function is skewed towards high mass objects in overdense regions of the Universe and towards low mass objects in underdense regions.