The Hubble Space Telescope (HST) has obtained multi-epoch observations providing the opportunity for a comprehensive variability search aiming to uncover new variables. We have therefore undertaken the task of creating a catalog of variable sources based on version 3 of the Hubble Source Catalog (HSC), which relies on publicly available images obtained with the WFPC2, ACS, and WFC3 instruments onboard the HST. We adopted magnitude-dependent thresholding in median absolute deviation (a robust measure of light curve scatter) combined with sophisticated preprocessing techniques and visual quality control to identify and validate variable sources observed by Hubble with the same instrument and filter combination five or more times. The Hubble Catalog of Variables (HCV) includes 84,428 candidate variable sources (out of 3.7 million HSC sources that were searched for variability) with $V \leq 27$ mag; for 11,115 of them the variability is detected in more than one filter. The data points in the light curves of the variables in the HCV catalog range from five to 120 points (typically having less than ten points); the time baseline ranges from under a day to over 15 years; while $\sim$8% of all variables have amplitudes in excess of 1 mag. Visual inspection performed on a subset of the candidate variables suggests that at least 80% of the candidate variables that passed our automated quality control are true variable sources rather than spurious detections resulting from blending, residual cosmic rays, and calibration errors. The HCV is the first, homogeneous catalog of variable sources created from archival HST data and currently is the deepest catalog of variables available. The catalog includes variable stars in our Galaxy and nearby galaxies, as well as transients and variable active galactic nuclei. (abbreviated)

Refraction by the atmosphere causes the positions of sources to depend on the airmass through which an observation was taken. This shift is dependent on the underlying spectral energy of the source and the filter or bandpass through which it is observed. Wavelength-dependent refraction within a single passband is often referred to as differential chromatic refraction (DCR). With a new generation of astronomical surveys undertaking repeated observations of the same part of the sky over a range of different airmasses and parallactic angles, DCR should be a detectable and measurable astrometric signal. In this paper we introduce a novel procedure that takes this astrometric signal and uses it to infer the underlying spectral energy distribution of a source; we solve for multiple latent images at specific wavelengths via a generalized deconvolution procedure built on robust statistics. We demonstrate the utility of such an approach for estimating a partially deconvolved image, at higher spectral resolution than the input images, for surveys such as the Large Synoptic Survey Telescope (LSST).

Over the past century, major advances in astronomy and astrophysics have been largely driven by improvements in instrumentation and data collection. With the amassing of high quality data from new telescopes, and especially with the advent of deep and large astronomical surveys, it is becoming clear that future advances will also rely heavily on how those data are analyzed and interpreted. New methodologies derived from advances in statistics, computer science, and machine learning are beginning to be employed in sophisticated investigations that are not only bringing forth new discoveries, but are placing them on a solid footing. Progress in wide-field sky surveys, interferometric imaging, precision cosmology, exoplanet detection and characterization, and many subfields of stellar, Galactic and extragalactic astronomy, has resulted in complex data analysis challenges that must be solved to perform scientific inference. Research in astrostatistics and astroinformatics will be necessary to develop the state-of-the-art methodology needed in astronomy. Overcoming these challenges requires dedicated, interdisciplinary research. We recommend: (1) increasing funding for interdisciplinary projects in astrostatistics and astroinformatics; (2) dedicating space and time at conferences for interdisciplinary research and promotion; (3) developing sustainable funding for long-term astrostatisics appointments; and (4) funding infrastructure development for data archives and archive support, state-of-the-art algorithms, and efficient computing.

Future radio surveys will generate catalogues of tens of millions of radio sources, for which redshift estimates will be essential to achieve many of the science goals. However, spectroscopic data will be available for only a small fraction of these sources, and in most cases even the optical and infrared photometry will be of limited quality. Furthermore, radio sources tend to be at higher redshift than most optical sources and so a significant fraction of radio sources hosts differ from those for which most photometric redshift templates are designed. We therefore need to develop new techniques for estimating the redshifts of radio sources. As a starting point in this process, we evaluate a number of machine-learning techniques for estimating redshift, together with a conventional template-fitting technique. We pay special attention to how the performance is affected by the incompleteness of the training sample and by sparseness of the parameter space or by limited availability of ancillary multi-wavelength data. As expected, we find that the quality of the photometric-redshift degrades as the quality of the photometry decreases, but that even with the limited quality of photometry available for all sky-surveys, useful redshift information is available for the majority of sources, particularly at low redshift. We find that a template-fitting technique performs best with high-quality and almost complete multi-band photometry, especially if radio sources that are also X-ray emitting are treated separately. When we reduced the quality of photometry to match that available for the EMU all-sky radio survey, the quality of the template-fitting degraded and became comparable to some of the machine learning methods. Machine learning techniques currently perform better at low redshift than at high redshift, because of incompleteness of the currently available training data at high redshifts.

Matching astronomical catalogs in crowded regions of the sky is challenging both statistically and computationally due to the many possible alternative associations. Budav\'ari and Basu (2016) modeled the two-catalog situation as an Assignment Problem and used the famous Hungarian algorithm to solve it. Here we treat cross-identification of multiple catalogs by introducing a different approach based on integer linear programming. We first test this new method on problems with two catalogs and compare with the previous results. We then test the efficacy of the new approach on problems with three catalogs. The performance and scalability of the new approach is discussed in the context of large surveys.

Cosmological $N$-body simulations play a vital role in studying models for the evolution of the Universe. To compare to observations and make a scientific inference, statistic analysis on large simulation datasets, e.g., finding halos, obtaining multi-point correlation functions, is crucial. However, traditional in-memory methods for these tasks do not scale to the datasets that are forbiddingly large in modern simulations. Our prior paper proposes memory-efficient streaming algorithms that can find the largest halos in a simulation with up to $10^9$ particles on a small server or desktop. However, this approach fails when directly scaling to larger datasets. This paper presents a robust streaming tool that leverages state-of-the-art techniques on GPU boosting, sampling, and parallel I/O, to significantly improve performance and scalability. Our rigorous analysis of the sketch parameters improves the previous results from finding the centers of the $10^3$ largest halos to $\sim 10^4-10^5$, and reveals the trade-offs between memory, running time and number of halos. Our experiments show that our tool can scale to datasets with up to $\sim 10^{12}$ particles while using less than an hour of running time on a single GPU Nvidia GTX 1080.

The Hubble Source Catalog (HSC) combines lists of sources detected on images obtained with the WFPC2, ACS and WFC3 instruments aboard the Hubble Space Telescope (HST) available in the Hubble Legacy Archive. The catalog contains time-domain information with about two million of its sources detected with the same instrument and filter in at least five HST visits. The Hubble Catalog of Variables (HCV) project aims to identify HSC sources showing significant brightness variations. A magnitude-dependent threshold in the median absolute deviation of photometric measurements (an outlier-resistant measure of lightcurve scatter) is adopted as the variability-detection statistic. It is supplemented with a cut in $\chi_{\rm red}^2$ that removes sources with large photometric errors. A pre-processing procedure involving bad image identification, outlier rejection and computation of local magnitude zero-point corrections is applied to HSC lightcurves before computing the variability detection statistic. About 52000 HSC sources are identified as candidate variables, among which 7800 show variability in more than one filter. Visual inspection suggests that $\sim 70\%$ of the candidates detected in multiple filters are true variables while the remaining $\sim 30\%$ are sources with aperture photometry corrupted by blending, imaging artifacts or image processing anomalies. The candidate variables have AB magnitudes in the range 15-27$^{m}$ with the median 22$^{m}$. Among them are the stars in our own and nearby galaxies as well as active galactic nuclei.

The Hubble Catalog of Variables (HCV) project aims to identify the variable sources in the Hubble Source Catalog (HSC), which includes about 92 million objects with over 300 million measurements detected by the WFPC2, ACS and WFC3 cameras on board of the Hubble Space Telescope (HST), by using an automated pipeline containing a set of detection and validation algorithms. All the HSC sources with more than a predefined number of measurements in a single filter/instrument combination are pre-processed to correct systematic effect and to remove the bad measurements. The corrected data are used to compute a number of variability indexes to determine the variability status of each source. The final variable source catalog will contain variables stars, active galactic nuclei (AGNs), supernovae (SNs) or even new types of variables, reaching an unprecedented depth (V$\leq$27 mag). At the end of the project, the first release of the HCV will be available at the Mikulski Archive for Space Telescopes (MAST) and the ESA Hubble Science Archives. The HCV pipeline will be deployed at the Space Telescope Science Institute (STScI) so that an updated HCV may be generated following future releases of HSC.

We present a blind multiframe image-deconvolution method based on robust statistics. The usual shortcomings of iterative optimization of the likelihood function are alleviated by minimizing the M-scale of the residuals, which achieves more uniform convergence across the image. We focus on the deconvolution of astronomical images, which are among the most challenging due to their huge dynamic ranges and the frequent presence of large noise-dominated regions in the images. We show that high-quality image reconstruction is possible even in super-resolution and without the use of traditional regularization terms. Using a robust \r{ho}-function is straightforward to implement in a streaming setting and, hence our method is applicable to the large volumes of astronomy images. The power of our method is demonstrated on observations from the Sloan Digital Sky Survey (Stripe 82) and we briefly discuss the feasibility of a pipeline based on Graphical Processing Units for the next generation of telescope surveys.

We release the AllWISE counterparts and Gaia matches to 106,573 and 17,665 X-ray sources detected in the ROSAT 2RXS and XMMSL2 surveys with |b|>15. These are the brightest X-ray sources in the sky, but their position uncertainties and the sparse multi-wavelength coverage until now rendered the identification of their counterparts a demanding task with uncertain results. New all-sky multi-wavelength surveys of sufficient depth, like AllWISE and Gaia, and a new Bayesian statistics based algorithm, NWAY, allow us, for the first time, to provide reliable counterpart associations. NWAY extends previous distance and sky density based association methods and, using one or more priors (e.g., colors, magnitudes), weights the probability that sources from two or more catalogues are simultaneously associated on the basis of their observable characteristics. Here, counterparts have been determined using a WISE color-magnitude prior. A reference sample of 4524 XMM/Chandra and Swift X-ray sources demonstrates a reliability of ~ 94.7% (2RXS) and 97.4% (XMMSL2). Combining our results with Chandra-COSMOS data, we propose a new separation between stars and AGN in the X-ray/WISE flux-magnitude plane, valid over six orders of magnitude. We also release the NWAY code and its user manual. NWAY was extensively tested with XMM-COSMOS data. Using two different sets of priors, we find an agreement of 96% and 99% with published Likelihood Ratio methods. Our results were achieved faster and without any follow-up visual inspection. With the advent of deep and wide area surveys in X-rays (e.g. SRG/eROSITA, Athena/WFI) and radio (ASKAP/EMU, LOFAR, APERTIF, etc.) NWAY will provide a powerful and reliable counterpart identification tool.

We aim to construct an exceptionally deep (V ~< 27) catalog of variable objects in selected Galactic and extragalactic fields visited multiple times by the Hubble Space Telescope (HST). While HST observations of some of these fields were searched for specific types of variables before (most notably, the extragalactic Cepheids), we attempt a systematic study of the population of variable objects of all types at the magnitude range not easily accessible with ground-based telescopes. The variability timescales that can be probed range from hours to years depending on how often a particular field has been visited. For source extraction and cross-matching of sources between visits we rely on the Hubble Source Catalog which includes 10^7 objects detected with WFPC2, ACS, and WFC3 HST instruments. The lightcurves extracted from the HSC are corrected for systematic effects by applying local zero-point corrections and are screened for bad measurements. For each lightcurve we compute variability indices sensitive to a broad range of variability types. The indices characterize the overall lightcurve scatter and smoothness. Candidate variables are selected as having variability index values significantly higher than expected for objects of similar brightness in the given set of observations. The Hubble Catalog of Variables will be released in 2018.

Probabilistic cross-identification has been successfully applied to a number of problems in astronomy from matching simple point sources to associating stars with unknown proper motions and even radio observations with realistic morphology. Here we study the Bayes factor for clustered objects and focus in particular on galaxies to assess the effect of typical angular correlations. Numerical calculations provide the modified relationship, which (as expected) suppresses the evidence for the associations at the shortest separations where the 2-point auto-correlation function is large. Ultimately this means that the matching probability drops at somewhat shorter scales than in previous models.

Observational astronomy in the time-domain era faces several new challenges. One of them is the efficient use of observations obtained at multiple epochs. The work presented here addresses faint object detection with multi-epoch data, and describes an incremental strategy for separating real objects from artifacts in ongoing surveys, in situations where the single-epoch data are summaries of the full image data, such as single-epoch catalogs of flux and direction estimates for candidate sources. The basic idea is to produce low-threshold single-epoch catalogs, and use a probabilistic approach to accumulate catalog information across epochs; this is in contrast to more conventional strategies based on co-added or stacked image data across all epochs. We adopt a Bayesian approach, addressing object detection by calculating the marginal likelihoods for hypotheses asserting there is no object, or one object, in a small image patch containing at most one cataloged source at each epoch. The object-present hypothesis interprets the sources in a patch at different epochs as arising from a genuine object; the no-object (noise) hypothesis interprets candidate sources as spurious, arising from noise peaks. We study the detection probability for constant-flux objects in a simplified Gaussian noise setting, comparing results based on single exposures and stacked exposures to results based on a series of single-epoch catalog summaries. Computing the detection probability based on catalog data amounts to generalized cross-matching: it is the product of a factor accounting for matching of the estimated fluxes of candidate sources, and a factor accounting for matching of their estimated directions. We find that probabilistic fusion of multi-epoch catalog information can detect sources with only modest sacrifice in sensitivity and selectivity compared to stacking.

Modern astronomy increasingly relies upon systematic surveys, whose dedicated telescopes continuously observe the sky across varied wavelength ranges of the electromagnetic spectrum; some surveys also observe non-electromagnetic "messengers," such as high-energy particles or gravitational waves. Stars and galaxies look different through the eyes of different instruments, and their independent measurements have to be carefully combined to provide a complete, sound picture of the multicolor and eventful universe. The association of an object's independent detections is, however, a difficult problem scientifically, computationally, and statistically, raising varied challenges across diverse astronomical applications. The fundamental problem is finding records in survey databases with directions that match to within the direction uncertainties. Such astronomical versions of the record linkage problem are known by various terms in astronomy: cross-matching, cross-identification, and directional, positional, or spatio-temporal coincidence assessment. Astronomers have developed several statistical approaches for such problems, largely independently of related developments in other disciplines. Here we review emerging approaches that compute (Bayesian) probabilities for the hypotheses of interest: possible associations, or demographic properties of a cosmic population that depend on identifying associations. Many cross-identification tasks can be formulated within a hierarchical Bayesian partition model framework, with components that explicitly account for astrophysical effects (e.g., source brightness vs. wavelength, source motion, or source extent), selection effects, and measurement error. We survey recent developments, and highlight important open areas for future research.

One of the outstanding challenges of cross-identification is multiplicity: detections in crowded regions of the sky are often linked to more than one candidate associations of similar likelihoods. We map the resulting maximum likelihood partitioning to the fundamental assignment problem of discrete mathematics and efficiently solve the two-way catalog-level matching in the realm of combinatorial optimization using the so-called Hungarian algorithm. We introduce the method, demonstrate its performance in a mock universe where the true associations are known, and discuss the applicability of the new procedure to large surveys.

The Hubble Source Catalog is designed to help optimize science from the Hubble Space Telescope by combining the tens of thousands of visit-based source lists in the Hubble Legacy Archive into a single master catalog. Version 1 of the Hubble Source Catalog includes WFPC2, ACS/WFC, WFC3/UVIS, and WFC3/IR photometric data generated using SExtractor software to produce the individual source lists. The catalog includes roughly 80 million detections of 30 million objects involving 112 different detector/filter combinations, and about 160 thousand HST exposures. Source lists from Data Release 8 of the Hubble Legacy Archive are matched using an algorithm developed by Budavari & Lubow (2012). The mean photometric accuracy for the catalog as a whole is better than 0.10 mag, with relative accuracy as good as 0.02 mag in certain circumstances (e.g., bright isolated stars). The relative astrometric residuals are typically within 10 mas, with a value for the mode (i.e., most common value) of 2.3 mas. The absolute astrometric accuracy is better than $\sim$0.1 arcsec for most sources, but can be much larger for a fraction of fields that could not be matched to the PanSTARRS, SDSS, or 2MASS reference systems. In this paper we describe the database design with emphasis on those aspects that enable the users to fully exploit the catalog while avoiding common misunderstandings and potential pitfalls. We provide usage examples to illustrate some of the science capabilities and data quality characteristics, and briefly discuss plans for future improvements to the Hubble Source Catalog.

We present a data-driven method to infer the redshift distribution of an arbitrary dataset based on spatial cross-correlation with a reference population and we apply it to various datasets across the electromagnetic spectrum to show its potential and limitations. Our approach advocates the use of clustering measurements on all available scales, in contrast to previous works focusing only on linear scales. We also show how its accuracy can be enhanced by optimally sampling a dataset within its photometric space rather than applying the estimator globally. We show that the ultimate goal of this technique is to characterize the mapping between the space of photometric observables and redshift space as this characterization then allows us to infer the clustering-redshift p.d.f. of a single galaxy. We apply this technique to estimate the redshift distributions of luminous red galaxies and emission line galaxies from the SDSS, infrared sources from WISE and radio sources from FIRST. We show that consistent redshift distributions are found using both quasars and absorber systems as reference populations. This technique brings valuable information on the third dimension of astronomical datasets. It is widely applicable to a large range of extra-galactic surveys.

Understanding the diversity in spectra is the key to determining the physical parameters of galaxies. The optical spectra of galaxies are highly convoluted with continuum and lines which are potentially sensitive to different physical parameters. Defining the wavelength regions of interest is therefore an important question. In this work, we identify informative wavelength regions in a single-burst stellar populations model by using the CUR Matrix Decomposition. Simulating the Lick/IDS spectrograph configuration, we recover the widely used Dn(4000), Hbeta, and HdeltaA to be most informative. Simulating the SDSS spectrograph configuration with a wavelength range 3450-8350 Angstrom and a model-limited spectral resolution of 3 Angstrom, the most informative regions are: first region-the 4000 Angstrom break and the Hdelta line; second region-the Fe-like indices; third region-the Hbeta line; fourth region-the G band and the Hgamma line. A Principal Component Analysis on the first region shows that the first eigenspectrum tells primarily the stellar age, the second eigenspectrum is related to the age-metallicity degeneracy, and the third eigenspectrum shows an anti-correlation between the strengths of the Balmer and the Ca K and H absorptions. The regions can be used to determine the stellar age and metallicity in early-type galaxies which have solar abundance ratios, no dust, and a single-burst star formation history. The region identification method can be applied to any set of spectra of the user's interest, so that we eliminate the need for a common, fixed-resolution index system. We discuss future directions in extending the current analysis to late-type galaxies.

We present observations of a luminous galaxy at redshift z=6.573 --- the end of the reioinization epoch --- which has been spectroscopically confirmed twice. The first spectroscopic confirmation comes from slitless HST ACS grism spectra from the PEARS survey (Probing Evolution And Reionization Spectroscopically), which show a dramatic continuum break in the spectrum at restframe 1216 A wavelength. The second confirmation is done with Keck + DEIMOS. The continuum is not clearly detected with ground-based spectra, but high wavelength resolution enables the Lyman alpha emission line profile to be determined. We compare the line profile to composite line profiles at redshift z=4.5. The Lyman alpha line profile shows no signature of a damping wing attenuation, confirming that the intergalactic gas is ionized at redshift z=6.57. Spectra of Lyman breaks at yet higher redshifts will be possible using comparably deep observations with IR-sensitive grisms, even at redshifts where Lyman alpha is too attenuated by the neutral IGM to be detectable using traditional spectroscopy from the ground.

Object cross-identification in multiple observations is often complicated by the uncertainties in their astrometric calibration. Due to the lack of standard reference objects, an image with a small field of view can have significantly larger errors in its absolute positioning than the relative precision of the detected sources within. We present a new general solution for the relative astrometry that quickly refines the World Coordinate System of overlapping fields. The efficiency is obtained through the use of infinitesimal 3-D rotations on the celestial sphere, which do not involve trigonometric functions. They also enable an analytic solution to an important step in making the astrometric corrections. In cases with many overlapping images, the correct identification of detections that match together across different images is difficult to determine. We describe a new greedy Bayesian approach for selecting the best object matches across a large number of overlapping images. The methods are developed and demonstrated on the Hubble Legacy Archive, one of the most challenging data sets today. We describe a novel catalog compiled from many Hubble Space Telescope observations, where the detections are combined into a searchable collection of matches that link the individual detections. The matches provide descriptions of astronomical objects involving multiple wavelengths and epochs. High relative positional accuracy of objects is achieved across the Hubble images, often sub-pixel precision in the order of just a few milli-arcseconds. The result is a reliable set of high-quality associations that are publicly available online.

In the lead-up to the Square Kilometre Array (SKA) project, several next-generation radio telescopes and upgrades are already being built around the world. These include APERTIF (The Netherlands), ASKAP (Australia), eMERLIN (UK), VLA (USA), e-EVN (based in Europe), LOFAR (The Netherlands), Meerkat (South Africa), and the Murchison Widefield Array (MWA). Each of these new instruments has different strengths, and coordination of surveys between them can help maximise the science from each of them. A radio continuum survey is being planned on each of them with the primary science objective of understanding the formation and evolution of galaxies over cosmic time, and the cosmological parameters and large-scale structures which drive it. In pursuit of this objective, the different teams are developing a variety of new techniques, and refining existing ones. Here we describe these projects, their science goals, and the technical challenges which are being addressed to maximise the science return.

We present a data model describing the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata. This data model may be used to represent spectra, time series data, segments of SED (Spectral Energy Distributions) and other spectral or temporal associations.

The Simple Spectral Access (SSA) Protocol (SSAP) defines a uniform interface to remotely discover and access one dimensional spectra. SSA is a member of an integrated family of data access interfaces altogether comprising the Data Access Layer (DAL) of the IVOA. SSA is based on a more general data model capable of describing most tabular spectrophotometric data, including time series and spectral energy distributions (SEDs) as well as 1-D spectra; however the scope of the SSA interface as specified in this document is limited to simple 1-D spectra, including simple aggregations of 1-D spectra. The form of the SSA interface is simple: clients first query the global resource registry to find services of interest and then issue a data discovery query to selected services to determine what relevant data is available from each service; the candidate datasets available are described uniformly in a VOTable format document which is returned in response to the query. Finally, the client may retrieve selected datasets for analysis. Spectrum datasets returned by an SSA spectrum service may be either precomputed, archival datasets, or they may be virtual data which is computed on the fly to respond to a client request. Spectrum datasets may conform to a standard data model defined by SSA, or may be native spectra with custom project-defined content. Spectra may be returned in any of a number of standard data formats. Spectral data is generally stored externally to the VO in a format specific to each spectral data collection; currently there is no standard way to represent astronomical spectra, and virtually every project does it differently. Hence spectra may be actively mediated to the standard SSA-defined data model at access time by the service, so that client analysis programs do not have to be familiar with the idiosyncratic details of each data collection to be accessed.

We discuss a novel approach to identifying cosmic events in separate and independent observations. In our focus are the true events, such as supernova explosions, that happen once, hence, whose measurements are not repeatable. Their classification and analysis have to make the best use of all the available data. Bayesian hypothesis testing is used to associate streams of events in space and time. Probabilities are assigned to the matches by studying their rates of occurrence. A case study of Type Ia supernovae illustrates how to use lightcurves in the cross-identification process. Constraints from realistic lightcurves happen to be well-approximated by Gaussians in time, which makes the matching process very efficient. Model-dependent associations are computationally more demanding but can further boost our confidence.

The inclination of galaxies induces both reddening and extinction to their observed spectral energy distribution, which in turn impact the derived properties of the galaxies. Here we report a significant dependence of the error in photometric redshift (photo-z) on the inclination of disk galaxies from the Sloan Digital Sky Survey. The bias in the photo-z based on the template-fitting approach increases from -0.015 in face-on to 0.021 in edge-on galaxies. A Principal Component Analysis on the full sample of photometry reveals the inclination of the galaxies to be represented by the 2nd mode. The corresponding eigenspectrum resembles an extinction curve. The isolation of the inclination effect in a low-order mode demonstrates the significant reddening induced on the observed colors, leading to the over-estimated photo-z in galaxies of high inclinations. We present approaches to correct the photo-z and the other properties of the disk galaxies against the inclination effect.

Although relative errors can readily be calculated, the absolute astrometric accuracy of the source positions in the Chandra Source Catalog (CSC), Version 1.0, is a priori unknown. However, the cross-match with stellar objects from the Sloan Digital Sky Survey (SDSS) offers the opportunity to compare the apparent separations of the cross-matched pairs with the formally calculated errors. The analysis of these data allowed us to derive a value of 0.16" for the residual absolute astrometric error in CSC positions. This error will be added to the published position errors in the CSC from now on, starting with CSC, Version 1.1.

Here we introduce PHAT, the PHoto-z Accuracy Testing programme, an international initiative to test and compare different methods of photo-z estimation. Two different test environments are set up, one (PHAT0) based on simulations to test the basic functionality of the different photo-z codes, and another one (PHAT1) based on data from the GOODS survey. The accuracy of the different methods is expressed and ranked by the global photo-z bias, scatter, and outlier rates. Most methods agree well on PHAT0 but produce photo-z scatters that can differ by up to a factor of two even in this idealised case. A larger spread in accuracy is found for PHAT1. Few methods benefit from the addition of mid-IR photometry. Remaining biases and systematic effects can be explained by shortcomings in the different template sets and the use of priors on the one hand and an insufficient training set on the other hand. Scatters of 4-8% in Delta_z/(1+z) were obtained, consistent with other studies. However, somewhat larger outlier rates (>7.5% with Delta_z/(1+z)>0.15; >4.5% after cleaning) are found for all codes. There is a general trend that empirical codes produce smaller biases than template-based codes. The systematic, quantitative comparison of different photo-z codes presented here is a snapshot of the current state-of-the-art of photo-z estimation and sets a standard for the assessment of photo-z accuracy in the future. The rather large outlier rates reported here for PHAT1 on real data should be investigated further since they are most probably also present (and possibly hidden) in many other studies. The test data sets are publicly available and can be used to compare new methods to established ones and help in guiding future photo-z method development. (abridged)

Fitting the spectral energy distributions (SEDs) of galaxies is an almost universally used technique that has matured significantly in the last decade. Model predictions and fitting procedures have improved significantly over this time, attempting to keep up with the vastly increased volume and quality of available data. We review here the field of SED fitting, describing the modelling of ultraviolet to infrared galaxy SEDs, the creation of multiwavelength data sets, and the methods used to fit model SEDs to observed galaxy data sets. We touch upon the achievements and challenges in the major ingredients of SED fitting, with a special emphasis on describing the interplay between the quality of the available data, the quality of the available models, and the best fitting technique to use in order to obtain a realistic measurement as well as realistic uncertainties. We conclude that SED fitting can be used effectively to derive a range of physical properties of galaxies, such as redshift, stellar masses, star formation rates, dust masses, and metallicities, with care taken not to over-interpret the available data. Yet there still exist many issues such as estimating the age of the oldest stars in a galaxy, finer details ofdust properties and dust-star geometry, and the influences of poorly understood, luminous stellar types and phases. The challenge for the coming years will be to improve both the models and the observational data sets to resolve these uncertainties. The present review will be made available on an interactive, moderated web page (sedfitting.org), where the community can access and change the text. The intention is to expand the text and keep it up to date over the coming years.

We use measurements of the HI content, stellar mass and star formation rates in ~190 massive galaxies with stellar masses greater than 10^10 Msun, obtained from the Galex Arecibo SDSS Survey (GASS) described in Paper I (Catinella et al. 2010) to explore the global scaling relations associated with the bin-averaged ratio of the star formation rate over the HI mass, which we call the HI-based star formation efficiency (SFE). Unlike the mean specific star formation rate, which decreases with stellar mass and stellar mass surface density, the star formation efficiency remains relatively constant across the sample with a value close to SFE = 10^-9.5 yr^-1 (or an equivalent gas consumption timescale of ~3 Gyr). Specifically, we find little variation in SFE with stellar mass, stellar mass surface density, NUV-r color and concentration. We interpret these results as an indication that external processes or feedback mechanisms that control the gas supply are important for regulating star formation in massive galaxies. An investigation into the detailed distribution of SFEs reveals that approximately 5% of the sample shows high efficiencies with SFE > 10^-9 yr^-1, and we suggest that this is very likely due to a deficiency of cold gas rather than an excess star formation rate. Conversely, we also find a similar fraction of galaxies that appear to be gas-rich for their given specific star-formation rate, although these galaxies show both a higher than average gas fraction and lower than average specific star formation rate. Both of these populations are plausible candidates for "transition" galaxies, showing potential for a change (either decrease or increase) in their specific star formation rate in the near future. We also find that 36+/-5% of the total HI mass density and 47+/-5% of the total SFR density is found in galaxies with stellar mass greater than 10^10 Msun. [abridged]

Sky coverage is one of the most important pieces of information about astronomical observations. We discuss possible representations, and present algorithms to create and manipulate shapes consisting of generalized spherical polygons with arbitrary complexity and size on the celestial sphere. This shape specification integrates well with our Hierarchical Triangular Mesh indexing toolbox, whose performance and capabilities are enhanced by the advanced features presented here. Our portable implementation of the relevant spherical geometry routines comes with wrapper functions for database queries, which are currently being used within several scientific catalog archives including the Sloan Digital Sky Survey, the Galaxy Evolution Explorer and the Hubble Legacy Archive projects as well as the Footprint Service of the Virtual Observatory.

We present a multiscale approach to measurements of galaxy density, applied to a volume-limited sample constructed from SDSS DR5. We populate a rich parameter space by obtaining independent measurements of density on different scales for each galaxy, avoiding the implicit assumptions involved, e.g., in the construction of group catalogues. As the first application of this method, we study how the bimodality in galaxy colour distribution (u-r) depends on multiscale density. The u-r galaxy colour distribution is described as the sum of two gaussians (red and blue) with five parameters: the fraction of red galaxies (f_r) and the position and width of the red and blue peaks (mu_r, mu_b, sigma_r and sigma_b). Galaxies mostly react to their smallest scale (< 0.5 Mpc) environments: in denser environments red galaxies are more common (larger f_r), redder (larger mu_r) and with a narrower distribution (smaller sigma_r), while blue galaxies are redder (larger mu_b) but with a broader distribution (larger sigma_b). There are residual correlations of f_r and mu_b with 0.5 - 1 Mpc scale density, which imply that total or partial truncation of star formation can relate to a galaxy's environment on these scales. Beyond 1 Mpc (0.5 Mpc for mu_r) there are no positive correlations with density. However f_r (mu_r) anti-correlates with density on >2 (1) Mpc scales at fixed density on smaller scales. We examine these trends qualitatively in the context of the halo model, utilizing the properties of haloes within which the galaxies are embedded, derived by Yang et al, 2007 and applied to a group catalogue. This yields an excellent description of the trends with multiscale density, including the anti-correlations on large scales, which map the region of accretion onto massive haloes. Thus we conclude that galaxies become red only once they have been accreted onto haloes of a certain mass.

Physical processes influencing the properties of galaxies can be traced by the dependence and evolution of galaxy properties on their environment. A detailed understanding of this dependence can only be gained through comparison of observations with models, with an appropriate quantification of the rich parameter space describing the environment of the galaxy. We present a new, multiscale parameterization of galaxy environment which retains an observationally motivated simplicity whilst utilizing the information present on different scales. We examine how the distribution of galaxy (u-r) colours in the Sloan Digital Sky Survey (SDSS), parameterized using a double gaussian (red plus blue peak) fit, depends upon multiscale density. This allows us to probe the detailed dependence of galaxy properties on environment in a way which is independent of the halo model. Nonetheless, cross-correlation with the group catalogue constructed by Yang et al, 2007 shows that galaxy properties trace environment on different scales in a way which mimics that expected within the halo model. This provides independent support for the existence of virialized haloes, and important additional clues to the role played by environment in the evolution of the galaxy population. This work is described in full by Wilman et al., 2010, MNRAS, accepted

Extinction in galaxies affects their observed properties. In scenarios describing the distribution of dust and stars in individual disk galaxies, the amplitude of the extinction can be modulated by the inclination of the galaxies. In this work we investigate the inclination dependency in composite spectra of star-forming disk galaxies from the Sloan Digital Sky Survey Data Release 5. In a volume-limited sample within a redshift range 0.065-0.075 and a r-band Petrosian absolute magnitude range -19.5 to -22 which exhibits a flat distribution of inclination, the inclined relative to face-on extinction in the stellar continuum is found empirically to increase with inclination in the g, r, and i bands. Within the central 0.5 intrinsic half-light radius of the galaxies, the g-band relative extinction in the stellar continuum for the highly-inclined objects (axis ratio b/a = 0.1) is 1.2 mag, agreeing with previous studies. The extinction curve of the disk galaxies is given in the restframe wavelengths 3700-8000 angstrom, identified with major optical emission and absorption lines in diagnostics. The Balmer decrement remains constant with inclination, suggesting a different kind of dust configuration and/or reddening mechanism in the HII region from that in the stellar continuum. One factor is shown to be the presence of spatially non-uniform interstellar extinction, presumably caused by clumped dust in the vicinity of the HII region.

The spectroscopic Sloan Digital Sky Survey (SDSS) Data Release 7 (DR7) galaxy sample represents the final set of galaxies observed using the original SDSS target selection criteria. We analyse the clustering of galaxies within this sample, including both the Luminous Red Galaxy (LRG) and Main samples, and also include the 2-degree Field Galaxy Redshift Survey (2dFGRS) data. Baryon Acoustic Oscillations are observed in power spectra measured for different slices in redshift; this allows us to constrain the distance--redshift relation at multiple epochs. We achieve a distance measure at redshift z=0.275, of r_s(z_d)/D_V(0.275)=0.1390+/-0.0037 (2.7% accuracy), where r_s(z_d) is the comoving sound horizon at the baryon drag epoch, D_V(z)=[(1+z)^2D_A^2cz/H(z)]^(1/3), D_A(z) is the angular diameter distance and H(z) is the Hubble parameter. We find an almost independent constraint on the ratio of distances D_V(0.35)/D_V(0.2)=1.736+/-0.065, which is consistent at the 1.1sigma level with the best fit Lambda-CDM model obtained when combining our z=0.275 distance constraint with the WMAP 5-year data. The offset is similar to that found in previous analyses of the SDSS DR5 sample, but the discrepancy is now of lower significance, a change caused by a revised error analysis and a change in the methodology adopted, as well as the addition of more data. Using WMAP5 constraints on Omega_bh^2 and Omega_ch^2, and combining our BAO distance measurements with those from the Union Supernova sample, places a tight constraint on Omega_m=0.286+/-0.018 and H_0 = 68.2+/-2.2km/s/Mpc that is robust to allowing curvature and non-Lambda dark energy. This result is independent of the behaviour of dark energy at redshifts greater than those probed by the BAO and supernova measurements. (abridged)

We investigate the quality of associations of astronomical sources from multi-wavelength observations using simulated detections that are realistic in terms of their astrometric accuracy, small-scale clustering properties and selection functions. We present a general method to build such mock catalogs for studying associations, and compare the statistics of cross-identifications based on angular separation and Bayesian probability criteria. In particular, we focus on the highly relevant problem of cross-correlating the ultraviolet Galaxy Evolution Explorer (GALEX) and optical Sloan Digital Sky Survey (SDSS) surveys. Using refined simulations of the relevant catalogs, we find that the probability thresholds yield lower contamination of false associations, and are more efficient than angular separation. Our study presents a set of recommended criteria to construct reliable cross-match catalogs between SDSS and GALEX with minimal artifacts.

We present the power spectrum of the reconstructed halo density field derived from a sample of Luminous Red Galaxies (LRGs) from the Sloan Digital Sky Survey Seventh Data Release (DR7). The halo power spectrum has a direct connection to the underlying dark matter power for k <= 0.2 h/Mpc, well into the quasi-linear regime. This enables us to use a factor of ~8 more modes in the cosmological analysis than an analysis with kmax = 0.1 h/Mpc, as was adopted in the SDSS team analysis of the DR4 LRG sample (Tegmark et al. 2006). The observed halo power spectrum for 0.02 < k < 0.2 h/Mpc is well-fit by our model: chi^2 = 39.6 for 40 degrees of freedom for the best fit LCDM model. We find \Omega_m h^2 * (n_s/0.96)^0.13 = 0.141^{+0.009}_{-0.012} for a power law primordial power spectrum with spectral index n_s and \Omega_b h^2 = 0.02265 fixed, consistent with CMB measurements. The halo power spectrum also constrains the ratio of the comoving sound horizon at the baryon-drag epoch to an effective distance to z=0.35: r_s/D_V(0.35) = 0.1097^{+0.0039}_{-0.0042}. Combining the halo power spectrum measurement with the WMAP 5 year results, for the flat LCDM model we find \Omega_m = 0.289 +/- 0.019 and H_0 = 69.4 +/- 1.6 km/s/Mpc. Allowing for massive neutrinos in LCDM, we find \sum m_{\nu} < 0.62 eV at the 95% confidence level. If we instead consider the effective number of relativistic species Neff as a free parameter, we find Neff = 4.8^{+1.8}_{-1.7}. Combining also with the Kowalski et al. (2008) supernova sample, we find \Omega_{tot} = 1.011 +/- 0.009 and w = -0.99 +/- 0.11 for an open cosmology with constant dark energy equation of state w.

We measure the projected spatial correlation function w_p(r_p) from a large sample combining GALEX ultraviolet imaging with the SDSS spectroscopic sample. We study the dependence of the clustering strength for samples selected on (NUV - r)_abs color, specific star formation rate (SSFR), and stellar mass. We find that there is a smooth transition in the clustering of galaxies as a function of this color from weak clustering among blue galaxies to stronger clustering for red galaxies. The clustering of galaxies within the "green valley" has an intermediate strength, and is consistent with that expected from galaxy groups. The results are robust to the correction for dust extinction. The comparison with simple analytical modeling suggests that the halo occupation number increases with older star formation epochs. When splitting according to SSFR, we find that the SSFR is a more sensitive tracer of environment than stellar mass.

We present a detailed study of the Galaxy Evolution Explorer's photometric catalogs with special focus on the statistical properties of the All-sky and Medium Imaging Surveys. We introduce the concept of primaries to resolve the issue of multiple detections and follow a geometric approach to define clean catalogs with well-understood selection functions. We cross-identify the GALEX sources (GR2+3) with Sloan Digital Sky Survey (DR6) observations, which indirectly provides an invaluable insight about the astrometric model of the UV sources and allows us to revise the band merging strategy. We derive the formal description of the GALEX footprints as well as their intersections with the SDSS coverage along with analytic calculations of their areal coverage. The crossmatch catalogs are made available for the public. We conclude by illustrating the implementation of typical selection criteria in SQL for catalog subsets geared toward statistical analyses, e.g., correlation and luminosity function studies.

We present a novel technique to overcome the limitations of the applicability of Principal Component Analysis to typical real-life data sets, especially astronomical spectra. Our new approach addresses the issues of outliers, missing information, large number of dimensions and the vast amount of data by combining elements of robust statistics and recursive algorithms that provide improved eigensystem estimates step-by-step. We develop a generic mechanism for deriving reliable eigenspectra without manual data censoring, while utilising all the information contained in the observations. We demonstrate the power of the methodology on the attractive collection of the VIMOS VLT Deep Survey spectra that manifest most of the challenges today, and highlight the improvements over previous workarounds, as well as the scalability of our approach to collections with sizes of the Sloan Digital Sky Survey and beyond.

Under the unified model for active galactic nuclei (AGNs), narrow-line (Type 2) AGNs are, in fact, broad-line (Type 1) AGNs but each with a heavily obscured accretion disk. We would therefore expect the optical continuum emission from Type 2 AGN to be composed mainly of stellar light and non-variable on the time-scales of months to years. In this work we probe the spectroscopic variability of galaxies and narrow-line AGNs using the multi-epoch data in the Sloan Digital Sky Survey (SDSS) Data Release 6. The sample contains 18,435 sources for which there exist pairs of spectroscopic observations (with a maximum separation in time of ~700 days) covering a wavelength range of 3900-8900 angstrom. To obtain a reliable repeatability measurement between each spectral pair, we consider a number of techniques for spectrophotometric calibration resulting in an improved spectrophotometric calibration of a factor of two. From these data we find no obvious continuum and emission-line variability in the narrow-line AGNs on average -- the spectroscopic variability of the continuum is 0.07+/-0.26 mag in the g band and, for the emission-line ratios log10([NII]/Halpha) and log10([OIII]/Hbeta), the variability is 0.02+/-0.03 dex and 0.06+/-0.08 dex, respectively. From the continuum variability measurement we set an upper limit on the ratio between the flux of varying spectral component, presumably related to AGN activities, and that of host galaxy to be ~30%. We provide the corresponding upper limits for other spectral classes, including those from the BPT diagram, eClass galaxy classification, stars and quasars.

We present a rigorous mathematical solution to photometric redshift estimation and the more general inversion problem. The challenge we address is to meaningfully constrain unknown properties of astronomical sources based on given observables, usually multicolor photometry, with the help of a training set that provides an empirical relation between the measurements and the desired quantities. We establish a formalism that blurs the boundary between the traditional empirical and template-fitting algorithms, as both are just special cases that are discussed in detail to put them in context. The new approach enables the development of more sophisticated methods that go beyond the classic techniques to combine their advantages. We look at the directions for further improvement in the methodology, and examine the technical aspects of practical implementations. We show how training sets are to be constructed and used consistently for reliable estimation.

In the last decade we have seen an enormous increase in the size and quality of spectroscopic galaxy surveys, both at low and high redshift. New statistical techniques to analyse large portions of galaxy spectra are now finding favour over traditional index based methods. Here we will review a new robust and iterative Principal Component Analysis (PCA) algorithm, which solves several common issues with classic PCA. Application to the 4000AA break region of galaxies in the VIMOS VLT Deep Survey (VVDS) and Sloan Digital Sky Survey (SDSS) gives new high signal-to-noise ratio spectral indices easily interpretable in terms of recent star formation history. In particular, we identify a sample of post-starburst galaxies at z~0.7 and z~0.07. We quantify for the first time the importance of post-starburst galaxies, consistent with being descendants of gas-rich major mergers, for building the red sequence. Finally, we present a comparison with new low and high redshift "mock spectroscopic surveys" derived from a Millennium Run semi-analytic model.

We present a general probabilistic formalism for cross-identifying astronomical point sources in multiple observations. Our Bayesian approach, symmetric in all observations, is the foundation of a unified framework for object matching, where not only spatial information, but physical properties, such as colors, redshift and luminosity, can also be considered in a natural way. We provide a practical recipe to implement an efficient recursive algorithm to evaluate the Bayes factor over a set of catalogs with known circular errors in positions. This new methodology is crucial for studies leveraging the synergy of today's multi-wavelength observations and to enter the time-domain science of the upcoming survey telescopes.

We use the UV-optical color magnitude diagram in combination with spectroscopic and photometric measurements derived from the SDSS spectroscopic sample to measure the distribution of galaxies in the local universe (z<0.25) and their physical properties as a function of specific star formation rate (SSFR) and stellar mass. Throughout this study our emphasis is on the properties of galaxies on and off of a local "star-forming sequence." We discuss how the physical characteristics of galaxies along this sequence are related to scaling relations typically derived for galaxies of different morphological types. We find, among other trends that our measure of the star formation rate surface density is nearly constant along this sequence. We discuss this result and implications for galaxies at higher redshift. For the first time, we report on measurements of the local UV luminosity function versus galaxy structural parameters as well as inclination. We also split our sample into disk-dominated and bulge-dominated subsamples using the i-band Sersic index and find that disk-dominated galaxies occupy a very tight locus in SSFR vs. stellar mass space while bulge-dominated galaxies display a much larger spread of SSFR at fixed stellar mass. A significant fraction of galaxies with SSFR and SF surface density above those on the "star-forming sequence" are bulge-dominated. We can use our derived distribution functions to ask whether a significant fraction of these galaxies may be experiencing a final episode of star formation (possibly induced by a merger or other burst), soon to be quenched, by determining whether this population can explain the growth rate of the non-star-forming galaxies on the "red sequence." (Abridged)

We present the first measurements of the angular correlation function of galaxies selected in the far (1530 A) and near (2310 A) Ultraviolet from the GALEX survey fields overlapping SDSS DR5 in low galactic extinction regions. The area used covers 120 sqdeg (GALEX - MIS) down to magnitude AB = 22, yielding a total of 100,000 galaxies. The mean correlation length is ~ 3.7 \pm 0.6 Mpc and no significant trend is seen for this value as a function of the limiting apparent magnitude or between the GALEX bands. This estimate is close to that found from samples of blue galaxies in the local universe selected in the visible, and similar to that derived at z ~ 3 for LBGs with similar rest frame selection criteria. This result supports models that predict anti-biasing of star forming galaxies at low redshift, and brings an additional clue to the downsizing of star formation at z<1.

We have analyzed the bivariate distribution of galaxies as a function of ultraviolet-optical colors and absolute magnitudes in the local universe. The sample consists of galaxies with redshifts and optical photometry from the Sloan Digital Sky Survey (SDSS) main galaxy sample matched with detections in the near-ultraviolet (NUV) and far-ultraviolet (FUV) bands in the Medium Imaging Survey being carried out by the Galaxy Evolution Explorer (GALEX) satellite. In the (NUV-r)_{0.1} vs. M_{r,0.1} galaxy color-magnitude diagram, the galaxies separate into two well-defined blue and red sequences. The (NUV-r)_{0.1} color distribution at each M_{r,0.1} is not well fit by the sum of two Gaussians due to an excess of galaxies in between the two sequences. The peaks of both sequences become redder with increasing luminosity with a distinct blue peak visible up to M_{r,0.1}\sim-23. The r_{0.1}-band luminosity functions vary systematically with color, with the faint end slope and characteristic luminosity gradually increasing with color. After correcting for attenuation due to dust, we find that approximately one quarter of the color variation along the blue sequence is due to dust with the remainder due to star formation history and metallicity. Finally, we present the distribution of galaxies as a function of specific star formation rate and stellar mass. The specific star formation rates imply that galaxies along the blue sequence progress from low mass galaxies with star formation rates that increase somewhat with time to more massive galaxies with a more or less constant star formation rate. Above a stellar mass of ~10^10.5 M_{sun}, galaxies with low ratios of current to past averaged star formation rate begin to dominate.

We analyze the clustering properties of ultraviolet selected galaxies by using GALEX-SDSS data at z<0.6 and CFHTLS deep u' imaging at z=1. These datasets provide a unique basis at z< 1 which can be directly compared with high redshift samples built with similar selection criteria. We discuss the dependence of the correlation function parameters (r0, delta) on the ultraviolet luminosity as well as the linear bias evolution. We find that the bias parameter shows a gradual decline from high (b > 2) to low redshift (b ~ 0.79^{+0.1}_{-0.08}). When accounting for the fraction of the star formation activity enclosed in the different samples, our results suggest that the bulk of star formation migrated from high mass dark matter halos at z>2 (10^12 < M_min < 10^13 M_sun, located in high density regions), to less massive halos at low redshift (M_min < 10^12 M_sun, located in low density regions). This result extends the ``downsizing'' picture (shift of the star formation activity from high stellar mass systems at high z to low stellar mass at low z) to the dark matter distribution.

We describe the calibration status and data products pertaining to the GR2 and GR3 data releases of the Galaxy Evolution Explorer (GALEX). These releases have identical pipeline calibrations that are significantly improved over the GR1 data release. GALEX continues to survey the sky in the Far Ultraviolet (FUV, ~154 nm) and Near Ultraviolet (NUV, ~232 nm) bands, providing simultaneous imaging with a pair of photon counting, microchannel plate, delay line readout detectors. These 1.25 degree field-of-view detectors are well-suited to ultraviolet observations because of their excellent red rejection and negligible background. A dithered mode of observing and photon list output pose complex requirements on the data processing pipeline, entangling detector calibrations and aspect reconstruction algorithms. Recent improvements have achieved photometric repeatability of 0.05 and 0.03 mAB in the FUV and NUV, respectively. We have detected a long term drift of order 1% FUV and 6% NUV over the mission. Astrometric precision is of order 0.5" RMS in both bands. In this paper we provide the GALEX user with a broad overview of the calibration issues likely to be confronted in the current release. Improvements are likely as the GALEX mission continues into an extended phase with a healthy instrument, no consumables, and increased opportunities for guest investigations.

Using subspace methods, we study the distribution of physical components of galaxies in wavelength space. We find that it is valid to assume that the stellar and the gaseous components of galaxies span complementary subspaces. To first order, stellar and gaseous spectral features can be extracted from galaxy spectra through a simple matrix multiplication. By comparing the stellar continua obtained respectively using the model-based and the empirical approach through a commonality measure, we conclude that the latter may lose higher-order spectral features.

We use the Galaxy Evolution Explorer (GALEX) Medium and All-Sky-Imaging Survey (MIS & AIS) data from the first public data release (GR1), matched to the Sloan Digital Sky Survey (SDSS) DR3 catalog, to perform source classification. The GALEX surveys provide photometry in far- and near-UV bands and the SDSS in five optical bands (u,g,r,i,z). The GR1/DR3 overlapping areas are 363[83]deg^2 for the GALEX AIS[MIS], for sources within the 0.5deg central area of the GALEX fields. Our sample covers mostly |b|>30deg galactic latitudes. We present statistical properties of the GALEX/SDSS matched sources catalog, containing >2x10^6 objects detected in at least one UV band. We classify the matched sources by comparing the seven-band photometry to model colors constructed for different classes of astrophysical objects. For sources with photometric errors <0.3 mag, the corresponding typical AB-magnitude limits are m_FUV~21.5, m_NUV~22.5 for AIS, and m_FUV~24, m_NUV~24.5 for MIS. At AIS depth, the number of Galactic and extragalactic objects are comparable, but the latter predominate in the MIS. Based on our stellar models, we estimate the GALEX surveys detect hot White Dwarfs throughout the Milky Way halo (down to a radius of 0.04 R_sun at MIS depth), providing an unprecedented improvement in the Galactic WD census. Their observed surface density is consistent with Milky Way model predictions. We also select low-redshift QSO candidates, extending the known QSO samples to lower magnitudes, and providing candidates for detailed z~1 follow-up investigations. SDSS optical spectra available for a large subsample confirm the classification for the photometrically selected candidates with 97% purity for single hot stars, ~45%(AIS)/31%(MIS) for binaries containing a hot star and a cooler companion, and about 85% for QSOs.

We measure the large-scale real-space power spectrum P(k) using luminous red galaxies (LRGs) in the Sloan Digital Sky Survey (SDSS) and use this measurement to sharpen constraints on cosmological parameters from the Wilkinson Microwave Anisotropy Probe (WMAP). We employ a matrix-based power spectrum estimation method using Pseudo-Karhunen-Loeve eigenmodes, producing uncorrelated minimum-variance measurements in 20 k-bands of both the clustering power and its anisotropy due to redshift-space distortions, with narrow and well-behaved window functions in the range 0.01h/Mpc < k < 0.2h/Mpc. Results from the LRG and main galaxy samples are consistent, with the former providing higher signal-to-noise. Our results are robust to omitting angular and radial density fluctuations and are consistent between different parts of the sky. They provide a striking confirmation of the predicted large-scale LCDM power spectrum. Combining only SDSS LRG and WMAP data places robust constraints on many cosmological parameters that complement prior analyses of multiple data sets. The LRGs provide independent cross-checks on Om and the baryon fraction in good agreement with WMAP. Within the context of flat LCDM models, our LRG measurements complement WMAP by sharpening the constraints on the matter density, the neutrino density and the tensor amplitude by about a factor of two, giving Omega_m=0.24+-0.02 (1 sigma), sum m_nu < 0.9 eV (95%) and r<0.3 (95%). Baryon oscillations are clearly detected and provide a robust measurement of the comoving distance to the median survey redshift z=0.35 independent of curvature and dark energy properties. Within the LCDM framework, our power spectrum measurement improves the evidence for spatial flatness, sharpening the curvature constraint Omega_tot=1.05+-0.05 from WMAP alone to Omega_tot=1.003+-0.010. Assuming Omega_tot=1, the equation of state parameter is constrained to w=-0.94+-0.09, indicating the potential for more ambitious future LRG measurements to provide precision tests of the nature of dark energy. All these constraints are essentially independent of scales k>0.1h/Mpc and associated nonlinear complications, yet agree well with more aggressive published analyses where nonlinear modeling is crucial.

We analyse a z<0.1 galaxy sample from the Sloan Digital Sky Survey focusing on the variation of the galaxy colour bimodality with stellar mass and projected neighbour density Sigma, and on measurements of the galaxy stellar mass functions. The characteristic mass increases with environmental density from about 10^10.6 Msun to 10^10.9 Msun (Kroupa IMF, H_0=70) for Sigma in the range 0.1--10 per Mpc^2. The galaxy population naturally divides into a red and blue sequence with the locus of the sequences in colour-mass and colour-concentration index not varying strongly with environment. The fraction of galaxies on the red sequence is determined in bins of 0.2 in log Sigma and log mass (12 x 13 bins). The red fraction f_r generally increases continuously in both Sigma and mass such that there is a unified relation: f_r = F(Sigma,mass). Two simple functions are proposed which provide good fits to the data. These data are compared with analogous quantities in semi-analytical models based on the Millennium N-body simulation: the Bower et al. (2006) and Croton et al. (2006) models that incorporate AGN feedback. Both models predict a strong dependence of the red fraction on stellar mass and environment that is qualitatively similar to the observations. However, a quantitative comparison shows that the Bower et al. model is a significantly better match; this appears to be due to the different treatment of feedback in central galaxies.

We analyze a volume-limited sample of massive bulge-dominated galaxies with data from both the Sloan Digital Sky Survey and the Galaxy Evolution Explorer (GALEX) satellite. The galaxies have central velocity dispersions greater than 100 km/s and stellar surface mass densities that lie above the value where galaxies transition from actively star forming to passive systems. The sample is limited to redshifts 0.03<z<0.07. At these distances, the SDSS spectra sample the light from the bulge-dominated central regions of the galaxies. The GALEX NUV data provide high sensitivity to low rates of global star formation in these systems. Our sample of bulge-dominated galaxies exhibits a much larger dispersion in NUV-r colour than in optical g-r colour. Nearly all of the galaxies with bluer NUV-r colours are AGN. Both GALEX images and SDSS colour profiles demonstrate that the excess UV light is associated with an extended disk. We find that galaxies with red outer regions almost never have a young bulge or a strong AGN. Galaxies with blue outer regions have bulges and black holes that span a wide range in age and accretion rate. Galaxies with young bulges and strongly accreting black holes almost always have blue outer disks. Our suggested scenario is one in which the source of gas that builds the bulge and black hole is a low mass reservoir of cold gas in the disk.The presence of this gas is a necessary, but not sufficient condition for bulge and black hole growth. Some mechanism must transport this gas inwards in a time variable way. As the gas in the disk is converted into stars, the galaxies will turn red, but further gas infall can bring them back into the blue NUV-r sequence.(Abridged)

We present an analysis of the mid-infrared (MIR) and optical properties of type 1 (broad-line) quasars detected by the Spitzer Space Telescope. The MIR color-redshift relation is characterized to z=3, with predictions to z=7. We demonstrate how combining MIR and optical colors can yield even more efficient selection of active galactic nuclei (AGN) than MIR or optical colors alone. Composite spectral energy distributions (SEDs) are constructed for 259 quasars with both Sloan Digital Sky Survey and Spitzer photometry, supplemented by near-IR, GALEX, VLA and ROSAT data where available. We discuss how the spectral diversity of quasars influences the determination of bolometric luminosities and accretion rates; assuming the mean SED can lead to errors as large as a factor of 2 for individual quasars. Finally, we show that careful consideration of the shape of the mean quasar SED and its redshift dependence leads to a lower estimate of the fraction of reddened/obscured AGNs missed by optical surveys as compared to estimates derived from a single mean MIR to optical flux ratio.

[Abridged] We use mock catalogues based on the GALICS model (Hatton et al. 03) to explore the nature of galaxy clustering observed in the SDSS. We measure low and high order angular clustering statistic from these mock catalogues, after selecting galaxies the same way as for observations, and compare them directly to estimates from SDSS data. Note that we also present measurements of S3-S5 on the SDSS DR1. We find that our model is in general good agreement with observations in the scale/luminosity range where we can trust the predictions. This range is found to be limited (i) by the size of the dark matter simulation used -- which introduces finite volume effects at large scales -- and by the mass resolution of this simulation -- which introduces incompleteness at apparent magnitudes fainter than $r\sim 20$. We then focus on the small scale clustering properties of galaxies and investigate the behaviour of three different prescriptions for positioning galaxies within haloes of dark matter. We show that galaxies are poor tracers both of DM particles or DM sub-structures, within groups and clusters. Instead, SDSS data tells us that the distribution of galaxies lies somewhat in between these two populations. This confirms the general theoretical expectation from numerical simulations and semi-analytic modelling.

In this paper we describe a detailed analysis of the photometric uncertainties present within the Sloan Digital Sky Survey (SDSS) imaging survey based on repeat observations of approximately 200 square degrees of the sky. We show that, for the standard SDSS aperture systems (petrocounts, counts_model, psfcounts and cmodel_counts), the errors generated by the SDSS photometric pipeline under-estimate the observed scatter in the individual bands. The degree of disagreement is a strong function of aperture and magnitude (ranging from 20% to more than a factor of 2). We also find that the photometry in the five optical bands can be highly correlated for both point sources and galaxies, although the correlation for point sources is almost entirely due to variable objects. Without correcting for this covariance the SDSS color errors could be in over-estimated by a factor of two to three. Combining these opposing effects, the SDSS errors on the colors differ from the observed color variation by approximately 10-20% for most apertures and magnitudes. We provide a prescription to correct the errors derived from the SDSS photometric pipeline as a function of magnitude and a semi-analytic method for generating the appropriate covariance between the different photometric passbands. Given the intrinsic nature of these correlations, we expect that all current and future multi-band surveys will also observe strongly covariant magnitudes. The ability of these surveys to complete their science goals is largely dependent on color-based target selection and photometric redshifts; these results show the importance of spending a significant fraction of early survey operations on re-imaging to empirically determine the photometric covariance of any observing/reduction pipeline.

We discuss the construction of a photometric redshift catalogue of Luminous Red Galaxies (LRGs) from the Sloan Digital Sky Survey (SDSS), emphasizing the principal steps necessary for constructing such a catalogue -- (i) photometrically selecting the sample, (ii) measuring photometric redshifts and their error distributions, (iii) and estimating the true redshift distribution. We compare two photometric redshift algorithms for these data and find that they give comparable results. Calibrating against the SDSS and SDSS-2dF spectroscopic surveys, we find that the photometric redshift accuracy is $\sigma \sim 0.03$ for redshifts less than 0.55 and worsens at higher redshift ($\sim 0.06$). These errors are caused by photometric scatter, as well as systematic errors in the templates, filter curves, and photometric zeropoints. We also parametrize the photometric redshift error distribution with a sum of Gaussians, and use this model to deconvolve the errors from the measured photometric redshift distribution to estimate the true redshift distribution. We pay special attention to the stability of this deconvolution, regularizing the method with a prior on the smoothness of the true redshift distribution. The methods we develop are applicable to general photometric redshift surveys.

We construct and analyze a u-band selected galaxy sample from the SDSS Southern Survey, which covers 275 sq. deg. The sample includes 43223 galaxies with spectroscopic redshifts in the range 0.005<z<0.3 and with 14.5<u<20.5. The S/N in the u-band Petrosian aperture is improved by coadding multiple epochs of imaging data and by including sky-subtraction corrections. Luminosity functions for the near-UV ^{0.1}u band (322+-26 nm) are determined in redshift slices of width 0.02, which show a highly significant evolution in M^{star} of -0.8+-0.1 mag between z=0 and 0.3; with M^{star} = -18.84+-0.05 (AB mag), log phi^{star} = -2.06+-0.03 (Mpc^{-3}) and log rho_L = 19.11+-0.02 (W Hz^{-1} Mpc^{-3}) at z=0.1. The faint-end slope determined for z<0.06 is given by alpha=-1.05+-0.08. This is in agreement with recent determinations from GALEX at shorter wavelengths. Comparing our z<0.3 luminosity density measurements with 0.2<z<1.2 from COMBO-17, we find that the 280-nm density evolves as rho_L proportional to (1+z)^{beta} with beta=2.1+-0.2; and find no evidence for any change in slope over this redshift range. By comparing with other measurements of cosmic star formation history, we estimate that the effective dust attenuation at 280 nm has increased by 0.8+-0.3 mag between z=0 and 1.

We present the first measurement of the galaxy luminosity function at 1500A between 0.2<z<1.2 based on GALEX-VVDS observations (1000 spectroscopic redshifts for galaxies with NUV<24.5) and at higher z using existing datasets. Our main results are summarized as follows : (i) luminosity evolution is observed with Delta(Mstar)=-2.0 mag between z=0 and z=1 and Delta(Mstar)=-1.0mag between z=1 and z=3. This confirms that the star formation activity was significantly higher in the past. (ii) the LF slopes vary between -1.2< alpha <-1.65, with a marginally significant hint of increase at higher z. (iii) we split the sample in three restframe (B-I) intervals providing an approximate spectral type classification: Sb-Sd, Sd-Irr and unobscured starbursts. We find that the bluest class evolves less strongly in luminosity than the two other classes. On the other hand their number density increases sharply with z (15% in the local universe to 55% at z=1) while that of the reddest classes decreases.

In a companion paper (Arnouts et al. 2004) we presented new measurements of the galaxy luminosity function at 1500 Angstroms out to z~1 using GALEX-VVDS observations (1039 galaxies with NUV<24.5 and z>0.2) and at higher z using existing data sets. In this paper we use the same sample to study evolution of the FUV luminosity density. We detect evolution consistent with a (1+z)^{2.5+/-0.7} rise to z~1 and (1+z)^{0.5+/-0.4} for z>1. The luminosity density from the most UV-luminous galaxies (UVLG) is undergoing dramatic evolution (x30) between 0<z<1. UVLGs are responsible for a significant fraction (>25%) of the total FUV luminosity density at z<1. We measure dust attenuation and star formation rates of our sample galaxies and determine the star formation rate density as a function of redshift, both uncorrected and corrected for dust. We find good agreement with other measures of the SFR density in the rest ultraviolet and Halpha given the still significant uncertainties in the attenuation correction.

We present measurements of the FUV (1530A) and NUV (2310A) galaxy luminosity functions (LF) at low redshift (z<0.2) from GALEX observations matched to the 2dF Galaxy Redshift Survey. We split our FUV and NUV samples into two UV-bj color bins and two redshift bins. As observed at optical wavelengths, the local LF of the bluest galaxies tend to have steeper faint end slopes and fainter characteristic magnitudes M* than the reddest subsamples. We find evidence for color dependent evolution at very low redshift in both bands, with bright blue galaxies becoming dominant in the highest redshift bin. The evolution of the total LF is consistent with an 0.3 magnitude brightening between z=0 and 0.13, in agreement with the first analysis of deeper GALEX fields probing adjacent and higher redshifts.

We present measurements of the UV galaxy luminosity function and the evolution of luminosity density from GALEX observations matched to the Sloan Digital Sky Survey (SDSS). We analyze galaxies in the Medium Imaging Survey overlapping the SDSS DR1 with a total coverage of 44 deg^2. Using the combined GALEX+SDSS photometry, we compute photometric redshifts and study the LF in three redshift shells between z=0.07 and 0.25. The Schechter function fits indicate that the faint-end slope alpha is consistent with -1.1 at all redshifts but the characteristic UV luminosity M* brightens by 0.2 mag from z=0.07 to 0.25. In the lowest redshift bin, early and late type galaxies are studied separately and we confirm that red galaxies tend to be brighter and have a shallower slope alpha than blue ones. The derived luminosity densities are consistent with other GALEX results based on a local spectroscopic sample from 2dF and the evolution follows the trend reported by deeper studies.

We present an empirical algorithm for obtaining photometric redshifts of quasars using 5-band Sloan Digital Sky Survey (SDSS) photometry. Our algorithm generates an empirical model of the quasar color-redshift relation, compares the colors of a quasar candidate with this model, and calculates possible photometric redshifts. Using the 3814 quasars of the SDSS Early Data Release Quasar Catalog to generate a median color-redshift relation as a function of redshift we find that, for this same sample, 83% of our predicted redshifts are correct to within |Delta z|<0.3. The algorithm also determines the probability that the redshift is correct, allowing for even more robust photometric redshift determination for smaller, more restricted samples. We apply this technique to a set of 8740 quasar candidates selected by the final version of the SDSS quasar-selection algorithm. The photometric redshifts assigned to non-quasars are restricted to a few well-defined values. In addition, 90% of the objects with spectra that have photometric redshifts between 0.8 and 2.2 are quasars with accurate (|Delta z|<0.3) photometric redshifts. Many of these quasars lie in a single region of color space; judicious application of color-cuts can effectively select quasars with accurate photometric redshifts from the SDSS database. When the SDSS is complete, this technique will allow the determination of photometric redshifts for ~10^6 faint SDSS quasar candidates, enabling advances in our knowledge of the quasar luminosity function, gravitational lensing of quasars, and correlations among quasars and between galaxies.

We perform an objective classification of 170,000 galaxy spectra in the Sloan Digital Sky Survey (SDSS) using the Karhunen-Lo\`eve (KL) transform. With about one-sixth of the total set of galaxy spectra which will be obtained by the survey, we are able to carry out the most extensive analysis of its kind to date. The formalism proposed by Connolly and Szalay is adopted to correct for gappy regions in the spectra, and to derive eigenspectra and eigencoefficients. From this analysis, we show that this gap-correction formalism leads to a converging set of eigenspectra and KL-repaired spectra. Furthermore, KL eigenspectra of galaxies are found to be convergent not only as a function of iteration, but also as a function of the number of randomly selected galaxy spectra used in the analysis. From these data a set of ten eigenspectra of galaxy spectra are constructed, with rest-wavelength coverage 3450-8350 angstrom. The eigencoefficients describing these galaxies naturally place the spectra into several classes defined by the plane formed by the first three eigencoefficients of each spectrum. Spectral types, corresponding to different Hubble-types and galaxies with extreme emission lines, are identified for the 170,000 spectra and are shown to be complementary to existing spectral classifications. From a non-parametric classification technique, we find that the population of galaxies can be divided into three classes which correspond to early late- through to intermediate late-types galaxies. (Abridged)

Galaxy-galaxy lensing has emerged as a powerful probe of the dark matter halos of galaxies, but is subject to contamination if intrinsically aligned satellites of the lens galaxy are used as part of the source sample. We present a measurement of this intrinsic shear using 200,747 lens galaxies from the Sloan Digital Sky Survey (SDSS) spectroscopic sample and a sample of satellites selected using photometric redshifts. The mean intrinsic shear at transverse separations of 30--446$h^{-1}$ kpc is constrained to be $-0.0062<\Delta\gamma<+0.0066$ (99.9 per cent confidence, including identified systematics), which limits contamination of the galaxy-galaxy lensing signal to at most $\sim 15$ per cent on these scales. We present these limits as a function of transverse separation and lens luminosity. We furthermore investigate shear calibration biases in the SDSS, which can also affect galaxy-galaxy lensing, and conclude that the shear amplitude is calibrated to better than 18 per cent. This includes noise-induced calibration biases in the ellipticity, which are small for the sample considered here, but which can be more important if low signal-to-noise or poorly resolved source galaxies are used.

We present estimates of cosmological parameters from the application of the Karhunen-Loeve transform to the analysis of the 3D power spectrum of density fluctuations using Sloan Digital Sky Survey galaxy redshifts. We use Omega_m*h and f_b = Omega_b/Omega_m to describe the shape of the power spectrum, sigma8 for the (linearly extrapolated) normalization, and beta to parametrize linear theory redshift space distortions. On scales k < 0.16 h/Mpc, our maximum likelihood values are Omega_m*h = 0.264 +/-0.043, f_b = 0.286 +/- 0.065, sigma8 = 0.966 +/- 0.048, and beta = 0.45 +/- 0.12. When we take a prior on Omega_b from WMAP, we find Omega_m*h = 0.207 +/- 0.030, which is in excellent agreement with WMAP and 2dF. This indicates that we have reasonably measured the gross shape of the power spectrum but we have difficulty breaking the degeneracy between Omega_m*h and f_b because the baryon oscillations are not resolved in the current spectroscopic survey window function.

We present galaxy-galaxy lensing measurements over scales 0.025 to 10 Mpc/h in the Sloan Digital Sky Survey. Using a flux-limited sample of 127,001 lens galaxies with spectroscopic redshifts and mean luminosity <L> = L_* and 9,020,388 source galaxies with photometric redshifts, we invert the lensing signal to obtain the galaxy-mass correlation function xi_{gm}. We find xi_{gm} is consistent with a power-law, xi_{gm} = (r/r_0)^{-gamma}, with best-fit parameters gamma = 1.79 +/- 0.06 and r_0 = (5.4+/-0.7)(0.27/Omega_m)^{1/gamma} Mpc/h. At fixed separation, the ratio xi_{gg}/xi_{gm} = b/r where b is the bias and r is the correlation coefficient. Comparing to the galaxy auto-correlation function for a similarly selected sample of SDSS galaxies, we find that b/r is approximately scale independent over scales 0.2-6.7 Mpc/h, with mean <b/r> = (1.3+/-0.2)(Omega_m/0.27). We also find no scale dependence in b/r for a volume limited sample of luminous galaxies (-23.0 < M_r < -21.5). The mean b/r for this sample is <b/r>_{Vlim} = (2.0+/-0.7)(Omega_m/0.27). We split the lens galaxy sample into subsets based on luminosity, color, spectral type, and velocity dispersion, and see clear trends of the lensing signal with each of these parameters. The amplitude and logarithmic slope of xi_{gm} increases with galaxy luminosity. For high luminosities (L ~5 L_*), xi_{gm} deviates significantly from a power law. These trends with luminosity also appear in the subsample of red galaxies, which are more strongly clustered than blue galaxies.

We measure cosmological parameters using the three-dimensional power spectrum P(k) from over 200,000 galaxies in the Sloan Digital Sky Survey (SDSS) in combination with WMAP and other data. Our results are consistent with a ``vanilla'' flat adiabatic Lambda-CDM model without tilt (n=1), running tilt, tensor modes or massive neutrinos. Adding SDSS information more than halves the WMAP-only error bars on some parameters, tightening 1 sigma constraints on the Hubble parameter from h~0.74+0.18-0.07 to h~0.70+0.04-0.03, on the matter density from Omega_m~0.25+/-0.10 to Omega_m~0.30+/-0.04 (1 sigma) and on neutrino masses from <11 eV to <0.6 eV (95%). SDSS helps even more when dropping prior assumptions about curvature, neutrinos, tensor modes and the equation of state. Our results are in substantial agreement with the joint analysis of WMAP and the 2dF Galaxy Redshift Survey, which is an impressive consistency check with independent redshift survey data and analysis techniques. In this paper, we place particular emphasis on clarifying the physical origin of the constraints, i.e., what we do and do not know when using different data sets and prior assumptions. For instance, dropping the assumption that space is perfectly flat, the WMAP-only constraint on the measured age of the Universe tightens from t0~16.3+2.3-1.8 Gyr to t0~14.1+1.0-0.9 Gyr by adding SDSS and SN Ia data. Including tensors, running tilt, neutrino mass and equation of state in the list of free parameters, many constraints are still quite weak, but future cosmological measurements from SDSS and other sources should allow these to be substantially tightened.

We measure the large-scale real-space power spectrum P(k) using a sample of 205,443 galaxies from the Sloan Digital Sky Survey, covering 2417 square degrees with mean redshift z~0.1. We employ a matrix-based method using pseudo-Karhunen-Loeve eigenmodes, producing uncorrelated minimum-variance measurements in 22 k-bands of both the clustering power and its anisotropy due to redshift-space distortions, with narrow and well-behaved window functions in the range 0.02 h/Mpc < k < 0.3h/Mpc. We pay particular attention to modeling, quantifying and correcting for potential systematic errors, nonlinear redshift distortions and the artificial red-tilt caused by luminosity-dependent bias. Our final result is a measurement of the real-space matter power spectrum P(k) up to an unknown overall multiplicative bias factor. Our calculations suggest that this bias factor is independent of scale to better than a few percent for k<0.1h/Mpc, thereby making our results useful for precision measurements of cosmological parameters in conjunction with data from other experiments such as the WMAP satellite. As a simple characterization of the data, our measurements are well fit by a flat scale-invariant adiabatic cosmological model with h Omega_m =0.201+/- 0.017 and L* galaxy sigma_8=0.89 +/- 0.02 when fixing the baryon fraction Omega_b/Omega_m=0.17 and the Hubble parameter h=0.72; cosmological interpretation is given in a companion paper.

The redshift distribution of well-defined samples of distant early-type galaxies offers a means to test the predictions of monolithic and hierarchical galaxy formation scenarios. NICMOS maps of the entire Hubble Deep Field North in the F110W and F160W filters, when combined with the available WFPC2 data, allow us to calculate photometric redshifts and determine the morphological appearance of galaxies at rest-frame optical wavelengths out to z ~ 2.5. Here we report results for two subsamples of early-type galaxies, defined primarily by their morphologies in the F160W band, which were selected from the NICMOS data down to H160_{AB} < 24.0. The observed redshift distributions of our two early-type samples do not match that predicted by a monolithic collapse model, which shows an overabundance at z > 1.5. A hierarchical formation model better matches the redshift distribution of the HDF-N early-types at z > 1.5, but still does not adequately describe the observed early-types. The hierarchical model predicts significantly bluer colors on average than the observed early-type colors, and underpredicts the observed number of early-types at z < 1. [abridged]

We analyze the photometric information contained in individual pixels of galaxies in the Hubble Deep Field North (HDFN) using a new technique, _pixel-z_, that combines predictions of evolutionary synthesis models with photometric redshift template fitting. Each spectral energy distribution template is a result of modeling of the detailed physical processes affecting gas properties and star formation efficiency. The criteria chosen to generate the SED templates is that of sampling a wide range of physical characteristics such as age, star formation rate, obscuration and metallicity. A key feature of our method is the sophisticated use of error analysis to generate error maps that define the reliability of the template fitting on pixel scales and allow for the separation of the interplay among dust, metallicity and star formation histories. This technique offers a number of advantages over traditional integrated color studies. As a first application, we derive the star formation and metallicity histories of galaxies in the HDFN. Our results show that the comoving density of star formation rate, determined from the UV luminosity density of sources in the HDFN, increases monotonically with redshift out to at least redshift of 5. This behavior can plausibly be explained by a smooth increase of the UV luminosity density with redshift coupled with an increase in the number of star forming regions as a function of redshift. We also find that the information contained in individual pixels in a galaxy can be linked to its morphological history. Finally, we derive the metal enrichment rate history of the universe and find it in good agreement with predictions based on the evolving HI content of Lyman-alpha QSO absorption line systems.

We present measurements of the angular cross-correlation between luminous red galaxies from the Sloan Digital Sky Survey and the cosmic microwave background temperature maps from the Wilkinson Microwave Anisotropy Probe. We find a statistically significant achromatic positive correlation between these two data sets, which is consistent with the expected signal from the late Integrated Sachs-Wolfe (ISW) effect. We do not detect any anti-correlation on small angular scales as would be produced from a large Sunyaev-Zel'dovich (SZ) effect, although we do see evidence for some SZ effect for our highest redshift samples. Assuming a flat universe, our preliminary detection of the ISW effect provides independent physical evidence for the existence of dark energy.

Understanding the clustering of galaxies has long been a goal of modern observational cosmology. Utilizing our photometric redshift technique a volume limited sample containing more than 2 million galaxies is constructed from the SDSS galaxy catalog. In the largest such analysis to date, we study the angular clustering as a function of luminosity and spectral type. Using Limber's equation we calculate the clustering length for the full data set as r0=5.77+/-0.10 Mpc/h. We find that r0 increases with luminosity by a factor of 1.6 over the sampled luminosity range, in agreement with previous redshift surveys. We also find that both the clustering length and the slope of the correlation function depend on the galaxy type. In particular, by splitting the galaxies in four groups by their rest-frame type we find a bimodal behavior in their clustering properties. Galaxies with spectral types similar to elliptical galaxies have a correlation length of 6.59 +/- 0.17 Mpc/h and a slope of the angular correlation function of 0.96 +/- 0.05 while blue galaxies have a clustering length of 4.51 +/- 0.19 Mpc/h and a slope of 0.68 +/- 0.09. The two intermediate color groups behave like their more extreme 'siblings', rather than showing a gradual transition in slope. We discuss these correlations in the context of current cosmological models for structure formation.

We investigate the continuum and emission line properties of 4576 SDSS quasars as a function of their optical/UV SEDs. The optical/UV color distribution of our sample is roughly Gaussian, but with a red tail; we distinguish between 1) intrinsically blue (optically flat) quasars, 2) intrinsically red (optically steep) quasars, and 3) the 273 (6%) of our quasars whose continua are inconsistent with a single power-law and appear redder due to SMC-like dust reddening rather than synchrotron emission. The color distribution suggests that the population of moderately dust reddened broad-line quasars is smaller than that of unobscured quasars, but we estimate that a further 10% of the luminous quasar population is missing from the SDSS sample because of dust extinction with E(B-V)<0.5. We also investigate the emission and absorption line properties of these quasars as a function of color with regard to Boroson & Green type eigenvectors. Intrinsically red (optically steep) quasars tend to have narrower Balmer lines and weaker CIV, CIII], HeII and 3000A bump emission as compared with bluer (optically flatter) quasars. The change in strength of the 3000A bump appears to be dominated by the Balmer continuum and not by FeII emission. The dust reddened quasars have even narrower Balmer lines and weaker 3000A bumps, in addition to having considerably larger equivalent widths of [OII] and [OIII] emission. The fraction of broad absorption line quasars (BALQSOs) increases from ~3.4% for the bluest quasars to perhaps as large as 20% for the dust reddened quasars, but the intrinsic color distribution is affected by dust reddening. (abridged)

The build-up of stellar mass in galaxies is the consequence of their past star formation and merging histories. Here we report measurements of rest-frame optical light and calculations of stellar mass at high redshift based on an infrared-selected sample of galaxies from the Hubble Deep Field North. The bright envelope of rest-frame B-band galaxy luminosities is similar from 0<z<3, and the co-moving luminosity density is constant to within a factor of 3 over that redshift range. However, galaxies at higher redshifts are bluer, and stellar population modeling indicates that they had significantly lower mass-to-light ratios than those of present-day L* galaxies. This leads to a global stellar mass density, Omega_*(z), which rises with time from z=3 to the present. This measurement essentially traces the integral of the cosmic star formation history that has been the subject of previous investigations. 50-75% of the present-day stellar mass density had formed by z ~ 1, but at z ~= 2.7 we find only 3-14% of today's stars were present. This increase in Omega_* with time is broadly consistent with observations of the evolving global star formation rate once dust extinction is taken into account, but is steeper at 1<z<3 than predicted by some recent semi-analytic models of galaxy formation. The observations appear to be inconsistent with scenarios in which the bulk of stars in present-day galactic spheroids formed at z>>2.

The Early Data Release from the Sloan Digital Sky survey provides one of the largest multicolor photometric catalogs currently available to the astronomical community. In this paper we present the first application of photometric redshifts to the $\sim 6$ million extended sources within these data (with 1.8 million sources having $r' < 21$). Utilizing a range of photometric redshift techniques, from empirical to template and hybrid techniques, we investigate the statistical and systematic uncertainties present within the redshift estimates for the EDR data. For $r'<21$ we find that the redshift estimates provide realistic redshift histograms with an rms uncertainty in the photometric redshift relation of 0.035 at $r'<18$ and rising to 0.1 at $r'<21$. We conclude by describing how these photometric redshifts and derived quantities, such as spectral type, restframe colors and absolute magnitudes, are stored within the SDSS database. We provide sample queries for searching on photometric redshifts and list the current caveats and issues that should be understood before using these photometric redshifts in statistical analyses of the SDSS galaxies.

Early photometric data from the Sloan Digital Sky Survey (SDSS) contain angular positions for 1.5 million galaxies. In companion papers, the angular correlation function $w(\theta)$ and 2D power spectrum $C_l$ of these galaxies are presented. Here we invert Limber's equation to extract the 3D power spectrum from the angular results. We accomplish this using an estimate of $dn/dz$, the redshift distribution of galaxies in four different magnitude slices in the SDSS photometric catalog. The resulting 3D power spectrum estimates from $w(\theta)$ and $C_l$ agree with each other and with previous estimates over a range in wavenumbers $0.03 < k/{\rm h Mpc}^{-1} < 1$. The galaxies in the faintest magnitude bin ($21 < \rstar < 22$, which have median redshift $z_m=0.43$) are less clustered than the galaxies in the brightest magnitude bin ($18 < \rstar < 19$ with $z_m=0.17$), especially on scales where nonlinearities are important. The derived power spectrum agrees with that of Szalay et al. (2001) who go directly from the raw data to a parametric estimate of the power spectrum. The strongest constraints on the shape parameter $\Gamma$ come from the faintest galaxies (in the magnitude bin $21 < \rstar < 22$), from which we infer $\Gamma = 0.14^{+0.11}_{-0.06}$ (95% C.L.).

We propose a novel type filter for multicolor imaging to improve on the photometric redshift estimation of galaxies. An extra filter - specific to a certain photometric system - may be utilized with high efficiency. We present a case study of the Hubble Space Telescope's Advanced Camera for Surveys and show that one extra exposure could cut down the mean square error on photometric redshifts by 34% over the z<1.3 redshift range.

From SDSS commissioning photometric and spectroscopic data, we investigate the utility of photometric redshift techniques to the task of estimating QSO redshifts. We consider empirical methods (e.g. nearest-neighbor searches and polynomial fitting), standard spectral template fitting and hybrid approaches (i.e. training spectral templates from spectroscopic and photometric observations of QSOs). We find that in all cases, due to the presence of strong emission-lines within the QSO spectra, the nearest-neighbor and template fitting methods are superior to the polynomial fitting approach. Applying a novel reconstruction technique, we can, from the SDSS multicolor photometry, reconstruct a statistical representation of the underlying SEDs of the SDSS QSOs. Although, the reconstructed templates are based on only broadband photometry the common emission lines present within the QSO spectra can be recovered in the resulting spectral energy distributions. The technique should be useful in searching for spectral differences among QSOs at a given redshift, in searching for spectral evolution of QSOs, in comparing photometric redshifts for objects beyond the SDSS spectroscopic sample with those in the well calibrated photometric redshifts for objects brighter than 20th magnitude and in searching for systematic and time variable effects in the SDSS broad band photometric and spectral photometric calibrations.

We present photometric observations of an apparent Type Ia supernova (SN Ia) at a redshift of ~1.7, the farthest SN observed to date. SN 1997ff, was discovered in a repeat observation by the HST of the HDF-), and serendipitously monitored with NICMOS on HST throughout the GTO campaign. The SN type can be determined from the host galaxy type:an evolved, red elliptical lacking enough recent star formation to provide a significant population of core-collapse SNe. The class- ification is further supported by diagnostics available from the observed colors and temporal behavior of the SN, both of which match a typical SN Ia. The photo- metric record of the SN includes a dozen flux measurements in the I, J, and H bands spanning 35 days in the observed frame. The redshift derived from the SN photometry, z=1.7+/-0.1, is in excellent agreement with the redshift estimate of z=1.65+/-0.15 derived from the U_300,B_450,V_606,I_814,J_110,J_125,H_160, H_165,K_s photometry of the galaxy. Optical and near-infrared spectra of the host provide a very tentative spectroscopic redshift of 1.755. Fits to observations of the SN provide constraints for the redshift-distance relation of SNe~Ia and a powerful test of the current accelerating Universe hypothesis. The apparent SN brightness is consistent with that expected in the decelerating phase of the preferred cosmological model, Omega_M~1/3, Omega_Lambda~2/3. It is inconsistent with grey dust or simple luminosity evolution, candidate astro- physical effects which could mimic past evidence for an accelerating Universe from SNe Ia at z~0.5.We consider several sources of possible systematic error including lensing, SN misclassification, selection bias, and calibration errors. Currently, none of these effects appears likely to challenge our conclusions.

Understanding how the physical properties of galaxies (e.g. their spectral type or age) evolve as a function of redshift relies on having an accurate representation of galaxy spectral energy distributions. While it has been known for some time that galaxy spectra can be reconstructed from a handful of orthogonal basis templates, the underlying basis is poorly constrained. The limiting factor has been the lack of large samples of galaxies (covering a wide range in spectral type) with high signal-to-noise spectrophotometric observations. To alleviate this problem we introduce here a new technique for reconstructing galaxy spectral energy distributions directly from samples of galaxies with broadband photometric data and spectroscopic redshifts. Exploiting the statistical approach of the Karhunen-Loeve expansion, our iterative training procedure increasingly improves the eigenbasis, so that it provides better agreement with the photometry. We demonstrate the utility of this approach by applying these improved spectral energy distributions to the estimation of photometric redshifts for the HDF sample of galaxies. We find that in a small number of iterations the dispersion in the photometric redshifts estimator (a comparison between predicted and measured redshifts) can decrease by up to a factor of 2.

We present a novel approach to photometric redshifts, one that merges the advantages of both the template fitting and empirical fitting algorithms, without any of their disadvantages. This technique derives a set of templates, describing the spectral energy distributions of galaxies, from a catalog with both multicolor photometry and spectroscopic redshifts. The algorithm is essentially using the shapes of the templates as the fitting parameters. From simulated multicolor data we show that for a small training set of galaxies we can reconstruct robustly the underlying spectral energy distributions even in the presence of substantial errors in the photometric observations. We apply these techniques to the multicolor and spectroscopic observations of the Hubble Deep Field building a set of template spectra that reproduced the observed galaxy colors to better than 10%. Finally we demonstrate that these improved spectral energy distributions lead to a photometric-redshift relation for the Hubble Deep Field that is more accurate than standard template-based approaches.

We present a novel method capable of creating optimal eigenspectra from multicolor redshift surveys for photometric redshift estimation. Our iterative training algorithm modifies the templates to represent the photometric measurements better. We present a short description of our algorithm here. We show that the corrected templates give more precise photometric redshifts, essentially a ``free'' feature, since we were not fitting for the redshifts themselves.