Astrometric calibration of images with a small field of view is often inferior to the internal accuracy of the source detections due to the small number of accessible guide stars. One important experiment with such challenges is the Hubble Space Telescope (HST). A possible solution is to cross-calibrate overlapping fields instead of just relying on standard stars. Following the approach of \citet{2012ApJ...761..188B}, we use infinitesimal 3D rotations for fine-tuning the calibration but devise a better objective that is robust to a large number of false candidates in the initial set of associations. Using Bayesian statistics, we accommodate bad data by explicitly modeling the quality, which yields a formalism essentially identical to an $M$-estimation in robust statistics. Our results on simulated and real catalogs show great potentials for improving the HST calibration, and those with similar challenges.

SPIDERS (SPectroscopic IDentification of eROSITA Sources) is an SDSS-IV survey running in parallel to the eBOSS cosmology project. SPIDERS will obtain optical spectroscopy for large numbers of X-ray-selected AGN and galaxy cluster members detected in wide area eROSITA, XMM-Newton and ROSAT surveys. We describe the methods used to choose spectroscopic targets for two sub-programmes of SPIDERS: X-ray selected AGN candidates detected in the ROSAT All Sky and the XMM-Newton Slew surveys. We have exploited a Bayesian cross-matching algorithm, guided by priors based on mid-IR colour-magnitude information from the WISE survey, to select the most probable optical counterpart to each X-ray detection. We empirically demonstrate the high fidelity of our counterpart selection method using a reference sample of bright well-localised X-ray sources collated from XMM-Newton, Chandra and Swift-XRT serendipitous catalogues, and also by examining blank-sky locations. We describe the down-selection steps which resulted in the final set of SPIDERS-AGN targets put forward for spectroscopy within the eBOSS/TDSS/SPIDERS survey, and present catalogues of these targets. We also present catalogues of ~12000 ROSAT and ~1500 XMM-Newton Slew survey sources which have existing optical spectroscopy from SDSS-DR12, including the results of our visual inspections. On completion of the SPIDERS program, we expect to have collected homogeneous spectroscopic redshift information over a footprint of ~7500 deg$^2$ for >85 percent of the ROSAT and XMM-Newton Slew survey sources having optical counterparts in the magnitude range 17<r<22.5, producing a large and highly complete sample of bright X-ray-selected AGN suitable for statistical studies of AGN evolution and clustering.

We present a flexible template-based photometric redshift estimation framework, implemented in C#, that can be seamlessly integrated into a SQL database (or DB) server and executed on-demand in SQL. The DB integration eliminates the need to move large photometric datasets outside a database for redshift estimation, and utilizes the computational capabilities of DB hardware. The code is able to perform both maximum likelihood and Bayesian estimation, and can handle inputs of variable photometric filter sets and corresponding broad-band magnitudes. It is possible to take into account the full covariance matrix between filters, and filter zero points can be empirically calibrated using measurements with given redshifts. The list of spectral templates and the prior can be specified flexibly, and the expensive synthetic magnitude computations are done via lazy evaluation, coupled with a caching of results. Parallel execution is fully supported. For large upcoming photometric surveys such as the LSST, the ability to perform in-place photo-z calculation would be a significant advantage. Also, the efficient handling of variable filter sets is a necessity for heterogeneous databases, for example the Hubble Source Catalog, and for cross-match services such as SkyQuery. We illustrate the performance of our code on two reference photo-z estimation testing datasets, and provide an analysis of execution time and scalability with respect to different configurations. The code is available for download at https://github.com/beckrob/Photo-z-SQL.

We apply a novel spectral graph technique, that of locally-biased semi-supervised eigenvectors, to study the diversity of galaxies. This technique permits us to characterize empirically the natural variations in observed spectra data, and we illustrate how this approach can be used in an exploratory manner to highlight both large-scale global as well as small-scale local structure in Sloan Digital Sky Survey (SDSS) data. We use this method in a way that simultaneously takes into account the measurements of spectral lines as well as the continuum shape. Unlike Principal Component Analysis, this method does not assume that the Euclidean distance between galaxy spectra is a good global measure of similarity between all spectra, but instead it only assumes that local difference information between similar spectra is reliable. Moreover, unlike other nonlinear dimensionality methods, this method can be used to characterize very finely both small-scale local as well as large-scale global properties of realistic noisy data. The power of the method is demonstrated on the SDSS Main Galaxy Sample by illustrating that the derived embeddings of spectra carry an unprecedented amount of information. By using a straightforward global or unsupervised variant, we observe that the main features correlate strongly with star formation rate and that they clearly separate active galactic nuclei. Computed parameters of the method can be used to describe line strengths and their interdependencies. By using a locally-biased or semi-supervised variant, we are able to focus on typical variations around specific objects of astronomical interest. We present several examples illustrating that this approach can enable new discoveries in the data as well as a detailed understanding of very fine local structure that would otherwise be overwhelmed by large-scale noise and global trends in the data.

We propose a new method of constraining the redshifts of individual extragalactic sources based on celestial coordinates and their ensemble statistics. Techniques from integer linear programming are utilized to optimize simultaneously for the angular two-point cross- and autocorrelation functions. Our novel formalism introduced here not only transforms the otherwise hopelessly expensive, brute-force combinatorial search into a linear system with integer constraints but also is readily implementable in off-the-shelf solvers. We adopt Gurobi, a commercial optimization solver, and use Python to build the cost function dynamically. The preliminary results on simulated data show potential for future applications to sky surveys by complementing and enhancing photometric redshift estimators. Our approach is the first application of integer linear programming to astronomical analysis.

Data from the Herschel Space Observatory is freely available to the public but no uniformly processed catalogue of the observations has been published so far. To date, the Herschel Science Archive does not contain the exact sky coverage (footprint) of individual observations and supports search for measurements based on bounding circles only. Drawing on previous experience in implementing footprint databases, we built the Herschel Footprint Database and Web Services for the Herschel Space Observatory to provide efficient search capabilities for typical astronomical queries. The database was designed with the following main goals in mind: (a) provide a unified data model for meta-data of all instruments and observational modes, (b) quickly find observations covering a selected object and its neighbourhood, (c) quickly find every observation in a larger area of the sky, (d) allow for finding solar system objects crossing observation fields. As a first step, we developed a unified data model of observations of all three Herschel instruments for all pointing and instrument modes. Then, using telescope pointing information and observational meta-data, we compiled a database of footprints. As opposed to methods using pixellation of the sphere, we represent sky coverage in an exact geometric form allowing for precise area calculations. For easier handling of Herschel observation footprints with rather complex shapes, two algorithms were implemented to reduce the outline. Furthermore, a new visualisation tool to plot footprints with various spherical projections was developed. Indexing of the footprints using Hierarchical Triangular Mesh makes it possible to quickly find observations based on sky coverage, time and meta-data. The database is accessible via a web site (http://herschel.vo.elte.hu) and also as a set of REST web service functions.

We apply clustering-based redshift inference to all extended sources from the Sloan Digital Sky Survey photometric catalogue, down to magnitude r = 22. We map the relationships between colours and redshift, without assumption of the sources' spectral energy distributions (SED). We identify and locate star-forming, quiescent galaxies, and AGN, as well as colour changes due to spectral features, such as the 4000 \AA{} break, redshifting through specific filters. Our mapping is globally in good agreement with colour-redshift tracks computed with SED templates, but reveals informative differences, such as the need for a lower fraction of M-type stars in certain templates. We compare our clustering-redshift estimates to photometric redshifts and find these two independent estimators to be in good agreement at each limiting magnitude considered. Finally, we present the global clustering-redshift distribution of all Sloan extended sources, showing objects up to z ~ 0.8. While the overall shape agrees with that inferred from photometric redshifts, the clustering redshift technique results in a smoother distribution, with no indication of structure in redshift space suggested by the photometric redshift estimates (likely artifacts imprinted by their spectroscopic training set). We also infer a higher fraction of high redshift objects. The mapping between the four observed colours and redshift can be used to estimate the redshift probability distribution function of individual galaxies. This work is an initial step towards producing a general mapping between redshift and all available observables in the photometric space, including brightness, size, concentration, and ellipticity.

We present the methodology and data behind the photometric redshift database of the Sloan Digital Sky Survey Data Release 12 (SDSS DR12). We adopt a hybrid technique, empirically estimating the redshift via local regression on a spectroscopic training set, then fitting a spectrum template to obtain K-corrections and absolute magnitudes. The SDSS spectroscopic catalog was augmented with data from other, publicly available spectroscopic surveys to mitigate target selection effects. The training set is comprised of $1,976,978$ galaxies, and extends up to redshift $z\approx 0.8$, with a useful coverage of up to $z\approx 0.6$. We provide photometric redshifts and realistic error estimates for the $208,474,076$ galaxies of the SDSS primary photometric catalog. We achieve an average bias of $\overline{\Delta z_{\mathrm{norm}}} = 5.84 \times 10^{-5}$, a standard deviation of $\sigma \left(\Delta z_{\mathrm{norm}}\right)=0.0205$, and a $3\sigma$ outlier rate of $P_o=4.11\%$ when cross-validating on our training set. The published redshift error estimates and photometric error classes enable the selection of galaxies with high quality photometric redshifts. We also provide a supplementary error map that allows additional, sophisticated filtering of the data.

We created the Herschel Footprint Database and web services for the Herschel Space Observatory imaging data. For this database we set up a unified data model for the PACS and SPIRE Herschel instruments, from the pointing and header information of each observation, generated and stored sky coverages (footprints) of the observations in their exact geometric form. With this tool we extend the capabilities of the Herschel Science Archive by providing an effective search tool that is able to find observations for selected sky locations (objects), or even in larger areas in the sky.

Crossmatching catalogs at different wavelengths is a difficult problem in astronomy, especially when the objects are not point-like. At radio wavelengths an object can have several components corresponding, for example, to a core and lobes. {Considering not all radio detections correspond to visible or infrared sources, matching these catalogs can be challenging.} Traditionally this is done by eye for better quality, which does not scale to the large data volumes expected from the next-generation of radio telescopes. We present a novel automated procedure, using Bayesian hypothesis testing, to achieve reliable associations by explicit modelling of a particular class of radio-source morphology. {The new algorithm not only assesses the likelihood of an association between data at two different wavelengths, but also tries to assess whether different radio sources are physically associated, are double-lobed radio galaxies, or just distinct nearby objects.} Application to the SWIRE and ATLAS CDF-S catalogs shows that this method performs well without human intervention.

We present photometric redshifts and associated probability distributions for all detected sources in the Extended Chandra Deep Field South (ECDFS). The work makes use of the most up-to-date data from the Cosmic Assembly Near-IR Deep Legacy Survey (CANDELS) and the Taiwan ECDFS Near-Infrared Survey (TENIS) in addition to other data. We also revisit multi-wavelength counterparts for published X-ray sources from the 4Ms-CDFS and 250ks-ECDFS surveys, finding reliable counterparts for 1207 out of 1259 sources ($\sim 96\%$). Data used for photometric redshifts include intermediate-band photometry deblended using the TFIT method, which is used for the first time in this work. Photometric redshifts for X-ray source counterparts are based on a new library of AGN/galaxy hybrid templates appropriate for the faint X-ray population in the CDFS. Photometric redshift accuracy for normal galaxies is 0.010 and for X-ray sources is 0.014, and outlier fractions are $4\%$ and $5.4\%$ respectively. The results within the CANDELS coverage area are even better as demonstrated both by spectroscopic comparison and by galaxy-pair statistics. Intermediate-band photometry, even if shallow, is valuable when combined with deep broad-band photometry. For best accuracy, templates must include emission lines.

Not only source catalogs are extracted from astronomy observations. Their sky coverage is always carefully recorded and used in statistical analyses, such as correlation and luminosity function studies. Here we present a novel method for catalog matching, which inherently builds on the coverage information for better performance and completeness. A modified version of the Zones Algorithm is introduced for matching partially overlapping observations, where irrelevant parts of the data are excluded up front for efficiency. Our design enables searches to focus on specific areas on the sky to further speed up the process. Another important advantage of the new method over traditional techniques is its ability to quickly detect dropouts, i.e., the missing components that are in the observed regions of the celestial sphere but did not reach the detection limit in some observations. These often provide invaluable insight into the spectral energy distribution of the matched sources but rarely available in traditional associations.

The fraction of galaxies with red colours depends sensitively on environment, and on the way in which environment is measured. To distinguish competing theories for the quenching of star formation, a robust and complete description of environment is required, to be applied to a large sample of galaxies. The environment of galaxies can be described using the density field of neighbours on multiple scales - the multiscale density field. We are using the Millennium simulation and a simple HOD prescription which describes the multiscale density field of Sloan Digital Sky Survey DR7 galaxies to investigate the dependence of the fraction of red galaxies on the environment. Using a volume limited sample where we have sufficient galaxies in narrow density bins, we have more dynamic range in halo mass and density for satellite galaxies than for central galaxies. Therefore we model the red fraction of central galaxies as a constant while we use a functional form to describe the red fraction of satellites as a function of halo mass which allows us to distinguish a sharp from a gradual transition. While it is clear that the data can only be explained by a gradual transition, an analysis of the multiscale density field on different scales suggests that colour segregation within the haloes is needed to explain the results. We also rule out a sharp transition for central galaxies, within the halo mass range sampled.

We introduce \emph{Astronomy and Computing}, a new journal for the growing population of people working in the domain where astronomy overlaps with computer science and information technology. The journal aims to provide a new communication channel within that community, which is not well served by current journals, and to help secure recognition of its true importance within modern astronomy. In this inaugural editorial, we describe the rationale for creating the journal, outline its scope and ambitions, and seek input from the community in defining in detail how the journal should work towards its high-level goals.

Multi-wavelength astronomical studies require cross-identification of detections of the same celestial objects in multiple catalogs based on spherical coordinates and other properties. Because of the large data volumes and spherical geometry, the symmetric N-way association of astronomical detections is a computationally intensive problem, even when sophisticated indexing schemes are used to exclude obviously false candidates. Legacy astronomical catalogs already contain detections of more than a hundred million objects while the ongoing and future surveys will produce catalogs of billions of objects with multiple detections of each at different times. The varying statistical error of position measurements, moving and extended objects, and other physical properties make it necessary to perform the cross-identification using a mathematically correct, proper Bayesian probabilistic algorithm, capable of including various priors. One time, pair-wise cross-identification of these large catalogs is not sufficient for many astronomical scenarios. Consequently, a novel system is necessary that can cross-identify multiple catalogs on-demand, efficiently and reliably. In this paper, we present our solution based on a cluster of commodity servers and ordinary relational databases. The cross-identification problems are formulated in a language based on SQL, but extended with special clauses. These special queries are partitioned spatially by coordinate ranges and compiled into a complex workflow of ordinary SQL queries. Workflows are then executed in a parallel framework using a cluster of servers hosting identical mirrors of the same data sets.

In this work we present an atlas of composite spectra of galaxies based on the data of the Sloan Digital Sky Survey Data Release 7 (SDSS DR7). Galaxies are classified by colour, nuclear activity and star-formation activity to calculate average spectra of high signal-to-noise ratio and resolution (S/N = 132 - 4760 at {Dlambda = 1 A), using an algorithm that is robust against outliers. Besides composite spectra, we also compute the first five principal components of the distributions in each galaxy class to characterize the nature of variations of individual spectra around the averages. The continua of the composite spectra are fitted with BC03 stellar population synthesis models to extend the wavelength coverage beyond the coverage of the SDSS spectrographs. Common derived parameters of the composites are also calculated: integrated colours in the most popular filter systems, line strength measurements, and continuum absorption indices (including Lick indices). These derived parameters are compared with the distributions of parameters of individual galaxies and it is shown on many examples that the composites of the atlas cover much of the parameter space spanned by SDSS galaxies. By co-adding thousands of spectra, a total integration time of several months can be reached, which results in extremely low noise composites. The variations in redshift not only allow for extending the spectral coverage bluewards to the original wavelength limit of the SDSS spectrographs, but also make higher spectral resolution achievable. The composite spectrum atlas is available online at http://www.vo.elte.hu/compositeatlas.

We present a statistical study of the correlations and dimensionality of emission lines carried out on a sample of over 40,000 Sloan Digital Sky Survey (SDSS) galaxies. Using principal component analysis, we found that the equivalent widths of the 11 strongest lines can be well represented using three parameters. We also explore correlations of the emission pattern with the eigenspace representation of the continuum spectrum. The observed relations are used to provide an empirical prescription for expectation values and variances of emission-line strengths as a function of spectral shape. We show that this estimation of emission lines has a sufficient accuracy to make it suitable for photometric applications. The method has already proved useful in SDSS photometric redshift estimation.

We show that redshift-space distortions of galaxy correlations have a strong effect on correlation functions with distinct, localized features, like the signature of the baryon acoustic oscillations (BAO). Near the line of sight, the features become sharper as a result of redshift-space distortions. We demonstrate this effect by measuring the correlation function in Gaussian simulations and the Millennium Simulation. We also analyze the SDSS DR7 main-galaxy sample (MGS), splitting the sample into slices 2.5 degrees on the sky in various rotations. Measuring 2D correlation functions in each slice, we do see a sharp bump along the line of sight. Using Mexican-hat wavelets, we localize it to (110 +/- 10) Mpc/h. Averaging only along the line of sight, we estimate its significance at a particular wavelet scale and location at 2.2 sigma. In a flat angular weighting in the (pi,r_p) coordinate system, the noise level is suppressed, pushing the bump's significance to 4 sigma. We estimate that there is about a 0.2% chance of getting such a signal anywhere in the vicinity of the BAO scale from a power spectrum lacking a BAO feature. However, these estimates of the significances make some use of idealized Gaussian simulations, and thus are likely a bit optimistic.

The cross-identification of sources in separate catalogs is one of the most basic tasks in observational astronomy. It is, however, surprisingly difficult and generally ill-defined. Recently Budav\'ari & Szalay (2008) formulated the problem in the realm of probability theory, and laid down the statistical foundations of an extensible methodology. In this paper, we apply their Bayesian approach to stars that, we know, can move measurably on the sky, with detectable proper motion, and show how to associate their observations. We study models on a sample of stars in the Sloan Digital Sky Survey, which allow for an unknown proper motion per object, and demonstrate the improvements over the analytic static model. Our models and conclusions are directly applicable to upcoming surveys such as PanSTARRS, the Dark Energy Survey, Sky Mapper, and the LSST, whose data sets will contain hundreds of millions of stars observed multiple times over several years.

We introduce the GALEX Arecibo SDSS Survey (GASS), an on-going large program that is gathering high quality HI-line spectra using the Arecibo radio telescope for an unbiased sample of ~1000 galaxies with stellar masses greater than 10^10 Msun and redshifts 0.025<z<0.05, selected from the SDSS spectroscopic and GALEX imaging surveys. The galaxies are observed until detected or until a low gas mass fraction limit (1.5-5%) is reached. This paper presents the first Data Release, consisting of ~20% of the final GASS sample. We use this data set to explore the main scaling relations of HI gas fraction with galaxy structure and NUV-r colour. A large fraction (~60%) of the galaxies in our sample are detected in HI. We find that the atomic gas fraction decreases strongly with stellar mass, stellar surface mass density and NUV-r colour, but is only weakly correlated with galaxy bulge-to-disk ratio (as measured by the concentration index of the r-band light). We also find that the fraction of galaxies with significant (more than a few percent) HI decreases sharply above a characteristic stellar surface mass density of 10^8.5 Msun kpc^-2. The fraction of gas-rich galaxies decreases much more smoothly with stellar mass. One of the key goals of GASS is to identify and quantify the incidence of galaxies that are transitioning between the blue, star-forming cloud and the red sequence of passively-evolving galaxies. Likely transition candidates can be identified as outliers from the mean scaling relations between gas fraction and other galaxy properties. [abridged]

Given multiband photometric data from the SDSS DR6, we estimate galaxy redshifts. We employ a Random Forest trained on color features and spectroscopic redshifts from 80,000 randomly chosen primary galaxies yielding a mapping from color to redshift such that the difference between the estimate and the spectroscopic redshift is small. Our methodology results in tight RMS scatter in the estimates limited by photometric errors. Additionally, this approach yields an error distribution that is nearly Gaussian with parameter estimates giving reliable confidence intervals unique to each galaxy photometric redshift.

We present a catalog of spectro-photometric redshifts for 1308 galaxies from the GRism ACS Program for Extragalactic Science (GRAPES) observations with the Hubble Space Telescope. These low-resolution spectra between 6000 A and 9500 A are supplemented with U, J, H, and Ks from various facilities, resulting in redshifts computed with ~40 spectral bins per galaxy. For 81 galaxies between 0.5<z<1.5 with spectroscopic redshifts, the standard deviation in the fractional error in (1+z) is 0.046. With this catalog, we compute the B-band luminosity function in this redshift range from 72 galaxies. Owing to the depth of the GRAPES survey, we are able to accurately constrain the faint-end slope by going to M_B~-18 mag at 0.8<z<1.2, nearly two magnitudes fainter than previous studies. The faint-end slope is alpha=-1.32+-0.07. When compared to numerous published values at various redshifts, we find strong evidence for a steepening of the faint-end slope with redshift which is expected in the hierarchical formation scenario of galaxies.

We present two related techniques to measure the two-point correlation function and the power spectrum with edge correction in any spatial dimensions. The underlying algorithm uses fast Fourier transforms for calculating the two-point function with an heuristically weighted edge corrected estimator. Once the correlation function is measured, we estimate the power spectrum via numerical integration of the Hankel transform connecting the two. We introduce an efficient numerical technique based on Gauss-Bessel-quadrature and double exponential transformation. This, combined with our, or any similar, two-point function estimator leads to a novel edge corrected estimator for power spectra. The pair of techniques presented are the Euclidean analogs of those developed and widely used in cosmic microwave background research for spherical maps.

We present the results of a determination of the galaxy luminosity function at ultraviolet wavelengths at redshifts of $z=0.0-0.1$ from GALEX data. We determined the luminosity function in the GALEX FUV and NUV bands from a sample of galaxies with UV magnitudes between 17 and 20 that are drawn from a total of 56.73 deg^2 of GALEX fields overlapping the b_j-selected 2dF Galaxy Redshift Survey. The resulting luminosity functions are fainter than previous UV estimates and result in total UV luminosity densities of 10^(25.55+/-0.12) ergs s^-1 Hz^-1 Mpc^-3 and 10^(25.72+/-0.12) ergs s^-1 Hz^-1 Mpc^-3 at 1530 Ang. and 2310 Ang., respectively. This corresponds to a local star formation rate density in agreement with previous estimates made with H-alpha-selected data for reasonable assumptions about the UV extinction.