We present the curation and verification of a new combined optical and near infrared dataset for cosmology and astrophysics, derived from the combination of $ugri$-band imaging from the Kilo Degree Survey (KiDS) and $ZY\!J\!H\!K_{\rm s}$-band imaging from the VISTA Kilo degree Infrared Galaxy (VIKING) survey. This dataset is unrivaled in cosmological imaging surveys due to its combination of area ($458$ deg$^2$ before masking), depth ($r\le25$), and wavelength coverage ($ugriZY\!J\!H\!K_{\rm s}$). The combination of survey depth, area, and (most importantly) wavelength coverage allows significant reductions in systematic uncertainties (i.e. reductions of between 10 and 60\% in bias, outlier rate, and scatter) in photometric-to-spectroscopic redshift comparisons, compared to the optical-only case at photo-$z$ above $0.7$. The complementarity between our optical and NIR surveys means that over $80\%$ of our sources, across all photo-$z$, have significant detections (i.e. not upper limits) in our $8$ reddest bands. We derive photometry, photo-$z$, and stellar masses for all sources in the survey, and verify these data products against existing spectroscopic galaxy samples. We demonstrate the fidelity of our higher-level data products by constructing the survey stellar mass functions in 8 volume-complete redshift bins. We find that these photometrically derived mass functions provide excellent agreement with previous mass evolution studies derived using spectroscopic surveys. The primary data products presented in this paper are publicly available at http://kids.strw.leidenuniv.nl/.

This paper defines the UK Infra-Red Telescope (UKIRT) Hemisphere Survey (UHS) and release of the remaining ~12,700 sq.deg of J-band survey data products. The UHS will provide continuous J and K-band coverage in the northern hemisphere from a declination of 0 deg to 60 deg by combining the existing Large Area Survey, Galactic Plane Survey and Galactic Clusters Survey conducted under the UKIRT Infra-red Deep Sky Survey (UKIDSS) programme with this new additional area not covered by UKIDSS. The released data includes J-band imaging and source catalogues over the new area, which, together with UKIDSS, completes the J-band UHS coverage over the full ~17,900 sq.deg area. 98 per cent of the data in this release have passed quality control criteria, the remaining 2 per cent being scheduled for re-observation. The median 5-sigma point source sensitivity of the released data is 19.6 mag (Vega). The median full width at half-maximum of the point spread function across the dataset is 0.75 arcsec. In this paper, we outline the survey management, data acquisition, processing and calibration, quality control and archiving as well as summarising the characteristics of the released data products. The data are initially available to a limited consortium with a world-wide release scheduled for August 2018.

The VLT Survey Telescope (VST) ATLAS is an optical ugriz survey aiming to cover ~4700deg^2 of the Southern sky to similar depths as the Sloan Digital Sky Survey (SDSS). From reduced images and object catalogues provided by the Cambridge Astronomical Surveys Unit we first find that the median seeing ranges from 0.8 arcsec FWHM in i to 1.0 arcsec in u, significantly better than the 1.2-1.5 arcsec seeing for SDSS. The 5 sigma magnitude limit for stellar sources is r_AB=22.7 and in all bands these limits are at least as faint as SDSS. SDSS and ATLAS are more equivalent for galaxy photometry except in the z band where ATLAS has significantly higher throughput. We have improved the original ESO magnitude zeropoints by comparing m<16 star magnitudes with APASS in gri, also extrapolating into u and z, resulting in zeropoints accurate to ~+-0.02 mag. We finally compare star and galaxy number counts in a 250deg^2 area with SDSS and other count data and find good agreement. ATLAS data products can be retrieved from the ESO Science Archive, while support for survey science analyses is provided by the OmegaCAM Science Archive (OSA), operated by the Wide-Field Astronomy Unit in Edinburgh.

We describe the VISTA Science Archive (VSA) and its first public release of data from five of the six VISTA Public Surveys. The VSA exists to support the VISTA Surveys through their lifecycle: the VISTA Public Survey consortia can use it during their quality control assessment of survey data products before submission to the ESO Science Archive Facility (ESO SAF); it supports their exploitation of survey data prior to its publication through the ESO SAF; and, subsequently, it provides the wider community with survey science exploitation tools that complement the data product repository functionality of the ESO SAF. This paper has been written in conjunction with the first public release of public survey data through the VSA and is designed to help its users understand the data products available and how the functionality of the VSA supports their varied science goals. We describe the design of the database and outline the database-driven curation processes that take data from nightly pipeline-processed and calibrated FITS files to create science-ready survey datasets. Much of this design, and the codebase implementing it, derives from our earlier WFCAM Science Archive (WSA), so this paper concentrates on the VISTA-specific aspects and on improvements made to the system in the light of experience gained in operating the WSA.

The new VISual and Infrared Telescope for Astronomy (VISTA) has started operations. Over its first five years it will be collecting data for six public surveys, one of these is the near-infrared YJKsVISTA survey of the Magellanic Clouds system (VMC). This survey comprises the LMC, the SMC, the Bridge connecting the two galaxies and two fields in the Stream. This paper provides an overview of the VMC survey strategy and presents first science results. The main goals of the VMC survey are the determination of the spatially resolved SFH and 3D structure of the Magellanic system. Therefore, the VMC survey is designed to reach stars as faint as the oldest main sequence turn-off point and to constrain the mean magnitude of pulsating variable stars such as RR Lyrae stars and Cepheids. This paper focuses on observations of VMC fields in the LMC obtained between November 2009 and March 2010. These observations correspond to a 7% completeness of the LMC fields. The VMC observations consist of multi-epoch measurements organised following a specific structure. The data were reduced using the VISTA Data Flow System pipeline whose source catalogues were produced and made available via the VISTA Science Archive. The analysis of the data shows that the sensitivity in each wave band agrees with expectations. Uncertainties and completeness of the data are also derived. The first science results, aimed at assessing the science quality of the VMC data, include an overview of the distribution of stars in colour-magnitude and colour-colour diagrams, the detection of planetary nebulae and of stellar clusters, and the Ks band light-curve of variable stars. The VMC survey represents a tremendous improvement, in spatial resolution and sensitivity, on previous panoramic observations of the Magellanic system in the near-infrared and complements nicely the deep observations at other wavelengths. (Abridged)

The two fastest near infrared survey telescopes are UKIRT-WFCAM and VISTA. The data from both these instruments are being archived by Wide Field Astronomy Unit (WFAU) at the IfA, Edinburgh, using the same curation pipeline, with some instrument specific processing. The final catalogues from these surveys will contain many tens of billions of detections. Data are taken for a range of large surveys and smaller PI programmes. The surveys vary from shallow hemisphere surveys to ultra deep single pointings with hundreds of individual epochs, each with a wide range of scientific goals, leading to a wide range of products and database tables being created. Processing of the main surveys must allow for the inclusion of specific high-level requirements from the survey teams, but automation reduces the amount of work by archive operators allowing a higher curation efficiency. The decision making processes which drive the curation pipeline are a crucial element for efficient archiving. This paper describes the main issues involved in automating the pipeline.

We present a description of the design and usage of a new synoptic pipeline and database model for time series photometry in the VISTA Data Flow System (VDFS). All UKIRT-WFCAM data and most of the VISTA main survey data will be processed and archived by the VDFS. Much of these data are multi-epoch, useful for finding moving and variable objects. Our new database design allows the users to easily find rare objects of these types amongst the huge volume of data being produced by modern survey telescopes. Its effectiveness is demonstrated through examples using Data Release 5 of the UKIDSS Deep Extragalactic Survey (DXS) and the WFCAM standard star data. The synoptic pipeline provides additional quality control and calibration to these data in the process of generating accurate light-curves. We find that 0.6+-0.1% of stars and 2.3+-0.6% of galaxies in the UKIDSS-DXS with K<15 mag are variable with amplitudes \Delta K>0.015 mag

The VISTA Data Flow System comprises nightly pipeline and archiving of near infrared data from UKIRT-WFCAM and VISTA. This includes multi-epoch data which can be used to find moving and variable objects. We have developed a new model for archiving these data which gives the user an extremely flexible and reliable data set that is easy to query through an SQL interface. We have introduced several new database tables into our schema for deep/synoptic datasets. We have also developed a set of curation procedures, which give additional quality control and automation. We discuss the methods used and show some example data. Our design is particularly effective on correlated data-sets, where the observations in different filters are synchronised. It is scalable to large VISTA datasets which will be observed in the next few years and to future surveys such as Pan-STARRS and LSST.

The UKIDSS Galactic Plane Survey (GPS) is one of the five near infrared Public Legacy Surveys that are being undertaken by the UKIDSS consortium, using the Wide Field Camera on the United Kingdom Infrared Telescope. It is surveying 1868 sq.deg. of the northern and equatorial Galactic plane at Galactic latitudes -5<b<5 in the J, H and K filters and a ~200 sq.deg. area of the Taurus-Auriga-Perseus molecular cloud complex in these three filters and the 2.12 um (1-0) H_2 filter. It will provide data on ~2 billion sources. Here we describe the properties of the dataset and provide a user's guide for its exploitation. We also present brief Demonstration Science results from DR2 and from the Science Verification programme. These results illustrate how GPS data will frequently be combined with data taken in other wavebands to produce scientific results. The Demonstration Science includes studies of: (i) the star formation region G28.983-0.603, cross matching with Spitzer-GLIMPSE data to identify YSOs; (ii) the M17 nebula; (iii) H_2 emission in the rho Ophiuchi dark cloud; (iv) X-ray sources in the Galactic Centre; (v) external galaxies in the Zone of Avoidance; (vi) IPHAS-GPS optical-infrared spectrophotometric typing. (abridged).

We describe the WFCAM Science Archive (WSA), which is the primary point of access for users of data from the wide-field infrared camera WFCAM on the United Kingdom Infrared Telescope (UKIRT), especially science catalogue products from the UKIRT Infrared Deep Sky Survey (UKIDSS). We describe the database design with emphasis on those aspects of the system that enable users to fully exploit the survey datasets in a variety of different ways. We give details of the database-driven curation applications that take data from the standard nightly pipeline-processed and calibrated files for the production of science-ready survey datasets. We describe the fundamentals of querying relational databases with a set of astronomy usage examples, and illustrate the results.

One of the major challenges providing large databases like the WFCAM Science Archive (WSA) is to minimize ingest times for pixel/image metadata and catalogue data. In this article we describe how the pipeline processed data are ingested into the database as the first stage in building a release database which will be succeeded by advanced processing (source merging, seaming, detection quality flagging etc.). To accomplish the ingestion procedure as fast as possible we use a mixed Python/C++ environment and run the required tasks in a simple parallel modus operandi where the data are split into daily chunks and then processed on different computers. The created data files can be ingested into the database immediately as they are available. This flexible way of handling the data allows the most usage of the available CPUs as the comparison with sequential processing shows.

The VDFS comprises the system to pipeline process and archive the data from infrared observations taken by both the WFCAM instrument on UKIRT and the forthcoming VISTA telescope. These include the largest near-IR surveys to date, such as UKIDSS, which produce terabyte sized catalogues of over 10^9 rows. Such large data volumes present a performance challenge when the catalogue data, stored in a relational database, require many iterations of astrometric and photometric recalibration. Here we present the VDFS recalibration solution that will be employed in the WSA from the forthcoming UKIDSS Data Release 4 and VSA from its inception.

The First Data Release (DR1) of the UKIRT Infrared Deep Sky Survey (UKIDSS) took place on 2006 July 21. UKIDSS is a set of five large near-infrared surveys, covering a complementary range of areas, depths, and Galactic latitudes. DR1 is the first large release of survey-quality data from UKIDSS and includes 320 sq degs of multicolour data to (Vega) K=18, complete (depending on the survey) in three to five bands from the set ZYJHK, together with 4 sq degs of deep JK data to an average depth K=21. In addition the release includes a similar quantity of data with incomplete filter coverage. In JHK, in regions of low extinction, the photometric uniformity of the calibration is better than 0.02 mag. in each band. The accuracy of the calibration in ZY remains to be quantified, and the same is true of JHK in regions of high extinction. The median image FWHM across the dataset is 0.82 arcsec. We describe changes since the Early Data Release in the implementation, pipeline and calibration, quality control, and archive procedures. We provide maps of the areas surveyed, and summarise the contents of each of the five surveys in terms of filters, areas, and depths. DR1 marks completion of 7 per cent of the UKIDSS 7-year goals.

The UKIRT Infrared Deep Sky Survey (UKIDSS) is a set of five large near-infrared surveys, covering a complementary range of areas, depths, and Galactic latitudes. The UKIDSS Second Data Release (DR2) includes the First Data Release (DR1), with minor improvements, plus new data for the LAS, GPS, GCS, and DXS, from observations made over 2006 May through July (when the UDS was unobservable). DR2 was staged in two parts. The first part excluded the GPS, and took place on 2007 March 1. The GPS was released on 2007 April 12. DR2 includes 282 sq. degs of multicolour data to (Vega) K=18, complete in the full YJHK set for the LAS, 57 sq. degs in the ZYJHK set for the GCS, and 236 sq. degs in the JHK set for the GPS. DR2 includes nearly 7 sq. degs of deep JK data (DXS, UDS) to an average depth K=21. In addition the release includes a comparable quantity of data where coverage of the filter set for any survey is incomplete. We document changes that have occurred since DR1 to the pipeline, calibration, and archive procedures. The two most noteworthy changes are presentation of the data in a single database (compared to two previously), and provision of additional error flags for detected sources, flagging potentially spurious artifacts, corrupted data and suspected cross-talk sources. We summarise the contents of each of the surveys in terms of filters, areas, and depths.

We present high-resolution imaging and low-resolution spectroscopy of the BL Lac object PKS 0537-441 (z = 0.893) and its environment. The observations were designed to clarify, whether the properties of PKS 0537-441 are affected by gravitational microlensing, or whether PKS 0537-441 and its environment act as a lensing system itself. Our observations show that neither case seems to be likely. We did not find a galaxy along the line-of-sight to the BL Lac as claimed previously, our spectroscopy shows that none of the four closest companion galaxies is at high redshift. Two of the four nearby companion galaxies to PKS 0537-441 are within 200 km/s of the systemic velocity of the BL Lac (z = 0.892 and 0.895, respectively). The third companion galaxy is at higher redshift (z = 0.947). The fourth companion galaxy shows evidence of Mg II absorption redwards of its systemic velocity and is perhaps a mini low ionization BAL QSO at z = 0.885. If the latter can be confirmed, PKS 0537-441 is the first BL Lacertae object being a member of a binary Quasar. We also detected extended [O II] emission in the off-nuclear spectrum of PKS 05371-441, which is most likely due to photoionization from the active nucleus. Alternatively, the extended [O II] emission is due to jet-cloud interaction with the counterjet of PKS 0537-441. Our clustering analysis indicates that PKS 0537-441 is located in a cluster environment as rich as Abell type 0-1. This is supported by the detection of four galaxies in the field with similar redshifts as the BL Lac (Delta z < 0.002). We found serendipitously even more galaxies at somewhat higher redshifts (z = 0.9-1). Thus, PKS0537-441 might be located in front of a galaxy cluster at somewhat higher redshift or even be part of a large-scale structure with an extension towards the BL Lac.

The FORS Deep Field project is a multi-colour, multi-object spectroscopic investigation of an approx. 7 times 7 region near the south galactic pole based mostly on observations carried out with the FORS instruments attached to the VLT telescopes. It includes the QSO Q 0103-260 (z = 3.36). The goal of this study is to improve our understanding of the formation and evolution of galaxies in the young Universe. In this paper the field selection, the photometric observations, and the data reduction are described. The source detection and photometry of objects in the FORS Deep Field is discussed in detail. A combined B and I selected UBgRIJKs photometric catalog of 8753 objects in the FDF is presented and its properties are briefly discussed. The formal 50% completeness limits for point sources, derived from the co-added images, are 25.64, 27.69, 26.86, 26.68, 26.37, 23.60 and 21.57 in U, B, g, R, I, J and Ks (Vega-system), respectively. A comparison of the number counts in the FORS Deep Field to those derived in other deep field surveys shows very good agreement.

We present the B-band Tully-Fisher relation (TFR) of 60 late-type galaxies with redshifts 0.1-1. The galaxies were selected from the FORS Deep Field with a limiting magnitude of R=23. Spatially resolved rotation curves were derived from spectra obtained with FORS2 at the VLT. High-mass galaxies with v_max>=150 km/s show little evolution, whereas the least massive systems in our sample are brighter by about 1-2 mag compared to their local counterparts. For the entire distant sample, the TFR slope is flatter than for local field galaxies (-5.77+-0.45 versus -7.92+-0.18). Thus, we find evidence for evolution of the slope of the TFR with redshift on the 3-sigma level. This is still true when we subdivide the sample into three redshift bins. We speculate that the flatter tilt of our sample is caused by the evolution of luminosities and an additional population of blue galaxies at z>=0.2. The mass dependence of the TFR evolution also leads to variations for different galaxy types in magnitude-limited samples, suggesting that selection effects can account for the discrepant results of previous TFR studies on the luminosity evolution of late-type galaxies.

Dedicating a major fraction of its guaranteed time, the FORS consortium established a FORS Deep Field which contains a known QSO at z = 3.36. It was imaged in UBgRIz with FORS at the VLT as well as in J and Ks with the NTT. Covering an area 6-8 times larger as the HDFs but with similar depth in the optical it is one of the largest deep fields up to date to investigate i) galaxy evolution in the field from present up to z $\sim$ 5, ii) the galaxy distribution in the line of sight to the QSO, iii) the high-z QSO environment and iv) the galaxy-galaxy lensing signal in such a large field. In this presentation a status report of the FORS Deep Field project is given. In particular, the field selection, the imaging results (number counts, photometric redshifts etc.) and the first spectroscopic results are presented.