The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products, adding further area, homogenized photometric calibration, photometric redshifts and weak lensing shear measurements to the first two releases. A dedicated pipeline embedded in the Astro-WISE information system is used for the production of the main release. Modifications with respect to earlier releases are described in detail. Photometric redshifts have been derived using both Bayesian template fitting, and machine-learning techniques. For the weak lensing measurements, optimized procedures based on the THELI data reduction and lensfit shear measurement packages are used. In DR3 stacked ugri images, weight maps, masks, and source lists for 292 new survey tiles (~300 sq.deg) are made available. The multi-band catalogue, including homogenized photometry and photometric redshifts, covers the combined DR1, DR2 and DR3 footprint of 440 survey tiles (447 sq.deg). Limiting magnitudes are typically 24.3, 25.1, 24.9, 23.8 (5 sigma in a 2 arcsec aperture) in ugri, respectively, and the typical r-band PSF size is less than 0.7 arcsec. The photometric homogenization scheme ensures accurate colors and an absolute calibration stable to ~2% for gri and ~3% in u. Separately released are a weak lensing shear catalogue and photometric redshifts based on two different machine-learning techniques.

After its first implementation in 2003 the Astro-WISE technology has been rolled out in several European countries and is used for the production of the KiDS survey data. In the multi-disciplinary Target initiative this technology, nicknamed WISE technology, has been further applied to a large number of projects. Here, we highlight the data handling of other astronomical applications, such as VLT-MUSE and LOFAR, together with some non-astronomical applications such as the medical projects Lifelines and GLIMPS, the MONK handwritten text recognition system, and business applications, by amongst others, the Target Holding. We describe some of the most important lessons learned and describe the application of the data-centric WISE type of approach to the Science Ground Segment of the Euclid satellite.

The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the survey is mapping the large-scale matter distribution in the Universe back to a redshift of ~0.5. Secondary science cases are manifold, covering topics such as galaxy evolution, Milky Way structure, and the detection of high-redshift clusters and quasars. KiDS is an ESO Public Survey and dedicated to serving the astronomical community with high-quality data products derived from the survey data, as well as with calibration data. Public data releases will be made on a yearly basis, the first two of which are presented here. For a total of 148 survey tiles (~160 sq.deg.) astrometrically and photometrically calibrated, coadded ugri images have been released, accompanied by weight maps, masks, source lists, and a multi-band source catalog. A dedicated pipeline and data management system based on the Astro-WISE software system, combined with newly developed masking and source classification software, is used for the data production of the data products described here. The achieved data quality and early science projects based on the data products in the first two data releases are reviewed in order to validate the survey data. Early scientific results include the detection of nine high-z QSOs, fifteen candidate strong gravitational lenses, high-quality photometric redshifts and galaxy structural parameters for hundreds of thousands of galaxies. (Abridged)

Astro-WISE is a scientific information system for the data processing of optical images. In this paper we review main features of Astro-WISE and describe the current status of the system.

The LOFAR Long-Term Archive (LTA) is a multi-Petabyte scale data storage for the processed data of LOFAR telescope. We describe the adaptation of the WISE concept implemented by Target consortium for the LOFAR LTA and changes we introduced to it to accommodate LOFAR data. This paper describes an example of a new information system created on the basis of Astro-WISE for a wider range and scale of data.

The recent explosion of recorded digital data and its processed derivatives threatens to overwhelm researchers when analysing their experimental data or when looking up data items in archives and file systems. While current hardware developments allow to acquire, process and store 100s of terabytes of data at the cost of a modern sports car, the software systems to handle these data are lagging behind. This general problem is recognized and addressed by various scientific communities, e.g., DATAGRID/EGEE federates compute and storage power over the high-energy physical community, while the astronomical community is building an Internet geared Virtual Observatory, connecting archival data. These large projects either focus on a specific distribution aspect or aim to connect many sub-communities and have a relatively long trajectory for setting standards and a common layer. Here, we report "first light" of a very different solution to the problem initiated by a smaller astronomical IT community. It provides the abstract "scientific information layer" which integrates distributed scientific analysis with distributed processing and federated archiving and publishing. By designing new abstractions and mixing in old ones, a Science Information System with fully scalable cornerstones has been achieved, transforming data systems into knowledge systems. This break-through is facilitated by the full end-to-end linking of all dependent data items, which allows full backward chaining from the observer/researcher to the experiment. Key is the notion that information is intrinsic in nature and thus is the data acquired by a scientific experiment. The new abstraction is that software systems guide the user to that intrinsic information by forcing full backward and forward chaining in the data modelling.

We have studied the dust along the line-of-sight towards the Galactic Center using Short Wavelength Spectrometer (SWS) data obtained with the Infrared Space Observatory (ISO). We focussed on the wavelength region from 8-13 micron which is dominated by the strong silicate absorption feature. Using the absorption profiles observed towards Galactic Center Sources (GCS) 3 and 4, which are C-rich Wolf-Rayet Stars, as reference objects, we are able to disentangle the interstellar silicate absorption and the silicate emission intrinsic to the source, toward Sgr A* and derive a very accurate profile for the intrinsic 9.7 micron band. The interstellar absorption band is smooth and featureless and is well reproduced using a mixture of 15.1% amorphous pyroxene and 84.9% of amorphous olivine by mass, all in spherical sub-micron-sized grains. There is no direct evidence for substructure due to interstellar crystalline silicates. We are able to determine an upper limit to the degree of crystallinity of silicates in the diffuse interstellar medium (ISM), and conclude that the crystalline fraction of the interstellar silicates is 0.2% (+/- 0.2%) by mass. This is much lower than the degree of crystallinity observed in silicates in the circumstellar environment of evolved stars, the main contributors of dust to the ISM. There are two possible explanations for this discrepancy. First, an amorphization process occurs in the ISM on a time scale significantly shorter than the destruction time scale, possibly caused by particle bombardment by heavyweight ions. Second, we consider the possibility that the crystalline silicates in stellar ejecta are diluted by an additional source of amorphous silicates, in particular supernovae.