A Kavli foundation sponsored workshop on the theme \emph{Petabytes to Science} was held 12$^{th}$ to 14$^{th}$ of February 2019 in Las Vegas. The aim of the this workshop was to discuss important trends and technologies which may support astronomy. We also tackled how to better shape the workforce for the new trends and how we should approach education and public outreach. This document was coauthored during the workshop and edited in the weeks after. It comprises the discussions and highlights many recommendations which came out of the workshop. We shall distill parts of this document and formulate potential white papers for the decadal survey.

LSST has chosen Kubernetes as the platform for deploying and operating the LSST Science Platform. We first present the background reasoning behind this decision, including both instrument-agnostic as well as LSST-specific requirements. We then discuss the basic principles of Kubernetes and Helm, and how they are used as the deployment base for the LSST Science Platform. Furthermore, we provide an example of how an external group may use these publicly available software resources to deploy their own instance of the LSST Science Platform, and customize it to their needs. Finally, we discuss how more astronomy software can follow these patterns to gain similar benefits.

Fundamental coding and software development skills are increasingly necessary for success in nearly every aspect of astronomical and astrophysical research as large surveys and high resolution simulations become the norm. However, professional training in these skills is inaccessible or impractical for many members of our community. Students and professionals alike have been expected to acquire these skills on their own, apart from formal classroom curriculum or on-the-job training. Despite the recognized importance of these skills, there is little opportunity to develop them - even for interested researchers. To ensure a workforce capable of taking advantage of the computational resources and the large volumes of data coming in the next decade, we must identify and support ways to make software development training widely accessible to community members, regardless of affiliation or career level. To develop and sustain a technology capable astronomical and astrophysical workforce, we recommend that agencies make funding and other resources available in order to encourage, support and, in some cases, require progress on necessary training, infrastructure and policies. In this white paper, we focus on recommendations for how funding agencies can lead in the promotion of activities to support the astronomy and astrophysical workforce in the 2020s.

Many astronomy data centres still work on filesystems. Industry has moved on; current practice in computing infrastructure is to achieve Big Data scalability using object stores rather than POSIX file systems. This presents us with opportunities for portability and reuse of software underlying processing and archive systems but it also causes problems for legacy implementations in current data centers.

Commodity cloud computing, as provided by commercial vendors such as Amazon, Google, and Microsoft, has revolutionized computing in many sectors. With the advent of a new class of big data, public access astronomical facility such as LSST, DKIST, and WFIRST, there exists a real opportunity to combine these missions with cloud computing platforms and fundamentally change the way astronomical data is collected, processed, archived, and curated. Making these changes in a cross-mission, coordinated way can provide unprecedented economies of scale in personnel, data collection and management, archiving, algorithm and software development and, most importantly, science.

Software is a critical part of modern research, and yet there are insufficient mechanisms in the scholarly ecosystem to acknowledge, cite, and measure the impact of research software. The majority of academic fields rely on a one-dimensional credit model whereby academic articles (and their associated citations) are the dominant factor in the success of a researcher's career. In the petabyte era of astronomical science, citing software and measuring its impact enables academia to retain and reward researchers that make significant software contributions. These highly skilled researchers must be retained to maximize the scientific return from petabyte-scale datasets. Evolving beyond the one-dimensional credit model requires overcoming several key challenges, including the current scholarly ecosystem and scientific culture issues. This white paper will present these challenges and suggest practical solutions for elevating the role of software as a product of the research enterprise.

(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the Solar System, exploring the transient optical sky, and mapping the Milky Way. LSST will be a wide-field ground-based system sited at Cerro Pach\'{o}n in northern Chile. The telescope will have an 8.4 m (6.5 m effective) primary mirror, a 9.6 deg$^2$ field of view, and a 3.2 Gigapixel camera. The standard observing sequence will consist of pairs of 15-second exposures in a given field, with two such visits in each pointing in a given night. With these repeats, the LSST system is capable of imaging about 10,000 square degrees of sky in a single filter in three nights. The typical 5$\sigma$ point-source depth in a single visit in $r$ will be $\sim 24.5$ (AB). The project is in the construction phase and will begin regular survey operations by 2022. The survey area will be contained within 30,000 deg$^2$ with $\delta<+34.5^\circ$, and will be imaged multiple times in six bands, $ugrizy$, covering the wavelength range 320--1050 nm. About 90\% of the observing time will be devoted to a deep-wide-fast survey mode which will uniformly observe a 18,000 deg$^2$ region about 800 times (summed over all six bands) during the anticipated 10 years of operations, and yield a coadded map to $r\sim27.5$. The remaining 10\% of the observing time will be allocated to projects such as a Very Deep and Fast time domain survey. The goal is to make LSST data products, including a relational database of about 32 trillion observations of 40 billion objects, available to the public and scientists around the world.

The Large Synoptic Survey Telescope (LSST) is a large-aperture, wide-field, ground-based survey system that will image the sky in six optical bands from 320 to 1050 nm, uniformly covering approximately $18,000$deg$^2$ of the sky over 800 times. The LSST is currently under construction on Cerro Pach\'on in Chile, and expected to enter operations in 2022. Once operational, the LSST will explore a wide range of astrophysical questions, from discovering "killer" asteroids to examining the nature of Dark Energy. The LSST will generate on average 15 TB of data per night, and will require a comprehensive Data Management system to reduce the raw data to scientifically useful catalogs and images with minimum human intervention. These reductions will result in a real-time alert stream, and eleven data releases over the 10-year duration of LSST operations. To enable this processing, the LSST project is developing a new, general-purpose, high-performance, scalable, well documented, open source data processing software stack for O/IR surveys. Prototypes of this stack are already capable of processing data from existing cameras (e.g., SDSS, DECam, MegaCam), and form the basis of the Hyper-Suprime Cam (HSC) Survey data reduction pipeline.

The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard is, however, showing its age. Developed in the late 1970s the FITS authors made a number of implementation choices for the format that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not appreciate the challenges which we would be facing today in astronomical computing. Difficulties we now face include, but are not limited to, having to address the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets and the need to capture significantly more complex metadata and data relationships. There are members of the community today who find some (or all) of these limitations unworkable, and have decided to move ahead with storing data in other formats. This reaction should be taken as a wakeup call to the FITS community to make changes in the FITS standard, or to see its usage fall. In this paper we detail some selected important problems which exist within the FITS standard today. It is not our intention to prescribe specific remedies to these issues; rather, we hope to call attention of the FITS and greater astronomical computing communities to these issues in the hopes that it will spur action to address them.

The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard, however, is showing its age. Developed in the late 1970s, the FITS authors made a number of implementation choices that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not anticipate the challenges which we are facing today in astronomical computing. Difficulties we now face include, but are not limited to, addressing the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets, and capturing significantly more complex metadata and data relationships. There are members of the community today who find some or all of these limitations unworkable, and have decided to move ahead with storing data in other formats. If this fragmentation continues, we risk abandoning the advantages of broad interoperability, and ready archivability, that the FITS format provides for astronomy. In this paper we detail some selected important problems which exist within the FITS standard today. These problems may provide insight into deeper underlying issues which reside in the format and we provide a discussion of some lessons learned. It is not our intention here to prescribe specific remedies to these issues; rather, it is to call attention of the FITS and greater astronomical computing communities to these problems in the hope that it will spur action to address them.

We present, as a case study, a description of the partnership between an observatory (JCMT) and a data centre (CADC) that led to the development of the JCMT Science Archive (JSA). The JSA is a successful example of a service designed to use Virtual Observatory (VO) technologies from the start. We describe the motivation, process and lessons learned from this approach.

The extensible N-Dimensional Data Format (NDF) was designed and developed in the late 1980s to provide a data model suitable for use in a variety of astronomy data processing applications supported by the UK Starlink Project. Starlink applications were used extensively, primarily in the UK astronomical community, and form the basis of a number of advanced data reduction pipelines today. This paper provides an overview of the historical drivers for the development of NDF and the lessons learned from using a defined hierarchical data model for many years in data reduction software, data pipelines and in data acquisition systems.

ORAC-DR is a general purpose data reduction pipeline system designed to be instrument and observatory agnostic. The pipeline works with instruments as varied as infrared integral field units, imaging arrays and spectrographs, and sub-millimeter heterodyne arrays & continuum cameras. This paper describes the architecture of the pipeline system and the implementation of the core infrastructure. We finish by discussing the lessons learned since the initial deployment of the pipeline system in the late 1990s.

DevOps is the contemporary term for a software development culture that purposefully blurs distinction between software development and IT operations by treating "infrastructure as code." DevOps teams typically implement practices summarised by the colloquial directive to "eat your own dogfood;" meaning that software tools developed by a team should be used internally rather thrown over the fence to operations or users. We present a brief overview of how DevOps techniques bring proven software engineering practices to IT operations. We then discuss the application of these practices to astronomical observatories.

The Submillimetre Common User Bolometer Array 2 (SCUBA-2) is an instrument operating on the 15-m James Clerk Maxwell Telescope, nominally consisting of 5120 bolometers in each of two simultaneous imaging bands centred over 450 and 850 um. The camera is operated by scanning across the sky and recording data at a rate of 200 Hz. As the largest of a new generation of multiplexed kilopixel bolometer cameras operating in the (sub)millimetre, SCUBA-2 data analysis represents a significant challenge. We describe the production of maps using the Sub-Millimetre User Reduction Facility (SMURF) in which we have adopted a fast, iterative approach to map-making that enables data reduction on single, modern, high-end desktop computers, with execution times that are typically shorter than the observing times. SMURF is used in an automated setting, both at the telescope for real-time feedback to observers, as well as for the production of science products for the JCMT Science Archive at the Canadian Astronomy Data Centre. Three detailed case studies are used to: (i) explore convergence properties of the map-maker using simple prior constraints (Uranus -- a point source); (ii) achieve the white-noise limit for faint point-source studies (extragalactic blank-field survey of the Lockman Hole); and (iii) demonstrate that our strategy is capable of recovering angular scales comparable to the size of the array footprint (approximately 5 arcmin) for bright extended sources (star-forming region M17).

SCUBA-2 is an innovative 10000 pixel bolometer camera operating at submillimetre wavelengths on the James Clerk Maxwell Telescope (JCMT). The camera has the capability to carry out wide-field surveys to unprecedented depths, addressing key questions relating to the origins of galaxies, stars and planets. With two imaging arrays working simultaneously in the atmospheric windows at 450 and 850 microns, the vast increase in pixel count means that SCUBA-2 maps the sky 100-150 times faster than the previous SCUBA instrument. In this paper we present an overview of the instrument, discuss the physical characteristics of the superconducting detector arrays, outline the observing modes and data acquisition, and present the early performance figures on the telescope. We also showcase the capabilities of the instrument via some early examples of the science SCUBA-2 has already undertaken. In February 2012, SCUBA-2 began a series of unique legacy surveys for the JCMT community. These surveys will take 2.5 years and the results are already providing complementary data to the shorter wavelength, shallower, larger-area surveys from Herschel. The SCUBA-2 surveys will also provide a wealth of information for further study with new facilities such as ALMA, and future telescopes such as CCAT and SPICA.

For more than a decade the Joint Astronomy Centre has been developing software tools to simplify observing and make it possible to use the telescopes in many different operational modes. In order to support remote operations the data handling systems need to be in place to allow observation preparation, flexible queue scheduling, data quality pipelines and science archives all to be connected in a data-driven environment. We discuss the history of these developments at UKIRT and JCMT and how the decision to combine software development at both telescopes led each to get features that they could not have justified if they were treated independently.

We have carried out a pilot study for the SCUBA-2 'All-Sky' Survey, SASSy, a wide and shallow mapping project at 850 microns, designed to find rare objects, both Galactic and extragalactic. Two distinct sets of exploratory observations were undertaken, and used to test the SASSy approach and data reduction pipeline. The first was a 0.5 by 0.5 degrees map around the nearby galaxy NGC 2559. The galaxy was easily detected at 156 mJy, but no other convincing sources are present in the map. Comparison with other galaxies with similar wavelength coverage indicates that NGC 2559 has relatively warm dust. The second observations cover 1 square degree around the W5-E HII region. As well as diffuse structure in the map, a filtering approach was able to extract 27 compact sources with signal-to-noise greater than 6. By matching with data at other wavelengths we can see that the SCUBA-2 data can be used to discriminate the colder cores. Together these observations show that the SASSy project will be able to meet its original goals of detecting new bright sources which will be ideal for follow-up observations with other facilities.

SCUBA-2 is the largest submillimetre array camera in the world and was commissioned on the James Clerk Maxwell Telescope (JCMT) with two arrays towards the end of 2009. A period of shared-risks observing was then completed and the full planned complement of 8 arrays, 4 at 850 microns and 4 at 450 microns, are now installed and ready to be commissioned. SCUBA-2 has 10,240 bolometers, corresponding to a data rate of 8 MB/s when sampled at the nominal rate of 200 Hz. The pipeline produces useful maps in near real time at the telescope and often publication quality maps in the JCMT Science Archive (JSA) hosted at the Canadian Astronomy Data Centre (CADC).

The sub-millimetre wavelength regime is perhaps the most poorly explored over large areas of the sky, despite the considerable effort that has been expended in making deep maps over small regions. As a consequence the properties of the sub-millimetre sky as a whole, and of rare bright objects in particular, remains largely unknown. Here we describe a forthcoming survey (the SCUBA-2 ``All-Sky'' Survey, or SASSy) designed to address this issue by making a large-area map of approximately one-fifth of the sky visible from the JCMT (4800 square degrees) down to a 1 sigma noise level of 30 mJy/beam. This map forms the pilot for a much larger survey, which will potentially map the remaining sky visible from the JCMT, with the region also visible to ALMA as a priority. SASSy has been awarded 500 hours for the 4800 square degree pilot phase and will commence after the commissioning of SCUBA-2, expected in early 2008.

SCUBA-2 is an innovative 10,000 pixel submillimeter camera due to be delivered to the James Clerk Maxwell Telescope in late 2006. The camera is expected to revolutionize submillimeter astronomy in terms of the ability to carry out wide-field surveys to unprecedented depths addressing key questions relating to the origins of galaxies, stars and planets. This paper presents an update on the project with particular emphasis on the laboratory commissioning of the instrument. The assembly and integration will be described as well as the measured thermal performance of the instrument. A summary of the performance results will be presented from the TES bolometer arrays, which come complete with in-focal plane SQUID amplifiers and multiplexed readouts, and are cooled to 100mK by a liquid cryogen-free dilution refrigerator. Considerable emphasis has also been placed on the operating modes of the instrument and the common-user aspect of the user interface and data reduction pipeline. These areas will also be described in the paper.

We present a new evidence of the accretion disc in active galactic nuclei, by examining the properties of the Ha emission line versus viewing angle, in a sample of superluminal (SL) quasars. Both line velocity width (FWHM) and rest equivalent width (EW) correlate with viewing angle. These correlations are quantitatively consistent with a disc geometry for both the line and the underlying continuum source.

The Submillimetre Common User Bolometer Array (SCUBA) instrument has been operating on the James Clerk Maxwell Telescope (JCMT) since 1997. The data archive is now sufficiently large that it can be used to investigate instrumental properties and the variability of astronomical sources. This paper describes the automated calibration and reduction scheme used to process the archive data with particular emphasis on `jiggle-map' observations of compact sources. We demonstrate the validity of our automated approach at both 850- and 450-microns and apply it to several of the JCMT secondary flux calibrators. We determine light curves for the variable sources IRC+10216 and OH231.8. This automation is made possible by using the ORAC-DR data reduction pipeline, a flexible and extensible data reduction pipeline that is used on UKIRT and the JCMT.

We report the results of J-band infrared spectroscopy of a sample of 13 z=1 field galaxies drawn from the Canada-France Redshift Survey, targeting galaxies whose redshifts place the rest frame H-alpha line emission from HII regions in between the bright night sky OH lines. As a result we detect emission down to a flux limit of ~10^{-16} ergs cm^{-2} s^{-1} corresponding to a luminosity limit of ~10^{41} ergs at this redshift for a H_0=50 km s^{-1} Mpc,^{-1} q_0=0.5 cosmology. From these luminosities we derive estimates of the star-formation rates in these galaxies which are independent of previous estimates based upon their rest-frame ultraviolet (2800A) luminosity. The mean star-formation rate at z=1, from this sample, is found to be at least three times as high as the ultraviolet estimates. The standard dust extinction in these galaxies is inferred to be A_V=0.5-1.0 mags, comparable to local field galaxies, suggesting that the bulk of star-formation is not heavily obscured unless one uses greyer extinction laws. Star-forming galaxies have the bluest colours and a preponderance of disturbed/interacting morphologies. We also investigate the effects of particular star-formation histories, in particular the role of bursts vs continuous star-formation in changing the detailed distribution of UV to H-alpha emission. Generally we find that models dominated by short, overlapping, bursts at typically 0.2 Gyr intervals provide a better model for the data than a constant rate of star-formation. The star-formation history of the Universe from Balmer lines is compiled and found to be typically 2--3\times higher than that inferred from the UV {\em at all redshifts}. It can not yet be clearly established whether the star-formation rate falls off or remains constant at high-redshift.