We report the detection of a bright fast radio burst, FRB\,191108, with Apertif on the Westerbork Synthesis Radio Telescope (WSRT). The interferometer allows us to localise the FRB to a narrow $5\arcsec\times7\arcmin$ ellipse by employing both multibeam information within the Apertif phased-array feed (PAF) beam pattern, and across different tied-array beams. The resulting sight line passes close to Local Group galaxy M33, with an impact parameter of only 18\,kpc with respect to the core. It also traverses the much larger circumgalactic medium of M31, the Andromeda Galaxy. We find that the shared plasma of the Local Group galaxies could contribute $\sim$10\% of its dispersion measure of 588\,pc\,cm$^{-3}$, but not detectable scintillation, temporal scattering, or significant Faraday rotation in this case. FRB\,191108 has a Faraday rotation measure of +474\,$\pm\,3$\,rad\,m$^{-2}$, which is too large to be explained by either the Milky Way or the intergalactic medium. This indicates a dense local magneto-ionic environment in the source host galaxy---as has been argued for other FRBs. We found no accompanying persistent radio sources in the Apertif imaging survey data.

Searches for optical transients are usually performed with a cadence of days to weeks, optimised for supernova discovery. The optical fast transient sky is still largely unexplored, with only a few surveys to date having placed meaningful constraints on the detection of extragalactic transients evolving at sub-hour timescales. Here, we present the results of deep searches for dim, minute-timescale extragalactic fast transients using the Dark Energy Camera, a core facility of our all-wavelength and all-messenger Deeper, Wider, Faster programme. We used continuous 20s exposures to systematically probe timescales down to 1.17 minutes at magnitude limits $g > 23$ (AB), detecting hundreds of transient and variable sources. Nine candidates passed our strict criteria on duration and non-stellarity, all of which could be classified as flare stars based on deep multi-band imaging. Searches for fast radio burst and gamma-ray counterparts during simultaneous multi-facility observations yielded no counterparts to the optical transients. Also, no long-term variability was detected with pre-imaging and follow-up observations using the SkyMapper optical telescope. We place upper limits for minute-timescale fast optical transient rates for a range of depths and timescales. Finally, we demonstrate that optical $g$-band light curve behaviour alone cannot discriminate between confirmed extragalactic fast transients such as prompt GRB flashes and Galactic stellar flares.

Repeating fast radio bursts (FRBs) present excellent opportunities to identify FRB progenitors and host environments, as well as decipher the underlying emission mechanism. Detailed studies of repeating FRBs might also hold clues to the origin of FRBs as a population. We aim to detect the first two repeating FRBs: FRB 121102 (R1) and FRB 180814.J0422+73 (R2), and characterise their repeat statistics. We also want to significantly improve the sky localisation of R2. We use the Westerbork Synthesis Radio Telescope to conduct extensive follow-up of these two repeating FRBs. The new phased-array feed system, Apertif, allows covering the entire sky position uncertainty of R2 with fine spatial resolution in one pointing. We characterise the energy distribution and the clustering of detected R1 bursts. We detected 30 bursts from R1. Our measurements indicate a dispersion measure of 563.5(2) pc cm$^{-3}$, suggesting a significant increase in DM over the past few years. We place an upper limit of 8% on the linear polarisation fraction of the brightest burst. We did not detect any bursts from R2. A single power-law might not fit the R1 burst energy distribution across the full energy range or widely separated detections. Our observations provide improved constraints on the clustering of R1 bursts. Our stringent upper limits on the linear polarisation fraction imply a significant depolarisation, either intrinsic to the emission mechanism or caused by the intervening medium, at 1400 MHz that is not observed at higher frequencies. The non-detection of any bursts from R2 implies either a highly clustered nature of the bursts, a steep spectral index, or a combination of both. Alternatively, R2 has turned off completely, either permanently or for an extended period of time.

Current and upcoming radio telescopes are being designed with increasing sensitivity to detect new and mysterious radio sources of astrophysical origin. While this increased sensitivity improves the likelihood of discoveries, it also makes these instruments more susceptible to the deleterious effects of Radio Frequency Interference (RFI). The challenge posed by RFI is exacerbated by the high data-rates achieved by modern radio telescopes, which require real-time processing to keep up with the data. Furthermore, the high data-rates do not allow for permanent storage of observations at high resolution. Offline RFI mitigation is therefore not possible anymore. The real-time requirement makes RFI mitigation even more challenging because, on one side, the techniques used for mitigation need to be fast and simple, and on the other side they also need to be robust enough to cope with just a partial view of the data. The Apertif Radio Transient System (ARTS) is the real-time, time-domain, transient detection instrument of the Westerbork Synthesis Radio Telescope (WSRT), processing 73 Gb of data per second. Even with a deep learning classifier, the ARTS pipeline requires state-of-the-art real-time RFI mitigation to reduce the number of false-positive detections. Our solution to this challenge is RFIm, a high-performance, open-source, tuned, and extensible RFI mitigation library. The goal of this library is to provide users with RFI mitigation routines that are designed to run in real-time on many-core accelerators, such as Graphics Processing Units, and that can be highly-tuned to achieve code and performance portability to different hardware platforms and scientific use-cases. Results on the ARTS show that we can achieve real-time RFI mitigation, with a minimal impact on the total execution time of the search pipeline, and considerably reduce the number of false-positives.

Fundamental coding and software development skills are increasingly necessary for success in nearly every aspect of astronomical and astrophysical research as large surveys and high resolution simulations become the norm. However, professional training in these skills is inaccessible or impractical for many members of our community. Students and professionals alike have been expected to acquire these skills on their own, apart from formal classroom curriculum or on-the-job training. Despite the recognized importance of these skills, there is little opportunity to develop them - even for interested researchers. To ensure a workforce capable of taking advantage of the computational resources and the large volumes of data coming in the next decade, we must identify and support ways to make software development training widely accessible to community members, regardless of affiliation or career level. To develop and sustain a technology capable astronomical and astrophysical workforce, we recommend that agencies make funding and other resources available in order to encourage, support and, in some cases, require progress on necessary training, infrastructure and policies. In this white paper, we focus on recommendations for how funding agencies can lead in the promotion of activities to support the astronomy and astrophysical workforce in the 2020s.

We present a systematic evaluation of JPEG2000 (ISO/IEC 15444) as a transport data format to enable rapid remote searches for fast transient events as part of the Deeper Wider Faster program (DWF). DWF uses ~20 telescopes from radio to gamma-rays to perform simultaneous and rapid-response follow-up searches for fast transient events on millisecond-to-hours timescales. DWF search demands have a set of constraints that is becoming common amongst large collaborations. Here, we focus on the rapid optical data component of DWF led by the Dark Energy Camera (DECam) at CTIO. Each DECam image has 70 total CCDs saved as a ~1.2 gigabyte FITS file. Near real-time data processing and fast transient candidate identifications -- in minutes for rapid follow-up triggers on other telescopes -- requires computational power exceeding what is currently available on-site at CTIO. In this context, data files need to be transmitted rapidly to a foreign location for supercomputing post-processing, source finding, visualization and analysis. This step in the search process poses a major bottleneck, and reducing the data size helps accommodate faster data transmission. To maximise our gain in transfer time and still achieve our science goals, we opt for lossy data compression -- keeping in mind that raw data is archived and can be evaluated at a later time. We evaluate how lossy JPEG2000 compression affects the process of finding transients, and find only a negligible effect for compression ratios up to ~25:1. We also find a linear relation between compression ratio and the mean estimated data transmission speed-up factor. Adding highly customized compression and decompression steps to the science pipeline considerably reduces the transmission time -- validating its introduction to the DWF science pipeline and enabling science that was otherwise too difficult with current technology.

Despite the popularity of the Graphics Processing Unit (GPU) for general purpose computing, one should not forget about the practicality of the GPU for fast scientific visualisation. As astronomers have increasing access to three dimensional (3D) data from instruments and facilities like integral field units and radio interferometers, visualisation techniques such as volume rendering offer means to quickly explore spectral cubes as a whole. As most 3D visualisation techniques have been developed in fields of research like medical imaging and fluid dynamics, many transfer functions are not optimal for astronomical data. We demonstrate how transfer functions and graphics shaders can be exploited to provide new astronomy-specific explorative colouring methods. We present 12 shaders, including four novel transfer functions specifically designed to produce intuitive and informative 3D visualisations of spectral cube data. We compare their utility to classic colour mapping. The remaining shaders highlight how common computation like filtering, smoothing and line ratio algorithms can be integrated as part of the graphics pipeline. We discuss how this can be achieved by utilising the parallelism of modern GPUs along with a shading language, letting astronomers apply these new techniques at interactive frame rates. All shaders investigated in this work are included in the open source software shwirl (Vohl, 2017).

By applying a display ecology to the {\em Deeper, Wider, Faster} proactive, simultaneous telescope observing campaign, we have shown a dramatic reduction in the time taken to inspect DECam CCD images for potential transient candidates and to produce time-critical triggers to standby telescopes. We also show how facilitating rapid corroboration of potential candidates and the exclusion of non-candidates improves the accuracy of detection; and establish that a practical and enjoyable workspace can improve the experience of an otherwise taxing task for astronomers. We provide a critical road-test of two advanced displays in a research context -- a rare opportunity to demonstrate how they can be used rather than simply discuss how they might be used to accelerate discovery.

Radio survey datasets comprise an increasing number of individual observations stored as sets of multidimensional data. In large survey projects, astronomers commonly face limitations regarding: 1) interactive visual analytics of sufficiently large subsets of data; 2) synchronous and asynchronous collaboration; and 3) documentation of the discovery workflow. To support collaborative data inquiry, we present encube, a large-scale comparative visual analytics framework. Encube can utilise advanced visualization environments such as the CAVE2 (a hybrid 2D and 3D virtual reality environment powered with a 100 Tflop/s GPU-based supercomputer and 84 million pixels) for collaborative analysis of large subsets of data from radio surveys. It can also run on standard desktops, providing a capable visual analytics experience across the display ecology. Encube is composed of four primary units enabling compute-intensive processing, advanced visualisation, dynamic interaction, parallel data query, along with data management. Its modularity will make it simple to incorporate astronomical analysis packages and Virtual Observatory capabilities developed within our community. We discuss how encube builds a bridge between high-end display systems (such as CAVE2) and the classical desktop, preserving all traces of the work completed on either platform -- allowing the research process to continue wherever you are.

We present encube $-$ a qualitative, quantitative and comparative visualisation and analysis system, with application to high-resolution, immersive three-dimensional environments and desktop displays. encube extends previous comparative visualisation systems by considering: 1) the integration of comparative visualisation and analysis into a unified system; 2) the documentation of the discovery process; and 3) an approach that enables scientists to continue the research process once back at their desktop. Our solution enables tablets, smartphones or laptops to be used as interaction units for manipulating, organising, and querying data. We highlight the modularity of encube, allowing additional functionalities to be included as required. Additionally, our approach supports a high level of collaboration within the physical environment. We show how our implementation of encube operates in a large-scale, hybrid visualisation and supercomputing environment using the CAVE2 at Monash University, and on a local desktop, making it a versatile solution. We discuss how our approach can help accelerate the discovery rate in a variety of research scenarios.

As the quantity and resolution of spectral-cubes from optical/infrared and radio surveys increase, desktop-based visualization and analysis solutions must adapt and evolve. Novel immersive 3D environments such as the CAVE2 at Monash University can overcome personal computer's visualization limitations. CAVE2 is part advanced 2D/3D visualization space (80 stereo-capable screens providing a total of 84 million pixels) and part supercomputer ($\sim100$ TFLOPS of integrated GPU-based processing power). We present a novel visualization system enabling simultaneous 3D comparative visualization of $\sim100$ spectral-cubes. With CAVE2 augmented by our newly developed web-based controller interface, astronomers can easily organise spectral-cubes on the different display panels, apply real-time transforms to one or many spectral cubes, and request quantitative information about the displayed data. We also discuss how such a solution can help accelerate the discovery rate in varied research scenarios.

As the volume of data grows, astronomers are increasingly faced with choices on what data to keep -- and what to throw away. Recent work evaluating the JPEG2000 (ISO/IEC 15444) standards as a future data format standard in astronomy has shown promising results on observational data. However, there is still a need to evaluate its potential on other type of astronomical data, such as from numerical simulations. GERLUMPH (the GPU-Enabled High Resolution cosmological MicroLensing parameter survey) represents an example of a data intensive project in theoretical astrophysics. In the next phase of processing, the ~27 terabyte GERLUMPH dataset is set to grow by a factor of 100 -- well beyond the current storage capabilities of the supercomputing facility on which it resides. In order to minimise bandwidth usage, file transfer time, and storage space, this work evaluates several data compression techniques. Specifically, we investigate off-the-shelf and custom lossless compression algorithms as well as the lossy JPEG2000 compression format. Results of lossless compression algorithms on GERLUMPH data products show small compression ratios (1.35:1 to 4.69:1 of input file size) varying with the nature of the input data. Our results suggest that JPEG2000 could be suitable for other numerical datasets stored as gridded data or volumetric data. When approaching lossy data compression, one should keep in mind the intended purposes of the data to be compressed, and evaluate the effect of the loss on future analysis. In our case study, lossy compression and a high compression ratio do not significantly compromise the intended use of the data for constraining quasar source profiles from cosmological microlensing.

In the upcoming synoptic all--sky survey era of astronomy, thousands of new multiply imaged quasars are expected to be discovered and monitored regularly. Light curves from the images of gravitationally lensed quasars are further affected by superimposed variability due to microlensing. In order to disentangle the microlensing from the intrinsic variability of the light curves, the time delays between the multiple images have to be accurately measured. The resulting microlensing light curves can then be analyzed to reveal information about the background source, such as the size of the quasar accretion disc. In this paper we present the most extensive and coherent collection of simulated microlensing light curves; we have generated $>2.5$ billion light curves using the GERLUMPH high resolution microlensing magnification maps. Our simulations can be used to: train algorithms to measure lensed quasar time delays, plan future monitoring campaigns, and study light curve properties throughout parameter space. Our data are openly available to the community and are complemented by online eResearch tools, located at http://gerlumph.swin.edu.au .