Time domain radio astronomy observing campaigns frequently generate large volumes of data. Our goal is to develop automated methods that can identify events of interest buried within the larger data stream. The V-FASTR fast transient system was designed to detect rare fast radio bursts (FRBs) within data collected by the Very Long Baseline Array. The resulting event candidates constitute a significant burden in terms of subsequent human reviewing time. We have trained and deployed a machine learning classifier that marks each candidate detection as a pulse from a known pulsar, an artifact due to radio frequency interference, or a potential new discovery. The classifier maintains high reliability by restricting its predictions to those with at least 90% confidence. We have also implemented several efficiency and usability improvements to the V-FASTR web-based candidate review system. Overall, we found that time spent reviewing decreased and the fraction of interesting candidates increased. The classifier now classifies (and therefore filters) 80-90% of the candidates, with an accuracy greater than 98%, leaving only the 10-20% most promising candidates to be reviewed by humans.

The V-FASTR experiment on the Very Long Baseline Array was designed to detect dispersed pulses of milliseconds duration, such as fast radio bursts (FRBs). We use all V-FASTR data through February 2015 to report V-FASTR's upper limits on the rates of FRBs, and compare these with re-derived rates from Parkes FRB detection experiments. V-FASTR's operation at lambda=20 cm allows direct comparison with the 20 cm Parkes rate, and we derive a power-law limit of \gamma<-0.4 (95% confidence limit) on the index of FRB source counts, N(>S)\propto S^\gamma. Using the previously measured FRB rate and the unprecedented amount of survey time spent searching for FRBs at a large range of wavelengths (0.3 cm > \lambda > 90 cm), we also place frequency-dependent limits on the spectral distribution of FRBs. The most constraining frequencies place two-point spectral index limits of \alpha_{20cm}^{4cm} < 5.8 and \alpha_{90cm}^{20cm} > -7.6, where fluence F \propto f^\alpha if we assume true the burst rate reported by Champion et al. (2016) of R(F~0.6 Jy ms) = 7 x 10^3 sky^{-1} day^{-1} (for bursts of ~3 ms duration). This upper limit on \alpha suggests that if FRBs are extragalactic but non-cosmological, that on average they are not experiencing excessive free-free absorption due to a medium with high optical depth (assuming temperature ~8,000 K), which excessively invert their low-frequency spectrum. This in turn implies that the dispersion of FRBs arises in either or both of the intergalactic medium or the host galaxy, rather than from the source itself.

The advent of synoptic sky surveys has spurred the development of techniques for real-time classification of astronomical sources in order to ensure timely follow-up with appropriate instruments. Previous work has focused on algorithm selection or improved light curve representations, and naively convert light curves into structured feature sets without regard for the time span or phase of the light curves. In this paper, we highlight the violation of a fundamental machine learning assumption that occurs when archival light curves with long observational time spans are used to train classifiers that are applied to light curves with fewer observations. We propose two solutions to deal with the mismatch in the time spans of training and test light curves. The first is the use of classifier committees where each classifier is trained on light curves of different observational time spans. Only the committee member whose training set matches the test light curve time span is invoked for classification. The second solution uses hierarchical classifiers that are able to predict source types both individually and by sub-group, so that the user can trade-off an earlier, more robust classification with classification granularity. We test both methods using light curves from the MACHO survey, and demonstrate their usefulness in improving performance over similar methods that naively train on all available archival data.

We report on a detailed study of the measurement stability for WFC3 IR grism stare and spatial scan observations. The excess measurement noise for both modes is established by comparing the observed and theoretical measurement uncertainties. We find that the stare-mode observations produce differential measurements that are consistent and achieve $\sim\,1.3$ times photon-limited measurement precision. In contrast, the spatial-scan mode observations produce measurements which are inconsistent, non-Gaussian, and have higher excess noise corresponding to $\sim\,2$ times the photon-limited precision. The inferior quality of the spatial scan observations is due to spatial-temporal variability in the detector performance which we measure and map. The non-Gaussian nature of spatial scan measurements makes the use of $\chi^2$ and the determination of formal confidence intervals problematic and thus renders the comparison of spatial scan data with theoretical models or other measurements difficult. With better measurement stability and no evidence for non-Gaussianity, stare mode observations offer a significant advantage for characterizing transiting exoplanet systems.

A new generation of observational science instruments is dramatically increasing collected data volumes in a range of fields. These instruments include the Square Kilometre Array (SKA), Large Synoptic Survey Telescope (LSST), terrestrial sensor networks, and NASA satellites participating in "decadal survey" missions. Their unprecedented coverage and sensitivity will likely reveal wholly new categories of unexpected and transient events. Commensal methods passively analyze these data streams, recognizing anomalous events of scientific interest and reacting in real time. We report on a case example: V-FASTR, an ongoing commensal experiment at the Very Long Baseline Array (VLBA) that uses online adaptive pattern recognition to search for anomalous fast radio transients. V-FASTR triages a millisecond-resolution stream of data and promotes candidate anomalies for further offline analysis. It tunes detection parameters in real time, injecting synthetic events to continually retrain itself for optimum performance. This self-tuning approach retains sensitivity to weak signals while adapting to changing instrument configurations and noise conditions. The system has operated since July 2011, making it the longest-running real time commensal radio transient experiment to date.

We define a framework for determining constraints on the detection rate of fast transient events from a population of underlying sources, with a view to incorporating beam shape, frequency effects, scattering effects, and detection efficiency into the metric. We then demonstrate a method for combining independent datasets into a single event rate constraint diagram, using a probabilistic approach to the limits on parameter space. We apply this new framework to present the latest results from the V-FASTR experiment, a commensal fast transients search using the Very Long Baseline Array (VLBA). In the 20 cm band, V-FASTR now has the ability to probe the regions of parameter space of importance for the observed Lorimer and Keane fast radio transient candidates, by combining the information from observations with differing bandwidths, and properly accounting for the source dispersion measure, VLBA antenna beam shape, experiment time sampling, and stochastic nature of events. We then apply the framework to combine the results of the V-FASTR and ATA Fly's Eye experiments, demonstrating their complementarity. Expectations for fast transients experiments for the SKA Phase I dish array are then computed, and the impact of large differential bandwidths is discussed.

The Australian Square Kilometre Array Pathfinder (ASKAP) will give us an unprecedented opportunity to investigate the transient sky at radio wavelengths. In this paper we present VAST, an ASKAP survey for Variables and Slow Transients. VAST will exploit the wide-field survey capabilities of ASKAP to enable the discovery and investigation of variable and transient phenomena from the local to the cosmological, including flare stars, intermittent pulsars, X-ray binaries, magnetars, extreme scattering events, interstellar scintillation, radio supernovae and orphan afterglows of gamma ray bursts. In addition, it will allow us to probe unexplored regions of parameter space where new classes of transient sources may be detected. In this paper we review the known radio transient and variable populations and the current results from blind radio surveys. We outline a comprehensive program based on a multi-tiered survey strategy to characterise the radio transient sky through detection and monitoring of transient and variable sources on the ASKAP imaging timescales of five seconds and greater. We also present an analysis of the expected source populations that we will be able to detect with VAST.

We present the first results from the V-FASTR experiment, a commensal search for fast transient radio bursts using the Very Long Baseline Array (VLBA). V-FASTR is unique in that the widely spaced VLBA antennas provide a discriminant against non-astronomical signals and a mechanism for the localization and identification of events that is not possible with single dishes or short baseline interferometers. Thus far V-FASTR has accumulated over 1300 hours of observation time with the VLBA, between 90 cm and 3 mm wavelength (327 MHz - 86 GHz), providing the first limits on fast transient event rates at high radio frequencies (>1.4 GHz). V-FASTR has blindly detected bright individual pulses from seven known pulsars but has not detected any single-pulse events that would indicate high redshift impulsive bursts of radio emission. At 1.4 GHz, V-FASTR puts limits on fast transient event rates comparable with the PALFA survey at the Arecibo telescope, but generally at lower sensitivities, and comparable to the "fly's eye" survey at the Allen Telescope Array, but with less sky coverage. We also illustrate the likely performance of the Phase 1 SKA dish array for an incoherent fast transient search fashioned on V-FASTR.

Recent discoveries of dispersed, non-periodic impulsive radio signals with single-dish radio telescopes have sparked significant interest in exploring the relatively uncharted space of fast transient radio signals. Here we describe V-FASTR, an experiment to perform a blind search for fast transient radio signals using the Very Long Baseline Array (VLBA). The experiment runs entirely in a commensal mode, alongside normal VLBA observations and operations. It is made possible by the features and flexibility of the DiFX software correlator that is used to process VLBA data. Using the VLBA for this type of experiment offers significant advantages over single-dish experiments, including a larger field of view, the ability to easily distinguish local radio-frequency interference from real signals and the possibility to localize detected events on the sky to milliarcsecond accuracy. We describe our software pipeline, which accepts short integration (~ms) spectrometer data from each antenna in real time during correlation and performs an incoherent dedispersion separately for each antenna, over a range of trial dispersion measures. The dedispersed data are processed by a sophisticated detector and candidate events are recorded. At the end of the correlation, small snippets of the raw data at the time of the events are stored for further analysis. We present the results of our event detection pipeline from some test observations of the pulsars B0329+54 and B0531+21 (the Crab pulsar).

Recent investigations reveal an important new class of transient radio phenomena that occur on sub-millisecond timescales. Often transient surveys' data volumes are too large to archive exhaustively. Instead, an on-line automatic system must excise impulsive interference and detect candidate events in real-time. This work presents a case study using data from multiple geographically distributed stations to perform simultaneous interference excision and transient detection. We present several algorithms that incorporate dedispersed data from multiple sites, and report experiments with a commensal real-time transient detection system on the Very Long Baseline Array (VLBA). We test the system using observations of pulsar B0329+54. The multiple-station algorithms enhanced sensitivity for detection of individual pulses. These strategies could improve detection performance for a future generation of geographically distributed arrays such as the Australian Square Kilometre Array Pathfinder and the Square Kilometre Array.

We are developing a purely commensal survey experiment for fast (<5s) transient radio sources. Short-timescale transients are associated with the most energetic and brightest single events in the Universe. Our objective is to cover the enormous volume of transients parameter space made available by ASKAP, with an unprecedented combination of sensitivity and field of view. Fast timescale transients open new vistas on the physics of high brightness temperature emission, extreme states of matter and the physics of strong gravitational fields. In addition, the detection of extragalactic objects affords us an entirely new and extremely sensitive probe on the huge reservoir of baryons present in the IGM. We outline here our approach to the considerable challenge involved in detecting fast transients, particularly the development of hardware fast enough to dedisperse and search the ASKAP data stream at or near real-time rates. Through CRAFT, ASKAP will provide the testbed of many of the key technologies and survey modes proposed for high time resolution science with the SKA.