SAGECal has been designed to find the most accurate calibration solutions for low radio frequency imaging observations, with minimum artefacts due to incomplete sky models. SAGECAL is developed to handle extremely large datasets, e.g., when the number of frequency bands greatly exceeds the number of available nodes on a compute cluster. Accurate calibration solutions are derived at the expense of large computational loads, which require distributed computing and modern compute devices, such as GPUs, to decrease runtimes. In this work, we investigate if the GPU version of SAGECal scales well enough to meet the requirements for the Square Kilometre Array and we compare its performance with the CPU version.

We present a stochastic, limited-memory Broyden Fletcher Goldfarb Shanno (LBFGS) algorithm that is suitable for handling very large amounts of data. A direct application of this algorithm is radio interferometric calibration of raw data at fine time and frequency resolution. Almost all existing radio interferometric calibration algorithms assume that it is possible to fit the dataset being calibrated into memory. Therefore, the raw data is averaged in time and frequency to reduce its size by many orders of magnitude before calibration is performed. However, this averaging is detrimental for the detection of some signals of interest that have narrow bandwidth and time duration such as fast radio bursts (FRBs). Using the proposed algorithm, it is possible to calibrate data at such a fine resolution that they cannot be entirely loaded into memory, thus preserving such signals. As an additional demonstration, we use the proposed algorithm for training deep neural networks and compare the performance against the mainstream first order optimization algorithms that are used in deep learning.

PySE is a Python software package for finding and measuring sources in radio telescope images. The software was designed to detect sources in the LOFAR telescope images, but can be used with images from other radio telescopes as well. We introduce the LOFAR Telescope, the context within which PySE was developed, the design of PySE, and describe how it is used. Detailed experiments on the validation and testing of PySE are then presented, along with results of performance testing. We discuss some of the current issues with the algorithms implemented in PySE and their inter- action with LOFAR images, concluding with the current status of PySE and its future development.

The Square Kilometre Array will be an amazing instrument for pulsar astronomy. While the full SKA will be sensitive enough to detect all pulsars in the Galaxy visible from Earth, already with SKA1, pulsar searches will discover enough pulsars to increase the currently known population by a factor of four, no doubt including a range of amazing unknown sources. Real time processing is needed to deal with the 60 PB of pulsar search data collected per day, using a signal processing pipeline required to perform more than 10 POps. Here we present the suggested design of the pulsar search engine for the SKA and discuss challenges and solutions to the pulsar search venture.

New and upcoming radio interferometers will produce unprecedented amounts of data that demand extremely powerful computers for processing. This is a limiting factor due to the large computational power and energy costs involved. Such limitations restrict several key data processing steps in radio interferometry. One such step is calibration where systematic errors in the data are determined and corrected. Accurate calibration is an essential component in reaching many scientific goals in radio astronomy and the use of consensus optimization that exploits the continuity of systematic errors across frequency significantly improves calibration accuracy. In order to reach full consensus, data at all frequencies need to be calibrated simultaneously. In the SKA regime, this can become intractable if the available compute agents do not have the resources to process data from all frequency channels simultaneously. In this paper, we propose a multiplexing scheme that is based on the alternating direction method of multipliers (ADMM) with cyclic updates. With this scheme, it is possible to simultaneously calibrate the full dataset using far fewer compute agents than the number of frequencies at which data are available. We give simulation results to show the feasibility of the proposed multiplexing scheme in simultaneously calibrating a full dataset when a limited number of compute agents are available.

Distributed radio interferometric calibration based on consensus optimization has been shown to improve the estimation of systematic errors in radio astronomical observations. The intrinsic continuity of systematic errors across frequency is used by a consensus polynomial to penalize traditional calibration. Consensus is achieved via the use of alternating direction method of multipliers (ADMM) algorithm. In this paper, we extend the existing distributed calibration algorithms to use ADMM with an adaptive penalty parameter update. Compared to a fixed penalty, its adaptive update has been shown to perform better in diverse applications of ADMM. In this paper, we compare two such popular penalty parameter update schemes: residual balance penalty update and spectral penalty update (Barzilai-Borwein). We apply both schemes to distributed radio interferometric calibration and compare their performance against ADMM with a fixed penalty parameter. Simulations show that both methods of adaptive penalty update improve the convergence of ADMM but the spectral penalty parameter update shows more stability.

Current and future astronomical survey facilities provide a remarkably rich opportunity for transient astronomy, combining unprecedented fields of view with high sensitivity and the ability to access previously unexplored wavelength regimes. This is particularly true of LOFAR, a recently-commissioned, low-frequency radio interferometer, based in the Netherlands and with stations across Europe. The identification of and response to transients is one of LOFAR's key science goals. However, the large data volumes which LOFAR produces, combined with the scientific requirement for rapid response, make automation essential. To support this, we have developed the LOFAR Transients Pipeline, or TraP. The TraP ingests multi-frequency image data from LOFAR or other instruments and searches it for transients and variables, providing automatic alerts of significant detections and populating a lightcurve database for further analysis by astronomers. Here, we discuss the scientific goals of the TraP and how it has been designed to meet them. We describe its implementation, including both the algorithms adopted to maximize performance as well as the development methodology used to ensure it is robust and reliable, particularly in the presence of artefacts typical of radio astronomy imaging. Finally, we report on a series of tests of the pipeline carried out using simulated LOFAR observations with a known population of transients.

We present the highest-quality polarisation profiles to date of 16 non-recycled pulsars and four millisecond pulsars, observed below 200 MHz with the LOFAR high-band antennas. Based on the observed profiles, we perform an initial investigation of expected observational effects resulting from the propagation of polarised emission in the pulsar magnetosphere and the interstellar medium. The predictions of magnetospheric birefringence in pulsars have been tested using spectra of the pulse width and fractional polarisation from multifrequency data. The derived spectra offer only partial support for the expected effects of birefringence on the polarisation properties, with only about half of our sample being consistent with the model's predictions. It is noted that for some pulsars these measurements are contaminated by the effects of interstellar scattering. For a number of pulsars in our sample, we have observed significant variations in the amount of Faraday rotation as a function of pulse phase, which is possibly an artefact of scattering. These variations are typically two orders of magnitude smaller than that observed at 1400 MHz by Noutsos et al. (2009), for a different sample of southern pulsars. In this paper we present a possible explanation for the difference in magnitude of this effect between the two frequencies, based on scattering. Finally, we have estimated the magnetospheric emission heights of low-frequency radiation from four pulsars, based on the phase lags between the flux-density and the PA profiles, and the theoretical framework of Blaskiewicz, Cordes & Wasserman (1991). These estimates yielded heights of a few hundred km; at least for PSR B1133+16, this is consistent with emission heights derived based on radius-to-frequency mapping, but is up to a few times larger than the recent upper limit based on pulsar timing.

LOFAR, the LOw-Frequency ARray, is a new-generation radio interferometer constructed in the north of the Netherlands and across europe. Utilizing a novel phased-array design, LOFAR covers the largely unexplored low-frequency range from 10-240 MHz and provides a number of unique observing capabilities. Spreading out from a core located near the village of Exloo in the northeast of the Netherlands, a total of 40 LOFAR stations are nearing completion. A further five stations have been deployed throughout Germany, and one station has been built in each of France, Sweden, and the UK. Digital beam-forming techniques make the LOFAR system agile and allow for rapid repointing of the telescope as well as the potential for multiple simultaneous observations. With its dense core array and long interferometric baselines, LOFAR achieves unparalleled sensitivity and angular resolution in the low-frequency radio regime. The LOFAR facilities are jointly operated by the International LOFAR Telescope (ILT) foundation, as an observatory open to the global astronomical community. LOFAR is one of the first radio observatories to feature automated processing pipelines to deliver fully calibrated science products to its user community. LOFAR's new capabilities, techniques and modus operandi make it an important pathfinder for the Square Kilometre Array (SKA). We give an overview of the LOFAR instrument, its major hardware and software components, and the core science objectives that have driven its design. In addition, we present a selection of new results from the commissioning phase of this new radio observatory.

Dispersion in the interstellar medium is a well known phenomenon that follows a simple relationship, which has been used to predict the time delay of dispersed radio pulses since the late 1960s. We performed wide-band simultaneous observations of four pulsars with LOFAR (at 40-190 MHz), the 76-m Lovell Telescope (at 1400 MHz) and the Effelsberg 100-m Telescope (at 8000 MHz) to test the accuracy of the dispersion law over a broad frequency range. In this paper we present the results of these observations which show that the dispersion law is accurate to better than 1 part in 100000 across our observing band. We use this fact to constrain some of the properties of the ISM along the line-of-sight and use the lack of any aberration or retardation effects to determine upper limits on emission heights in the pulsar magnetosphere. We also discuss the effect of pulse profile evolution on our observations, and the implications that it could have for precision pulsar timing projects such as the detection of gravitational waves with pulsar timing arrays.

Low frequency radio waves, while challenging to observe, are a rich source of information about pulsars. The LOw Frequency ARray (LOFAR) is a new radio interferometer operating in the lowest 4 octaves of the ionospheric "radio window": 10-240MHz, that will greatly facilitate observing pulsars at low radio frequencies. Through the huge collecting area, long baselines, and flexible digital hardware, it is expected that LOFAR will revolutionize radio astronomy at the lowest frequencies visible from Earth. LOFAR is a next-generation radio telescope and a pathfinder to the Square Kilometre Array (SKA), in that it incorporates advanced multi-beaming techniques between thousands of individual elements. We discuss the motivation for low-frequency pulsar observations in general and the potential of LOFAR in addressing these science goals. We present LOFAR as it is designed to perform high-time-resolution observations of pulsars and other fast transients, and outline the various relevant observing modes and data reduction pipelines that are already or will soon be implemented to facilitate these observations. A number of results obtained from commissioning observations are presented to demonstrate the exciting potential of the telescope. This paper outlines the case for low frequency pulsar observations and is also intended to serve as a reference for upcoming pulsar/fast transient science papers with LOFAR.

In this paper we present the results of a survey for radio transients using data obtained from the Very Large Array archive. We have reduced, using a pipeline procedure, 5037 observations of the most common pointings - i.e. the calibrator fields. These fields typically contain a relatively bright point source and are used to calibrate `target' observations: they are therefore rarely imaged themselves. The observations used span a time range ~ 1984 - 2008 and consist of eight different pointings, three different frequencies (8.4, 4.8 and 1.4 GHz) and have a total observing time of 435 hours. We have searched for transient and variable radio sources within these observations using components from the prototype LOFAR transient detection system. In this paper we present the methodology for reducing large volumes of Very Large Array data; and we also present a brief overview of the prototype LOFAR transient detection algorithms. No radio transients were detected in this survey, therefore we place an upper limit on the snapshot rate of GHz frequency transients > 8.0 mJy to rho less than or equal to 0.032 deg^-2 that have typical timescales 4.3 to 45.3 days. We compare and contrast our upper limit with the snapshot rates - derived from either detections or non-detections of transient and variable radio sources - reported in the literature. When compared with the current Log N - Log S distribution formed from previous surveys, we show that our upper limit is consistent with the observed population. Current and future radio transient surveys will hopefully further constrain these statistics, and potentially discover dominant transient source populations. In this paper we also briefly explore the current transient commissioning observations with LOFAR, and the impact they will make on the field.

The 2004 December 27 giant flare from SGR 1806-20 produced a radio nebula that was detectable for weeks. It was observed at a wide range of radio frequencies. We made a total of 19 WSRT observations. Most of these were performed quasi simultaneously at either two or three frequencies, starting 2005 January 4 and ending 2005 January 29. We reobserved the field in 2005 April/May, which facilitated an accurate subtraction of background sources. At 350 MHz, we find that the total intensity of the source is lower than expected from the GMRT 240 MHz and 610 MHz measurements and inconsistent with spectral indices published previously. Our 850 MHz flux densities, however, are consistent with earlier results. There is no compelling evidence for significant depolarization at any frequency. We do, however, find that polarization angles differ substantially from those at higher frequencies. Low frequency polarimetry and total intensity measurements provide a number of clues with regard to substructure in the radio nebula associated with SGR 1806-20. In general, for a more complete understanding of similar events, low frequency observations can provide new insights into the physics of the radio source.

Two WSRT observations were performed and five archival VLA data were reduced in order to redetect the enigmatic radio transient GCRT J1745-3009. The source was not redetected. We were, however, able to extract important new information from the discovery dataset. Our reanalysis excludes models that predict symmetric bursts, but the transient white dwarf pulsar is favoured. Although we now have more contraints on the properties of this source, we are still unsure about its basic model.

Context. Close-in giant extrasolar planets (''Hot Jupiters'') are believed to be strong emitters in the decametric radio range. Aims. We present the expected characteristics of the low-frequency magnetospheric radio emission of all currently known extrasolar planets, including the maximum emission frequency and the expected radio flux. We also discuss the escape of exoplanetary radio emission from the vicinity of its source, which imposes additional constraints on detectability. Methods. We compare the different predictions obtained with all four existing analytical models for all currently known exoplanets. We also take care to use realistic values for all input parameters. Results. The four different models for planetary radio emission lead to very different results. The largest fluxes are found for the magnetic energy model, followed by the CME model and the kinetic energy model (for which our results are found to be much less optimistic than those of previous studies). The unipolar interaction model does not predict any observable emission for the present exoplanet census. We also give estimates for the planetary magnetic dipole moment of all currently known extrasolar planets, which will be useful for other studies. Conclusions. Our results show that observations of exoplanetary radio emission are feasible, but that the number of promising targets is not very high. The catalog of targets will be particularly useful for current and future radio observation campaigns (e.g. with the VLA, GMRT, UTR-2 and with LOFAR).

LOFAR, the Low Frequency Array, is a new radio telescope under construction in the Netherlands, designed to operate between 30 and 240 MHz. The Transients Key Project is one of the four Key Science Projects which comprise the core LOFAR science case. The remit of the Transients Key Project is to study variable and transient radio sources detected by LOFAR, on timescales from milliseconds to years. This will be achieved via both regular snapshot monitoring of historical and newly-discovered radio variables and, most radically, the development of a `Radio Sky Monitor' which will survey a large fraction of the northern sky on a daily basis.