The SunPy Project developed a 13-question survey to understand the software and hardware usage of the solar physics community. 364 members of the solar physics community, across 35 countries, responded to our survey. We found that 99$\pm$0.5% of respondents use software in their research and 66% use the Python scientific software stack. Students are twice as likely as faculty, staff scientists, and researchers to use Python rather than Interactive Data Language (IDL). In this respect, the astrophysics and solar physics communities differ widely: 78% of solar physics faculty, staff scientists, and researchers in our sample uses IDL, compared with 44% of astrophysics faculty and scientists sampled by Momcheva and Tollerud (2015). 63$\pm$4% of respondents have not taken any computer-science courses at an undergraduate or graduate level. We also found that most respondents utilize consumer hardware to run software for solar-physics research. Although 82% of respondents work with data from space-based or ground-based missions, some of which (e.g. the Solar Dynamics Observatory and Daniel K. Inouye Solar Telescope) produce terabytes of data a day, 14% use a regional or national cluster, 5% use a commercial cloud provider, and 29% use exclusively a laptop or desktop. Finally, we found that 73$\pm$4% of respondents cite scientific software in their research, although only 42$\pm$3% do so routinely.

One of the main science motivations for the ESA PLAnetary Transit and Oscillations (PLATO) mission is to measure exoplanet transit radii with 3% precision. In addition to flares and starspots, stellar oscillations and granulation will enforce fundamental noise floors for transiting exoplanet radius measurements. We simulate light curves of Earth-sized exoplanets transiting continuum intensity images of the Sun taken by the HMI instrument aboard SDO to investigate the uncertainties introduced on the exoplanet radius measurements by stellar granulation and oscillations. After modeling the solar variability with a Gaussian process, we find that the amplitude of solar oscillations and granulation is of order 100 ppm -- similar to the depth of an Earth transit -- and introduces a fractional uncertainty on the depth of transit of 0.73% assuming four transits are observed over the mission duration. However, when we translate the depth measurement into a radius measurement of the planet, we find a much larger radius uncertainty of 3.6%. This is due to a degeneracy between the transit radius ratio, the limb-darkening, and the impact parameter caused by the inability to constrain the transit impact parameter in the presence of stellar variability. We find that surface brightness inhomogeneity due to photospheric granulation contributes a lower limit of only 2 ppm to the photometry in-transit. The radius uncertainty due to granulation and oscillations, combined with the degeneracy with the transit impact parameter, accounts for a significant fraction of the error budget of the PLATO mission, before detector or observational noise is introduced to the light curve. If it is possible to constrain the impact parameter or to obtain follow-up observations at longer wavelengths where limb-darkening is less significant, this may enable higher precision radius measurements.

In this paper we present a curated dataset from the NASA Solar Dynamics Observatory (SDO) mission in a format suitable for machine learning research. Beginning from level 1 scientific products we have processed various instrumental corrections, downsampled to manageable spatial and temporal resolutions, and synchronized observations spatially and temporally. We illustrate the use of this dataset with two example applications: forecasting future EVE irradiance from present EVE irradiance and translating HMI observations into AIA observations. For each application we provide metrics and baselines for future model comparison. We anticipate this curated dataset will facilitate machine learning research in heliophysics and the physical sciences generally, increasing the scientific return of the SDO mission. This work is a direct result of the 2018 NASA Frontier Development Laboratory Program. Please see the appendix for access to the dataset.

We explore the connection between starspots and plages of three main-sequence stars by studying the chromospheric and photospheric activity over several rotation periods. We present simultaneous photometry and high-resolution ($R\sim 31,500$) spectroscopy of KIC 9652680, a young, superflare-producing G1 star with a rotation period of 1.4 days. Its Kepler light curve shows rotational modulation consistent with a bright hemisphere followed by a relatively dark hemisphere, generating photometric variability with a semi-amplitude of 4%. We find that KIC 9652680 is darkest when its $S$-index of Ca II H & K emission is at its maximum. We interpret this anti-correlation between flux and $S$ to indicate that dark starspots in the photosphere are co-located with the bright plages in the chromosphere, as they are on the Sun. Moving to lower masses and slower rotators, we present K2 observations with simultaneous spectroscopy of EPIC 211928486 (K5V) and EPIC 211966629 (K4V), two active stars in the 650 Myr-old open cluster Praesepe. The K2 photometry reveals that both stars have rotation periods of 11.7 days; while their flux varies by 1 and 2% respectively, their Ca II H & K $S$-indices seem to hold relatively constant as a function of rotational phase. This suggests that extended chromospheric networks of plages are not concentrated into regions of emission centered on the starspots that drive rotational modulation, unlike KIC 9652680. We also note that the Ca II emission of EPIC 211928486 dipped and recovered suddenly over the duration of one rotation, suggesting that the evolution timescale of plages may be of order the rotation period.

Solar activity, such as flares, produce bursts of high-energy radiation that temporarily enhance the D-region of the ionosphere and attenuate low-frequency radio waves. To track these Sudden Ionospheric Disturbances (SIDs), which disrupt communication signals and perturb satellite orbits, Scherrer et al. (2008) developed an international, ground-based network of around 500 SID monitors that measure the signal strength of low-frequency radio waves. However, these monitors suffer from a host of noise contamination issues that preclude their use for rigorous scientific analysis. As such, we attempt to create an algorithm to automatically identify noisy, contaminated SID data sets from clean ones. To do so, we develop a set of features to characterize times series measurements from SID monitors and use these features, along with a binary classifer called a support vector machine, to automatically assess the quality of the SID data. We compute the True Skill Score, a metric that measures the performance of our classifier, and find that it is ~0.75+/-0.06. We find features characterizing the difference between the daytime and nighttime signal strength of low-frequency radio waves most effectively discern noisy data sets from clean ones.

The precise physical process that triggers solar flares is not currently understood. Here we attempt to capture the signature of this mechanism in solar image data of various wavelengths and use these signatures to predict flaring activity. We do this by developing an algorithm that [1] automatically generates features in 5.5 TB of image data taken by the Solar Dynamics Observatory of the solar photosphere, chromosphere, transition region, and corona during the time period between May 2010 and May 2014, [2] combines these features with other features based on flaring history and a physical understanding of putative flaring processes, and [3] classifies these features to predict whether a solar active region will flare within a time period of $T$ hours, where $T$ = 2 and 24. We find that when optimizing for the True Skill Score (TSS), photospheric vector magnetic field data combined with flaring history yields the best performance, and when optimizing for the area under the precision-recall curve, all the data are helpful. Our model performance yields a TSS of $0.84 \pm 0.03$ and $0.81 \pm 0.03$ in the $T$ = 2 and 24 hour cases, respectively, and a value of $0.13 \pm 0.07$ and $0.43 \pm 0.08$ for the area under the precision-recall curve in the $T$ = 2 and 24 hour cases, respectively. These relatively high scores are similar to, but not greater than, other attempts to predict solar flares. Given the similar values of algorithm performance across various types of models reported in the literature, we conclude that we can expect a certain baseline predictive capacity using these data. This is the first attempt to predict solar flares using photospheric vector magnetic field data as well as multiple wavelengths of image data from the chromosphere, transition region, and corona.

Of all the activity observed on the Sun, two of the most energetic events are flares and Coronal Mass Ejections (CMEs). Usually, solar active regions that produce large flares will also produce a CME, but this is not always true (Yashiro et al., 2005). Despite advances in numerical modeling, it is still unclear which circumstances will produce a CME (Webb & Howard, 2012). Therefore, it is worthwhile to empirically determine which features distinguish flares associated with CMEs from flares that are not. At this time, no extensive study has used physically meaningful features of active regions to distinguish between these two populations. As such, we attempt to do so by using features derived from [1] photospheric vector magnetic field data taken by the Solar Dynamics Observatory's Helioseismic and Magnetic Imager instrument and [2] X-ray flux data from the Geostationary Operational Environmental Satellite's X-ray Flux instrument. We build a catalog of active regions that either produced both a flare and a CME (the positive class) or simply a flare (the negative class). We then use machine-learning algorithms to [1] determine which features distinguish these two populations, and [2] forecast whether an active region that produces an M- or X-class flare will also produce a CME. We compute the True Skill Statistic, a forecast verification metric, and find that it is a relatively high value of approximately 0.8 plus or minus 0.2. We conclude that a combination of six parameters, which are all intensive in nature, will capture most of the relevant information contained in the photospheric magnetic field.

Solar active region (AR) 12192 of October 2014 hosts the largest sunspot group in 24 years. It is the most prolific flaring site of Cycle 24, but surprisingly produced no coronal mass ejection (CME) from the core region during its disk passage. Here, we study the magnetic conditions that prevented eruption and the consequences that ensued. We find AR 12192 to be "big but mild"; its core region exhibits weaker non-potentiality, stronger overlying field, and smaller flare-related field changes compared to two other major flare-CME-productive ARs (11429 and 11158). These differences are present in the intensive-type indices (e.g., means) but generally not the extensive ones (e.g., totals). AR 12192's large amount of magnetic free energy does not translate into CME productivity. The unexpected behavior suggests that AR eruptiveness is limited by some relative measure of magnetic non-potentiality over the restriction of background field, and that confined flares may leave weaker photospheric and coronal imprints compared to their eruptive counterparts.

Time-dependent three-dimensional magnetohydrodynamics (MHD) simulation modules are implemented at the Joint Science Operation Center (JSOC) of Solar Dynamics Observatory (SDO). The modules regularly produce three-dimensional data of the time-relaxed minimum-energy state of the solar corona using global solar-surface magnetic-field maps created from Helioseismic Magnetic Imager (HMI) full-disk magnetogram data. With the assumption of polytropic gas with specific heat ratio of 1.05, three types of simulation products are currently generated: i) simulation data with medium spatial resolution using the definitive calibrated synoptic map of the magnetic field with a cadence of one Carrington rotation, ii) data with low spatial resolution using the definitive version of the synchronic frame format of the magnetic field, with a cadence of one day, and iii) low-resolution data using near-real-time (NRT) synchronic format of the magnetic field on daily basis. The MHD data available in the JSOC database are three-dimensional, covering heliocentric distances from 1.025 to 4.975 solar radii, and contain all eight MHD variables: the plasma density, temperature and three components of motion velocity, and three components of the magnetic field. This article describes details of the MHD simulations as well as the production of the input magnetic-field maps, and details of the products available at the JSOC database interface. In order to assess the merits and limits of the model, we show the simulated data in early 2011 and compare with the actual coronal features observed by the Atmospheric Imaging Assembly (AIA) and the near-Earth in-situ data.

We attempt to forecast M-and X-class solar flares using a machine-learning algorithm, called Support Vector Machine (SVM), and four years of data from the Solar Dynamics Observatory's Helioseismic and Magnetic Imager, the first instrument to continuously map the full-disk photospheric vector magnetic field from space. Most flare forecasting efforts described in the literature use either line-of-sight magnetograms or a relatively small number of ground-based vector magnetograms. This is the first time a large dataset of vector magnetograms has been used to forecast solar flares. We build a catalog of flaring and non-flaring active regions sampled from a database of 2,071 active regions, comprised of 1.5 million active region patches of vector magnetic field data, and characterize each active region by 25 parameters. We then train and test the machine-learning algorithm and we estimate its performances using forecast verification metrics with an emphasis on the True Skill Statistic (TSS). We obtain relatively high TSS scores and overall predictive abilities. We surmise that this is partly due to fine-tuning the SVM for this purpose and also to an advantageous set of features that can only be calculated from vector magnetic field data. We also apply a feature selection algorithm to determine which of our 25 features are useful for discriminating between flaring and non-flaring active regions and conclude that only a handful are needed for good predictive abilities.

The Helioseismic and Magnetic Imager (HMI) began near-continuous full-disk solar measurements on 1 May 2010 from the Solar Dynamics Observatory (SDO). An automated processing pipeline keeps pace with observations to produce observable quantities, including the photospheric vector magnetic field, from sequences of filtergrams. The primary 720s observables were released in mid 2010, including Stokes polarization parameters measured at six wavelengths as well as intensity, Doppler velocity, and the line-of-sight magnetic field. More advanced products, including the full vector magnetic field, are now available. Automatically identified HMI Active Region Patches (HARPs) track the location and shape of magnetic regions throughout their lifetime. The vector field is computed using the Very Fast Inversion of the Stokes Vector (VFISV) code optimized for the HMI pipeline; the remaining 180 degree azimuth ambiguity is resolved with the Minimum Energy (ME0) code. The Milne-Eddington inversion is performed on all full-disk HMI observations. The disambiguation, until recently run only on HARP regions, is now implemented for the full disk. Vector and scalar quantities in the patches are used to derive active region indices potentially useful for forecasting; the data maps and indices are collected in the SHARP data series, hmi.sharp_720s. Patches are provided in both CCD and heliographic coordinates. HMI provides continuous coverage of the vector field, but has modest spatial, spectral, and temporal resolution. Coupled with limitations of the analysis and interpretation techniques, effects of the orbital velocity, and instrument performance, the resulting measurements have a certain dynamic range and sensitivity and are subject to systematic errors and uncertainties that are characterized in this report.

A new data product from the Helioseismic and Magnetic Imager (HMI) onboard the Solar Dynamics Observatory (SDO) called Space-weather HMI Active Region Patches (SHARPs) is now available. SDO/HMI is the first space-based instrument to map the full-disk photospheric vector magnetic field with high cadence and continuity. The SHARP data series provide maps in patches that encompass automatically tracked magnetic concentrations for their entire lifetime; map quantities include the photospheric vector magnetic field and its uncertainty, along with Doppler velocity, continuum intensity, and line-of-sight magnetic field. Furthermore, keywords in the SHARP data series provide several parameters that concisely characterize the magnetic-field distribution and its deviation from a potential-field configuration. These indices may be useful for active-region event forecasting and for identifying regions of interest. The indices are calculated per patch and are available on a twelve-minute cadence. Quick-look data are available within approximately three hours of observation; definitive science products are produced approximately five weeks later. SHARP data are available at http://jsoc.stanford.edu and maps are available in either of two different coordinate systems. This article describes the SHARP data products and presents examples of SHARP data and parameters.