Machine learning is nowadays the methodology of choice for flare forecasting and supervised techniques, in both their traditional and deep versions, are becoming the most frequently used ones for prediction in this area of space weather. Yet, machine learning has not been able so far to realize an operating warning system for flaring storms and the scientific literature of the last decade suggests that its performances in the prediction of intense solar flares are not optimal. The main difficulties related to forecasting solar flaring storms are probably two. First, most methods are conceived to provide probabilistic predictions and not to send binary yes/no indications on the consecutive occurrence of flares along an extended time range. Second, flaring storms are typically characterized by the explosion of high energy events, which are seldom recorded in the databases of space missions; as a consequence, supervised methods are trained on very imbalanced historical sets, which makes them particularly ineffective for the forecasting of intense flares. Yet, in this study we show that supervised machine learning could be utilized in a way to send timely warnings about the most violent and most unexpected flaring event of the last decade, and even to predict with some accuracy the energy budget daily released by magnetic reconnection during the whole time course of the storm. Further, we show that the combination of sparsity-enhancing machine learning and feature ranking could allow the identification of the prominent role that energy played as an Active Region property in the forecasting process.

Maximum Entropy is an image reconstruction method conceived to image a sparsely occupied field of view and therefore particularly appropriate to achieve super-resolution effects. Although widely used in image deconvolution, this method has been formulated in radio astronomy for the analysis of observations in the spatial frequency domain, and an Interactive Data Language (IDL) code has been implemented for image reconstruction from solar X-ray Fourier data. However, this code relies on a non-convex formulation of the constrained optimization problem addressed by the Maximum Entropy approach and this sometimes results in unreliable reconstructions characterized by unphysical shrinking effects. This paper introduces a new approach to Maximum Entropy based on the constrained minimization of a convex functional. In the case of observations recorded by the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI), the resulting code provides the same super-resolution effects of the previous algorithm, while working properly also when that code produces unphysical reconstructions. Results are also provided of testing the algorithm with synthetic data simulating observations of the Spectrometer/Telescope for Imaging X-rays (STIX) in Solar Orbiter. The new code is available in the {\em{HESSI}} folder of the Solar SoftWare (SSW)tree.

Solar flares originate from magnetically active regions but not all solar active regions give rise to a flare. Therefore, the challenge of solar flare prediction benefits by an intelligent computational analysis of physics-based properties extracted from active region observables, most commonly line-of-sight or vector magnetograms of the active-region photosphere. For the purpose of flare forecasting, this study utilizes an unprecedented 171 flare-predictive active region properties, mainly inferred by the Helioseismic and Magnetic Imager onboard the Solar Dynamics Observatory (SDO/HMI) in the course of the European Union Horizon 2020 FLARECAST project. Using two different supervised machine learning methods that allow feature ranking as a function of predictive capability, we show that: i) an objective training and testing process is paramount for the performance of every supervised machine learning method; ii) most properties include overlapping information and are therefore highly redundant for flare prediction; iii) solar flare prediction is still - and will likely remain - a predominantly probabilistic challenge.

Image saturation has been an issue for several instruments in solar astronomy, mainly at EUV wavelengths. However, with the launch of the Atmospheric Imaging Assembly (AIA) as part of the payload of the Solar Dynamic Observatory (SDO) image saturation has become a big data issue, involving around 10^$ frames of the impressive dataset this beautiful telescope has been providing every year since February 2010. This paper introduces a novel desaturation method, which is able to recover the signal in the saturated region of any AIA image by exploiting no other information but the one contained in the image itself. This peculiar methodological property, jointly with the unprecedented statistical reliability of the desaturated images, could make this algorithm the perfect tool for the realization of a reconstruction pipeline for AIA data, able to work properly even in the case of long-lasting, very energetic flaring events.

The Spectrometer/Telescope for Imaging X-rays (STIX) will look at solar flares across the hard X-ray window provided by the Solar Orbiter cluster. Similarly to the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI), STIX is a visibility-based imaging instrument, which will ask for Fourier-based image reconstruction methods. However, in this paper we show that, as for RHESSI, also for STIX count-based imaging is possible. Specifically, here we introduce and illustrate a mathematical model that mimics the STIX data formation process as a projection from the incoming photon flux into a vector made of 120 count components. Then we test the reliability of Expectation Maximization for image reconstruction in the case of several simulated configurations typical of flare morphology.

We describe two inversion methods for the reconstruction of hard X-ray solar images. The methods are tested against experimental visibilities recorded by the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI) and synthetic visibilities based on the design of the Spectrometer/Telescope for Imaging X-rays (STIX).

We describe here the application of a machine learning method for flare forecasting using vectors of properties extracted from images provided by the Helioseismic and Magnetic Imager in the Solar Dynamics Observatory (SDO/HMI). We also discuss how the method can be used to quantitatively assess the impact of such properties on the prediction process.

This paper reports on the re-analysis of solar flares in which the hard X-rays (HXRs) come predominantly from the corona rather than from the more usual chromospheric footpoints. All of the 26 previously analyzed event time intervals, over 13 flares, are re-examined for consistency with a flare model in which electrons are accelerated near the top of a magnetic loop that has a sufficiently high density to stop most of the electrons by Coulomb collisions before they can reach the footpoints. Of particular importance in the previous analysis was the finding that the length of the coronal HXR source increased with energy in the 20 - 30 keV range. However, after allowing for the possibility that footpoint emission at the higher energies affects the inferred length of the coronal HXR source, and using analysis techniques that suppress the possible influence of such footpoint emission, we conclude that there is no longer evidence that the length of the HXR coronal sources increase with increasing energy. In fact, for the 6 flares and 12 time intervals that satisfied our selection criteria, the loop lengths decreased on average by 1.0 +/- 0.2 arcsec between 20 and 30 keV, with a standard deviation of 3.5 arcsec. We find strong evidence that the peak of the coronal HXR source increases in altitude with increasing energy. For the thermal component of the emission, this is consistent with the standard CHSKP flare model in which magnetic reconnection in a coronal current sheet results in new hot loops being formed at progressively higher altitudes. The explanation for the nonthermal emission is not so clear.

'Flare Likelihood and Region Eruption Forecasting (FLARECAST)' is a Horizon 2020 project, which realized a technological platform for machine learning algorithms, with the objective of providing the space weather community with a prediction service for solar flares. This paper describes the FLARECAST service and shows how the methods implemented in the platform allow both flare prediction and a quantitative assessment of how the information contained in the space data utilized in the analysis may impact the forecasting process.

The hard X-ray emission in a solar flare is typically characterized by a number of discrete sources, each with its own spectral, temporal, and spatial variability. Establishing the relationship amongst these sources is critical to determine the role of each in the energy release and transport processes that occur within the flare. In this paper we present a novel method to identify and characterize each source of hard X-ray emission. The method permits a quantitative determination of the most likely number of subsources present, and of the relative probabilities that the hard X-ray emission in a given subregion of the flare is represented by a complicated multiple source structure or by a simpler single source. We apply the method to a well-studied flare on 2002~February~20 in order to assess competing claims as to the number of chromospheric footpoint sources present, and hence to the complexity of the underlying magnetic geometry/toplogy. Contrary to previous claims of the need for multiple sources to account for the chromospheric hard X-ray emission at different locations and times, we find that a simple two-footpoint-plus-coronal-source model is the most probable explanation for the data. We also find that one of the footpoint sources moves quite rapidly throughout the event, a factor that presumably complicated previous analyses. The inferred velocity of the footpoint corresponds to a very high induced electric field, compatible with those in thin reconnecting current sheets.

This paper shows that compressed sensing realized by means of regularized deconvolution and the Finite Isotropic Wavelet Transform is effective and reliable in hard X-ray solar imaging. The method utilizes the Finite Isotropic Wavelet Transform with Meyer function as the mother wavelet. Further, compressed sensing is realized by optimizing a sparsity-promoting regularized objective function by means of the Fast Iterative Shrinkage-Thresholding Algorithm. Eventually, the regularization parameter is selected by means of the Miller criterion. The method is applied against both synthetic data mimicking the Spectrometer/Telescope Imaging X-rays (STIX) measurements and experimental observations provided by the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI). The performances of the method are compared with the results provided by standard visibility-based reconstruction methods. The results show that the application of the sparsity constraint and the use of a continuous, isotropic framework for the wavelet transform provide a notable spatial accuracy and significantly reduce the ringing effects due to the instrument point spread functions.

We introduce a hybrid approach to solar flare prediction, whereby a supervised regularization method is used to realize feature importance and an unsupervised clustering method is used to realize the binary flare/no-flare decision. The approach is validated against NOAA SWPC data.

How impulsive magnetic energy release leads to solar eruptions and how those eruptions are energized and evolve are vital unsolved problems in Heliophysics. The standard model for solar eruptions summarizes our current understanding of these events. Magnetic energy in the corona is released through drastic restructuring of the magnetic field via reconnection. Electrons and ions are then accelerated by poorly understood processes. Theories include contracting loops, merging magnetic islands, stochastic acceleration, and turbulence at shocks, among others. Although this basic model is well established, the fundamental physics is poorly understood. HXR observations using grazing-incidence focusing optics can now probe all of the key regions of the standard model. These include two above-the-looptop (ALT) sources which bookend the reconnection region and are likely the sites of particle acceleration and direct heating. The science achievable by a direct HXR imaging instrument can be summarized by the following science questions and objectives which are some of the most outstanding issues in solar physics (1) How are particles accelerated at the Sun? (1a) Where are electrons accelerated and on what time scales? (1b) What fraction of electrons is accelerated out of the ambient medium? (2) How does magnetic energy release on the Sun lead to flares and eruptions? A Focusing Optics X-ray Solar Imager (FOXSI) instrument, which can be built now using proven technology and at modest cost, would enable revolutionary advancements in our understanding of impulsive magnetic energy release and particle acceleration, a process which is known to occur at the Sun but also throughout the Universe.

Saturation affects a significant rate of images recorded by the Atmospheric Imaging Assembly on the Solar Dynamics Observatory. This paper describes a computational method and a technological pipeline for the de-saturation of such images, based on several mathematical ingredients like Expectation Maximization, image correlation and interpolation. An analysis of the computational properties and demands of the pipeline, together with an assessment of its reliability are performed against a set of data recorded from the Feburary 25 2014 flaring event.

The Atmospheric Imaging Assembly in the Solar Dynamics Observatory provides full Sun images every 1 seconds in each of 7 Extreme Ultraviolet passbands. However, for a significant amount of these images, saturation affects their most intense core, preventing scientists from a full exploitation of their physical meaning. In this paper we describe a mathematical and automatic procedure for the recovery of information in the primary saturation region based on a correlation/inversion analysis of the diffraction pattern associated to the telescope observations. Further, we suggest an interpolation-based method for determining the image background that allows the recovery of information also in the region of secondary saturation (blooming).

The Spectrometer/Telescope for Imaging X-rays (STIX) is a hard X-ray imaging spectroscopy device to be mounted in the Solar Orbiter cluster with the aim of providing images and spectra of solar flaring regions at different photon energies in the range from a few keV to around 150 keV. The imaging modality of this telescope is based on the Moire pattern concept and utilizes 30 sub-collimators, each one containing a pair of co-axial grids. This paper applies Fourier analysis to provide the first rigorous description of the data formation process in STIX. Specifically, we show that, under first harmonic approximation, the integrated counts measured by STIX sub-collimators can be interpreted as specific spatial Fourier components of the incoming photon flux, named visibilities. Fourier analysis also allows the quantitative assessment of the reliability of such interpretation. The description of STIX data in terms of visibilities has a notable impact on the image reconstruction process, since it fosters the application of Fourier-based imaging algorithms.

EUV images of solar flares provided by the Atmospheric Imaging Assembly onboard the Solar Dynamics Observatory (SDO/AIA) are often affected by saturation effects in their core, physically most interesting region. We introduce an image reconstruction procedure that allows recovering information in the primary saturation domain using the secondary images produced by the diffraction fringes as input data. Such a procedure is based on standard image-processing tools like correlation, convolution, and back-projection. Its effectiveness is tested in the case of SDO/AIA observations of the July 8 2013 flaring event.

According to a standard ohmic perspective, the injection of accelerated electrons into the flaring region violates local charge equilibrium and therefore, in response, return currents are driven by an electric field to equilibrate such charge violation. In this framework, the energy loss rate associated to these local currents has an ohmic nature and significantly shortens the acceleration electron path. In the present paper we adopt a different viewpoint and, specifically, we study the impact of the background drift velocity on the energy loss rate of accelerated electrons in solar flares. We first utilize the Rutherford cross-section to derive the formula of the energy loss rate when the collisional target has a finite temperature and the background instantaneously and coherently moves up to equilibrate the electron injection. We then use the continuity equation for electrons and imaging spectroscopy data provided by RHESSI to validate this model. Specifically, we show that this new formula for the energy loss rate provides a better fit of the experimental data with respect to the model based on the effects of standard ohmic return currents.

We analyze electron flux maps based on RHESSI hard X-ray imaging spectroscopy data for a number of extended coronal loop flare events. For each event, we determine the variation of the characteristic loop length $L$ with electron energy $E$, and we fit this observed behavior with models that incorporate an extended acceleration region and an exterior "propagation" region, and which may include collisional modification of the accelerated electron spectrum inside the acceleration region. The models are characterized by two parameters: the plasma density $n$ in, and the longitudinal extent $L_0$ of, the acceleration region. Determination of the best-fit values of these parameters permits inference of the volume that encompasses the acceleration region and of the total number of particles within it. It is then straightforward to compute values for the emission filling factor and for the {\it specific acceleration rate} (electrons s$^{-1}$ per ambient electron above a chosen reference energy). For the 24 events studied, the range of inferred filling factors is consistent with a value of unity. The inferred mean value of the specific acceleration rate above $E_0=20$ keV is $\sim10^{-2}$ s$^{-1}$, with a 1$\sigma$ spread of about a half-order-of-magnitude above and below this value. We compare these values with the predictions of several models, including acceleration by large-scale, weak (sub-Dreicer) fields, by strong (super-Dreicer) electric fields in a reconnecting current sheet, and by stochastic acceleration processes.

This paper is concerned with the image reconstruction problem when the measured data are solar hard X-ray modulation profiles obtained from the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI)} instrument. Our goal is to demonstrate that a statistical iterative method classically applied to the image deconvolution problem is very effective when utilized for the analysis of count modulation profiles in solar hard X-ray imaging based on Rotating Modulation Collimators. The algorithm described in this paper solves the maximum likelihood problem iteratively and encoding a positivity constraint into the iterative optimization scheme. The result is therefore a classical Expectation Maximization method this time applied not to an image deconvolution problem but to image reconstruction from count modulation profiles. The technical reason that makes our implementation particularly effective in this application is the use of a very reliable stopping rule which is able to regularize the solution providing, at the same time, a very satisfactory Cash-statistic (C-statistic). The method is applied to both reproduce synthetic flaring configurations and reconstruct images from experimental data corresponding to three real events. In this second case, the performance of Expectation Maximization, when compared to Pixon image reconstruction, shows a comparable accuracy and a notably reduced computational burden; when compared to CLEAN, shows a better fidelity with respect to the measurements with a comparable computational effectiveness. If optimally stopped, Expectation Maximization represents a very reliable method for image reconstruction in the RHESSI context when count modulation profiles are used as input data.

In this paper we propose a new statistical stopping rule for constrained maximum likelihood iterative algorithms applied to ill-posed inverse problems. To this aim we extend the definition of Tikhonov regularization in a statistical framework and prove that the application of the proposed stopping rule to the Iterative Space Reconstruction Algorithm (ISRA) in the Gaussian case and Expectation Maximization (EM) in the Poisson case leads to well defined regularization methods according to the given definition. We also prove that, if an inverse problem is genuinely ill-posed in the sense of Tikhonov, the same definition is not satisfied when ISRA and EM are optimized by classical stopping rule like Morozov's discrepancy principle, Pearson's test and Poisson discrepancy principle. The stopping rule is illustrated in the case of image reconstruction from data recorded by the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI). First, by using a simulated image consisting of structures analogous to those of a real solar flare we validate the fidelity and accuracy with which the proposed stopping rule recovers the input image. Second, the robustness of the method is compared with the other classical stopping rules and its advantages are shown in the case of real data recorded by RHESSI during two different flaring events.

Using {\em RHESSI} hard X-ray imaging spectroscopy observations, we analyze electron flux maps for a number of extended coronal loop flares. For each event, we fit a collisional model with an extended acceleration region to the observed variation of loop length with electron energy $E$, resulting in estimates of the plasma density in, and longitudinal extent of, the acceleration region. These quantities in turn allow inference of the number of particles within the acceleration region and hence the filling factor $f$ -- the ratio of the emitting volume to the volume that encompasses the emitting region(s). We obtain values of $f$ that lie mostly between 0.1 and 1.0; the (geometric) mean value is $f = 0.20 \times \div 3.9$, somewhat less than, but nevertheless consistent with, unity. Further, coupling information on the number of particles in the acceleration region with information on the total rate of acceleration of particles above a certain reference energy (obtained from spatially-integrated hard X-ray data) also allows inference of the specific acceleration rate (electron s$^{-1}$ per ambient electron above the chosen reference energy). We obtain a (geometric) mean value of the specific acceleration rate $\eta(20$ keV) $ = (6.0 \times / \div 3.4) \times 10^{-3}$ electrons s$^{-1}$ per ambient electron; this value has implications both for the global electrodynamics associated with replenishment of the acceleration region and for the nature of the particle acceleration process.

In order to study the acceleration and propagation of bremsstrahlung-producing electrons in solar flares, we analyze the evolution of the flare loop size with respect to energy at a variety of times. A GOES M3.7 loop-structured flare starting around 23:55 on 2002 April 14 is studied in detail using \textit{Ramaty High Energy Solar Spectroscopic Imager} (\textit{RHESSI}) observations. We construct photon and mean-electron-flux maps in 2-keV energy bins by processing observationally-deduced photon and electron visibilities, respectively, through several image-processing methods: a visibility-based forward-fit (FWD) algorithm, a maximum entropy (MEM) procedure and the uv-smooth (UVS) approach. We estimate the sizes of elongated flares (i.e., the length and width of flaring loops) by calculating the second normalized moments of the intensity in any given map. Employing a collisional model with an extended acceleration region, we fit the loop lengths as a function of energy in both the photon and electron domains. The resulting fitting parameters allow us to estimate the extent of the acceleration region which is between $\sim 13 \rm{arcsec}$ and $\sim 19 \rm{arcsec}$. Both forward-fit and uv-smooth algorithms provide substantially similar results with a systematically better fit in the electron domain.The consistency of the estimates from these methods provides strong support that the model can reliably determine geometric parameters of the acceleration region. The acceleration region is estimated to be a substantial fraction ($\sim 1/2$) of the loop extent, indicating that this dense flaring loop incorporates both acceleration and transport of electrons, with concurrent thick-target bremsstrahlung emission.

X-radiation from energetic electrons is the prime diagnostic of flare-accelerated electrons. The observed X-ray flux (and polarization state) is fundamentally a convolution of the cross-section for the hard X-ray emission process(es) in question with the electron distribution function, which is in turn a function of energy, direction, spatial location and time. To address the problems of particle propagation and acceleration one needs to infer as much information as possible on this electron distribution function, through a deconvolution of this fundamental relationship. This review presents recent progress toward this goal using spectroscopic, imaging and polarization measurements, primarily from the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI). Previous conclusions regarding the energy, angular (pitch angle) and spatial distributions of energetic electrons in solar flares are critically reviewed. We discuss the role and the observational evidence of several radiation processes: free-free electron-ion, free-free electron-electron, free-bound electron-ion bremsstrahlung, photoelectric absorption and Compton back-scatter (albedo), using both spectroscopic and imaging techniques. This unprecedented quality of data allows for the first time inference of the angular distributions of the X-ray-emitting electrons using albedo, improved model-independent inference of electron energy spectra and emission measures of thermal plasma. Moreover, imaging spectroscopy has revealed hitherto unknown details of solar flare morphology and detailed spectroscopy of coronal, footpoint and extended sources in flaring regions. Additional attempts to measure hard X-ray polarization were not sufficient to put constraints on the degree of anisotropy of electrons, but point to the importance of obtaining good quality polarization data.

Hard X-Ray bremsstrahlung continuum spectra, such as from solar flares, are commonly described in terms of power-law fits, either to the photon spectra themselves or to the electron spectra responsible for them. In applications various approximate relations between electron and photon spectral indices are often used for energies both above and below electron low-energy cutoffs. We examine the form of the exact relationships in various situations, and for various cross-sections, showing that empirical relations sometimes used can be highly misleading and consider how to improve fitting procedures. We obtain expressions for photon spectra from single, double and truncated power-law electron spectra for a variety of cross-sections and for the thin and thick target models and simple analytic expressions for the Bethe-Heitler cases. We show that above a low-energy cutoff the Kramers and Bethe-Heitler results match reasonably well with results for exact cross-sections up to energies around 100 keV; that below the low-energy cutoff, Kramers and other constant spectral index forms commonly used are very poor approximations to accurate results; but that our analytical forms are a very good match. Analytical forms of the Bethe-Heitler photon spectra from general power-law electron spectra are an excellent match to exact results for both thin and thick targets and they enable much faster spectral fitting than evaluation of the full spectral integrations.

Although both electron-ion and electron-electron bremsstrahlung contribute to the hard X-ray emission from solar flares, the latter is normally ignored. Such an omission is not justified at electron (and photon) energies above $\sim 300$ keV, and inclusion of the additional electron-electron bremsstrahlung in general makes the electron spectrum required to produce a given hard X-ray spectrum steeper at high energies. Unlike electron-ion bremsstrahlung, electron-electron bremsstrahlung cannot produce photons of all energies up to the maximum electron energy involved. The maximum possible photon energy depends on the angle between the direction of the emitting electron and the emitted photon, and this suggests a diagnostic for an upper cutoff energy and/or for the degree of beaming of the accelerated electrons. We analyze the large event of January 17, 2005 observed by RHESSI and show that the upward break around 400 keV in the observed hard X-ray spectrum is naturally accounted for by the inclusion of electron-electron bremsstrahlung. Indeed, the mean source electron spectrum recovered through a regularized inversion of the hard X-ray spectrum, using a cross-section that includes both electron-ion and electron-electron terms, has a relatively constant spectral index $\delta$ over the range from electron kinetic energy $E = 200$ keV to $E = 1$ MeV. However, the level of detail discernible in the recovered electron spectrum is not sufficient to determine whether or not any upper cutoff energy exists.

Kontar et al. (2004) have shown how to recover mean source electron spectra in solar flares through a physical constraint regularization analysis of the bremsstrahlung photon spectra that they produce. They emphasize the use of non-square inversion techniques, and preconditioning combined with physical properties of the spectra to achieve the most meaningful solution to the problem. Higher-order regularization techniques may be used to generate ${overline F}(E)$ forms with certain desirable properties (e.g., higher order derivatives). They further note that such analyses may be used to infer properties of the electron energy spectra at energies well above the maximum photon energy observed. In this paper we apply these techniques to data from a solar flare observed by RHESSI on 26 February, 2002. Results using different orders of regularization are presented and compared for various time intervals. Clear evidence is presented for a change in the value of the high-energy cutoff in the mean source electron spectrum with time. We also show how the construction of the injected (accelerated) electron spectrum $F_0(E_0)$ (assuming that Coulomb collisions in a cold target dominate the electron energetics) is facilitated by the use of higher-order regularization methods.

Hard X-ray spectra in solar flares provide knowledge of the electron spectrum that results from acceleration and propagation in the solar atmosphere. However, the inference of the electron spectra from solar X-ray spectra is an ill-posed inverse problem. Here we develop and apply an enhanced regularization algorithm for this process making use of physical constraints on the form of the electron spectrum. The algorithm incorporates various features not heretofore employed in the solar flare context: Generalized Singular Value Decomposition (GSVD) to deal with different orders of constraints; rectangular form of the cross-section matrix to extend the solution energy range; regularization with various forms of the smoothing operator; and "preconditioning" of the problem. We show by simulations that this technique yields electron spectra with considerably more information and higher quality than previous algorithms.