Machine learning is nowadays the methodology of choice for flare forecasting and supervised techniques, in both their traditional and deep versions, are becoming the most frequently used ones for prediction in this area of space weather. Yet, machine learning has not been able so far to realize an operating warning system for flaring storms and the scientific literature of the last decade suggests that its performances in the prediction of intense solar flares are not optimal. The main difficulties related to forecasting solar flaring storms are probably two. First, most methods are conceived to provide probabilistic predictions and not to send binary yes/no indications on the consecutive occurrence of flares along an extended time range. Second, flaring storms are typically characterized by the explosion of high energy events, which are seldom recorded in the databases of space missions; as a consequence, supervised methods are trained on very imbalanced historical sets, which makes them particularly ineffective for the forecasting of intense flares. Yet, in this study we show that supervised machine learning could be utilized in a way to send timely warnings about the most violent and most unexpected flaring event of the last decade, and even to predict with some accuracy the energy budget daily released by magnetic reconnection during the whole time course of the storm. Further, we show that the combination of sparsity-enhancing machine learning and feature ranking could allow the identification of the prominent role that energy played as an Active Region property in the forecasting process.

Maximum Entropy is an image reconstruction method conceived to image a sparsely occupied field of view and therefore particularly appropriate to achieve super-resolution effects. Although widely used in image deconvolution, this method has been formulated in radio astronomy for the analysis of observations in the spatial frequency domain, and an Interactive Data Language (IDL) code has been implemented for image reconstruction from solar X-ray Fourier data. However, this code relies on a non-convex formulation of the constrained optimization problem addressed by the Maximum Entropy approach and this sometimes results in unreliable reconstructions characterized by unphysical shrinking effects. This paper introduces a new approach to Maximum Entropy based on the constrained minimization of a convex functional. In the case of observations recorded by the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI), the resulting code provides the same super-resolution effects of the previous algorithm, while working properly also when that code produces unphysical reconstructions. Results are also provided of testing the algorithm with synthetic data simulating observations of the Spectrometer/Telescope for Imaging X-rays (STIX) in Solar Orbiter. The new code is available in the {\em{HESSI}} folder of the Solar SoftWare (SSW)tree.

Solar flares originate from magnetically active regions but not all solar active regions give rise to a flare. Therefore, the challenge of solar flare prediction benefits by an intelligent computational analysis of physics-based properties extracted from active region observables, most commonly line-of-sight or vector magnetograms of the active-region photosphere. For the purpose of flare forecasting, this study utilizes an unprecedented 171 flare-predictive active region properties, mainly inferred by the Helioseismic and Magnetic Imager onboard the Solar Dynamics Observatory (SDO/HMI) in the course of the European Union Horizon 2020 FLARECAST project. Using two different supervised machine learning methods that allow feature ranking as a function of predictive capability, we show that: i) an objective training and testing process is paramount for the performance of every supervised machine learning method; ii) most properties include overlapping information and are therefore highly redundant for flare prediction; iii) solar flare prediction is still - and will likely remain - a predominantly probabilistic challenge.

Image saturation has been an issue for several instruments in solar astronomy, mainly at EUV wavelengths. However, with the launch of the Atmospheric Imaging Assembly (AIA) as part of the payload of the Solar Dynamic Observatory (SDO) image saturation has become a big data issue, involving around 10^$ frames of the impressive dataset this beautiful telescope has been providing every year since February 2010. This paper introduces a novel desaturation method, which is able to recover the signal in the saturated region of any AIA image by exploiting no other information but the one contained in the image itself. This peculiar methodological property, jointly with the unprecedented statistical reliability of the desaturated images, could make this algorithm the perfect tool for the realization of a reconstruction pipeline for AIA data, able to work properly even in the case of long-lasting, very energetic flaring events.

The Spectrometer/Telescope for Imaging X-rays (STIX) will look at solar flares across the hard X-ray window provided by the Solar Orbiter cluster. Similarly to the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI), STIX is a visibility-based imaging instrument, which will ask for Fourier-based image reconstruction methods. However, in this paper we show that, as for RHESSI, also for STIX count-based imaging is possible. Specifically, here we introduce and illustrate a mathematical model that mimics the STIX data formation process as a projection from the incoming photon flux into a vector made of 120 count components. Then we test the reliability of Expectation Maximization for image reconstruction in the case of several simulated configurations typical of flare morphology.

We describe here the application of a machine learning method for flare forecasting using vectors of properties extracted from images provided by the Helioseismic and Magnetic Imager in the Solar Dynamics Observatory (SDO/HMI). We also discuss how the method can be used to quantitatively assess the impact of such properties on the prediction process.

'Flare Likelihood and Region Eruption Forecasting (FLARECAST)' is a Horizon 2020 project, which realized a technological platform for machine learning algorithms, with the objective of providing the space weather community with a prediction service for solar flares. This paper describes the FLARECAST service and shows how the methods implemented in the platform allow both flare prediction and a quantitative assessment of how the information contained in the space data utilized in the analysis may impact the forecasting process.

We propose a forecasting approach for solar flares based on data from Solar Cycle 24, taken by the Helioseismic and Magnetic Imager (HMI) on board the Solar Dynamics Observatory (SDO) mission. In particular, we use the Space-weather HMI Active Region Patches (SHARP) product that facilitates cut-out magnetograms of solar active regions (AR) in the Sun in near-real-time (NRT), taken over a five-year interval (2012 - 2016). Our approach utilizes a set of thirteen predictors, which are not included in the SHARP metadata, extracted from line-of-sight and vector photospheric magnetograms. We exploit several Machine Learning (ML) and Conventional Statistics techniques to predict flares of peak magnitude >M1 and >C1, within a 24 h forecast window. The ML methods used are multi-layer perceptrons (MLP), support vector machines (SVM) and random forests (RF). We conclude that random forests could be the prediction technique of choice for our sample, with the second best method being multi-layer perceptrons, subject to an entropy objective function. A Monte Carlo simulation showed that the best performing method gives accuracy ACC=0.93(0.00), true skill statistic TSS=0.74(0.02) and Heidke skill score HSS=0.49(0.01) for >M1 flare prediction with probability threshold 15% and ACC=0.84(0.00), TSS=0.60(0.01) and HSS=0.59(0.01) for >C1 flare prediction with probability threshold 35%.

We introduce a hybrid approach to solar flare prediction, whereby a supervised regularization method is used to realize feature importance and an unsupervised clustering method is used to realize the binary flare/no-flare decision. The approach is validated against NOAA SWPC data.

The Atmospheric Imaging Assembly in the Solar Dynamics Observatory provides full Sun images every 1 seconds in each of 7 Extreme Ultraviolet passbands. However, for a significant amount of these images, saturation affects their most intense core, preventing scientists from a full exploitation of their physical meaning. In this paper we describe a mathematical and automatic procedure for the recovery of information in the primary saturation region based on a correlation/inversion analysis of the diffraction pattern associated to the telescope observations. Further, we suggest an interpolation-based method for determining the image background that allows the recovery of information also in the region of secondary saturation (blooming).

This paper is concerned with the image reconstruction problem when the measured data are solar hard X-ray modulation profiles obtained from the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI)} instrument. Our goal is to demonstrate that a statistical iterative method classically applied to the image deconvolution problem is very effective when utilized for the analysis of count modulation profiles in solar hard X-ray imaging based on Rotating Modulation Collimators. The algorithm described in this paper solves the maximum likelihood problem iteratively and encoding a positivity constraint into the iterative optimization scheme. The result is therefore a classical Expectation Maximization method this time applied not to an image deconvolution problem but to image reconstruction from count modulation profiles. The technical reason that makes our implementation particularly effective in this application is the use of a very reliable stopping rule which is able to regularize the solution providing, at the same time, a very satisfactory Cash-statistic (C-statistic). The method is applied to both reproduce synthetic flaring configurations and reconstruct images from experimental data corresponding to three real events. In this second case, the performance of Expectation Maximization, when compared to Pixon image reconstruction, shows a comparable accuracy and a notably reduced computational burden; when compared to CLEAN, shows a better fidelity with respect to the measurements with a comparable computational effectiveness. If optimally stopped, Expectation Maximization represents a very reliable method for image reconstruction in the RHESSI context when count modulation profiles are used as input data.

In this paper we propose a new statistical stopping rule for constrained maximum likelihood iterative algorithms applied to ill-posed inverse problems. To this aim we extend the definition of Tikhonov regularization in a statistical framework and prove that the application of the proposed stopping rule to the Iterative Space Reconstruction Algorithm (ISRA) in the Gaussian case and Expectation Maximization (EM) in the Poisson case leads to well defined regularization methods according to the given definition. We also prove that, if an inverse problem is genuinely ill-posed in the sense of Tikhonov, the same definition is not satisfied when ISRA and EM are optimized by classical stopping rule like Morozov's discrepancy principle, Pearson's test and Poisson discrepancy principle. The stopping rule is illustrated in the case of image reconstruction from data recorded by the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI). First, by using a simulated image consisting of structures analogous to those of a real solar flare we validate the fidelity and accuracy with which the proposed stopping rule recovers the input image. Second, the robustness of the method is compared with the other classical stopping rules and its advantages are shown in the case of real data recorded by RHESSI during two different flaring events.

In order to study the acceleration and propagation of bremsstrahlung-producing electrons in solar flares, we analyze the evolution of the flare loop size with respect to energy at a variety of times. A GOES M3.7 loop-structured flare starting around 23:55 on 2002 April 14 is studied in detail using \textit{Ramaty High Energy Solar Spectroscopic Imager} (\textit{RHESSI}) observations. We construct photon and mean-electron-flux maps in 2-keV energy bins by processing observationally-deduced photon and electron visibilities, respectively, through several image-processing methods: a visibility-based forward-fit (FWD) algorithm, a maximum entropy (MEM) procedure and the uv-smooth (UVS) approach. We estimate the sizes of elongated flares (i.e., the length and width of flaring loops) by calculating the second normalized moments of the intensity in any given map. Employing a collisional model with an extended acceleration region, we fit the loop lengths as a function of energy in both the photon and electron domains. The resulting fitting parameters allow us to estimate the extent of the acceleration region which is between $\sim 13 \rm{arcsec}$ and $\sim 19 \rm{arcsec}$. Both forward-fit and uv-smooth algorithms provide substantially similar results with a systematically better fit in the electron domain.The consistency of the estimates from these methods provides strong support that the model can reliably determine geometric parameters of the acceleration region. The acceleration region is estimated to be a substantial fraction ($\sim 1/2$) of the loop extent, indicating that this dense flaring loop incorporates both acceleration and transport of electrons, with concurrent thick-target bremsstrahlung emission.