Originally complied for 1868-1967 and subsequently continued so that it now covers 150 years, the $aa$ index has become a vital resource for studying space climate change. However, there have been debates about the inter-calibration of data from the different stations. In addition, the effects of secular change in the geomagnetic field have not previously been allowed for. As a result, the components of the 'classical' $aa$ index for the southern and northern hemispheres ($aa_S$ and $aa_N$) have drifted apart. We here separately correct both $aa_S$ and $aa_N$ for both these effects using the same method as used to generate the classic $aa$ values but allowing ${\delta}$, the minimum angular separation of each station from a nominal auroral oval, to vary as calculated using the IGRF-12 and gufm1 models of the intrinsic geomagnetic field. Our approach is to correct the quantized aK-values for each station, originally scaled on the assumption that ${\delta}$ values are constant, with time-dependent scale factors that allow for the drift in ${\delta}$. This requires revisiting the intercalibration of successive stations used in making the $aa_S$ and $aa_N$ composites. These intercalibrations are defined using independent data and daily averages from 11 years before and after each station change and it is shown that they depend on the time of year. This procedure produces new homogenized hemispheric aa indices, $aa_{HS}$ and $aa_{HN}$, which show centennial-scale changes that are in very close agreement. Calibration problems with the classic $aa$ index are shown to have arisen from drifts in ${\delta}$ combined with simpler corrections which gave an incorrect temporal variation and underestimate the rise in $aa$ during the 20th century by about 15%.

Paper 1 [Lockwood et al., 2018] generated annual means of a new version of the $aa$ geomagnetic activity index which includes corrections for secular drift in the geographic coordinates of the auroral oval, thereby resolving the difference between the centennial-scale change in the northern and southern hemisphere indices, $aa_N$ and $aa_S$. However, other hemispheric asymmetries in the $aa$ index remain: in particular, the distributions of 3-hourly $aa_N$ and $aa_S$ values are different and the correlation between them is not high on this timescale ($r = 0.66$). In the present paper, a location-dependant station sensitivity model is developed using the $am$ index (derived from a much more extensive network of stations in both hemispheres) and used to reduce the difference between the hemispheric $aa$ indices and improve their correlation (to $r = 0.79$) by generating corrected 3-hourly hemispheric indices, $aa_{HN}$ and $aa_{HS}$, which also include the secular drift corrections detailed in Paper 1. These are combined into a new, 'homogeneous' $aa$ index, $aa_H$. It is shown that $aa_H$, unlike $aa$, reveals the 'equinoctial'-like time-of-day/time-of-year pattern that is found for the $am$ index.

Using the reconstruction of power input to the magnetosphere given in Paper 1 (arXiv:1708.04904), we reconstruct annual means of geomagnetic indices over the past 400 years to within a 1-sigma error of +/-20 pc. In addition, we study the behaviour of the lognormal distribution of daily and hourly values about these annual means and show that we can also reconstruct the fraction of geomagnetically-active (storm-like) days and (substorm-like) hours in each year to accuracies of 50-60 pc. The results are the first physics-based quantification of the space weather conditions in both the Dalton and Maunder minima. We predict terrestrial disturbance levels in future repeats of these minima, allowing for the weakening of Earth's dipole moment.

Using information on geomagnetic activity, sunspot numbers and cosmogenic isotopes, supported by historic eclipse images and in conjunction with models, it has been possible to reconstruct annual means of solar wind speed and number density and heliospheric magnetic field (HMF) intensity since 1611, when telescopic observations of sunspots began. These models are developed and tuned using data recorded by near-Earth interplanetary spacecraft and by solar magnetograms over the past 53 years. In this paper, we use these reconstructions to quantify power input into the magnetosphere over the past 400 years. For each year, both the annual mean power input is computed and its distribution in daily means. This is possible because the distribution of daily values divided by the annual mean is shown to maintain the same lognormal form with a constant variance. This study is another important step towards the development of a physics-based, long-term climatology of space weather conditions.

We use 5 test data series to quantify putative discontinuities around 1946 in 5 annual-mean sunspot number or group number sequences. The series tested are: the original and new versions of the Wolf/Zurich/International sunspot number composite [$R_{ISNv1}$ and $R_{ISNv2}$] ; the corrected version of $R_{ISNv1}$ [$R_C$]; the backbone group number [$R_{BB}$]; and the group number composite [$R_{UEA}$]. The test data are: the group number $N_G$ and total sunspot area $A_G$ from the RGO photoheliographic data; the CaK index from re-analysis of MWO CaII K spectroheliograms; the group number from the MWO sunspot drawings, $N_{MWO}$; and ionospheric critical frequencies at Slough [$foF2$]. The test data all vary with sunspot numbers, in some cases non-linearly. Tests use both before-and-after fit-residual comparison and correlation methods, applied to intervals iterated to minimise errors and eliminate the effect of the discontinuity date. It is not assumed that the correction required is by a constant factor, nor even linear in sunspot number. A non-linear correction is required by $R_C$, $R_{BB}$ and $R_{ISNv1}$, but not by $R_{ISNv2}$ or $R_{UEA}$. The test datasets give very similar results in all cases. By multiplying the probability distribution functions together we obtain the optimum correction for each data series that must be applied to pre-discontinuity data to make them consistent with the post-discontinuity data. It is shown that, on average, values for 1932-1943 are too small (relative to later values) by about 12.3% for $R_{ISNv1}$ but are too large for $R_{ISNv2}$ and $R_{BB}$ by 3.8% and 5.2%. The correction applied to generate $R_C$ from $R_{ISNv1}$ reduces this average factor to 0.5% but does not remove the non-linear variation, and other errors remain uncorrected. A test is provided by $R_{UEA}$, which is identical to the RGO $N_G$ values over the interval used.

We compare four sunspot-number data sequences against geomagnetic and terrestrial auroral observations. The comparisons are made for the original SIDC composite of Wolf-Zurich-International sunspot number [$R_{ISNv1}$], the group sunspot number [$R_{G}$] by Hoyt and Schatten (Solar Phys., 1998), the new "backbone" group sunspot number [$R_{BB}$] by Svalgaard and Schatten (Solar Phys., 2016), and the "corrected" sunspot number [$R_{C}$] by Lockwood at al. (J.G.R., 2014). Each sunspot number is fitted with terrestrial observations, or parameters derived from terrestrial observations to be linearly proportional to sunspot number, over a 30-year calibration interval of 1982-2012. The fits are then used to compute test sequences, which extend further back in time and which are compared to $R_{ISNv1}$, $R_{G}$, $R_{BB}$, and $R_{C}$. To study the long-term trends, comparisons are made using averages over whole solar cycles (minimum-to-minimum). The test variations are generated in four ways: i) using the IDV(1d) and IDV geomagnetic indices (for 1845-2013) fitted over the calibration interval using the various sunspot numbers and the phase of the solar cycle; ii) from the open solar flux (OSF) generated for 1845 - 2013 from four pairings of geomagnetic indices by Lockwood et al. (Ann. Geophys., 2014) and analysed using the OSF continuity model of Solanki at al. (Nature, 2000) which employs a constant fractional OSF loss rate; iii) the same OSF data analysed using the OSF continuity model of Owens and Lockwood (J.G.R., 2012) in which the fractional loss rate varies with the tilt of the heliospheric current sheet and hence with the phase of the solar cycle; iv) the occurrence frequency of low-latitude aurora for 1780-1980 from the survey of Legrand and Simon (Ann. Geophys., 1987). For all cases, $R_{BB}$ exceeds the test terrestrial series by an amount that increases as one goes back in time.

New sunspot data composites, some of which are radically different in the character of their long-term variation, are evaluated over the interval 1845-2014. The method commonly used to calibrate historic sunspot data, relative to modern-day data, is "daisy-chaining", whereby calibration is passed from one data subset to the neighbouring one, usually using regressions of the data subsets for the intervals of their overlap. Recent studies have illustrated serious pitfalls in these regressions and the resulting errors can be compounded by their repeated use as the data sequence is extended back in time. Hence the recent composite data series by Usoskin et al. (2016), $R_{UEA}$, is a very important advance because it avoids regressions, daisy-chaining and other common, but invalid, assumptions: this is achieved by comparing the statistics of "active day" fractions to those for a single reference dataset. We study six sunspot data series including $R_{UEA}$ and the new "backbone" data series $R_{BB}$, recently generated by Svalgaard and Schatten (2016) by employing both regression and daisy-chaining. We show that all six can be used with a continuity model to reproduce the main features of the open solar flux variation for 1845-2014, as reconstructed from geomagnetic activity data. However, some differences can be identified that are consistent with tests using a basket of other proxies for solar magnetic fields. Using data from a variety of sunspot observers, we illustrate problems with the method employed in $R_{BB}$ which cause it to increasingly overestimate sunspot numbers going back in time and we recommend using $R_{UEA}$ because it employs more robust procedures that avoid such problems.

More than 70 years ago it was recognised that ionospheric F2-layer critical frequencies $foF2$ had a strong relationship to sunspot number. Using historic datasets from the Slough and Washington ionosondes, we evaluate the best statistical fits of $foF2$ to sunspot numbers (at each Universal Time [UT] separately) in order to search for drifts and abrupt changes in the fit residuals over Solar Cycles 17 - 21. Polynomial fits are made both with and without allowance for the white-light facular area, which has been reported as being associated with cycle-to-cycle changes in the sunspot number - $foF2$ relationship. Over the interval studied here, the ISN, $R$, the backbone group number $Rbb$, and the corrected number $Rc$ largely differ in their allowance for the 'Waldmeier discontinuity' around 1945 (the correction factor for which for $R$, $Rbb$ and $Rc$ is, respectively, zero, effectively over 20%, and explicitly 11.6%). It is shown that for Solar Cycles 18 - 21, all three sunspot data sequences perform well, but that the fit residuals are lowest and most uniform for $Rbb$. We here use $foF2$ for those UTs for which $R$, $Rbb$, and $Rc$ all give correlations exceeding 0.99 for intervals both before and after the Waldmeier discontinuity. The error introduced by the Waldmeier discontinuity causes $R$ to underestimate the fitted values based on the $foF2$ data for 1932 - 1945 but $Rbb$ overestimates them by almost the same factor, implying that the correction for the Waldmeier discontinuity inherent in $Rbb$ is too large by a factor of two. Fit residuals are smallest and most uniform for $Rc$ and the ionospheric data support the optimum discontinuity multiplicative correction factor derived from the independent Royal Greenwich Observatory (RGO) sunspot group data for the same interval.

We use sunspot group observations from the Royal Greenwich Observatory (RGO) to investigate the effects of intercalibrating data from observers with different visual acuities. The tests are made by counting the number of groups $R_B$ above a variable cut-off threshold of observed total whole-spot area (uncorrected for foreshortening) to simulate what a lower acuity observer would have seen. The synthesised annual means of $R_B$ are then re-scaled to the observed RGO group number $R_A$ using a variety of regression techniques. It is found that a very high correlation between $R_A$ and $R_B$ ($r_{AB}$ > 0.98) does not prevent large errors in the intercalibration (e.g. sunspot maximum values can be over 30% too large even for such levels of $r_{AB}$). In generating the backbone sunspot number, Svalgaard and Schatten [2015] force regression fits to pass through the scatter plot origin which generates unreliable fits (the residuals do not form a normal distribution) and causes sunspot cycle amplitudes to be exaggerated in the intercalibrated data. It is demonstrated that the use of Quantile-Quantile (Q-Q) plots to test for a normal distribution is a useful indicator of erroneous and misleading regression fits. Ordinary least squares linear fits, not forced to pass through the origin, are sometimes reliable (although the optimum method used is shown to be different when matching peak and average sunspot group numbers). However other fits are only reliable if non-linear regression is used. From these results it is entirely possible that the inflation of solar cycle amplitudes in the backbone group sunspot number as one goes back in time, relative to related solar-terrestrial parameters, is entirely caused by the use of inappropriate and non-robust regression techniques to calibrate the sunspot data.