We present a method to derive the relative pointing offsets for SPIRE Fourier-Transform Spectrometer (FTS) solar system object (SSO) calibration targets, which were observed regularly throughout the Herschel mission. We construct ratios of the spectra for all observations of a given source with respect to a reference. The reference observation is selected iteratively to be the one with the highest observed continuum. Assuming that any pointing offset leads to an overall shift of the continuum level, then these ratios represent the relative flux loss due to mispointing. The mispointing effects are more pronounced for a smaller beam, so we consider only the FTS short wavelength array (SSW, 958-1546 GHz) to derive a pointing correction. We obtain the relative pointing offset by comparing the ratio to a grid of expected losses for a model source at different distances from the centre of the beam, under the assumption that the SSW FTS beam can be well approximated by a Gaussian. In order to avoid dependency on the point source flux conversion, which uses a particular observation of Uranus, we use extended source flux calibrated spectra to construct the ratios for the SSOs. In order to account for continuum variability, due to the changing distance from the Herschel telescope, the SSO ratios are normalised by the expected model ratios for the corresponding observing epoch. We confirm the accuracy of the derived pointing offset by comparing the results with a number of control observations, where the actual pointing of Herschel is known with good precision. Using the method we derived pointing offsets for repeated observations of Uranus (including observations centred on off-axis detectors), Neptune, Ceres and NGC7027. The results are used to validate and improve the point-source flux calibration of the FTS.