Stellar population studies of early-type galaxies (ETGs) based on their optical stellar continuum suggest that these are quiescent systems. However, emission lines and ultraviolet photometry reveal a diverse population. We use a new version of the STARLIGHT spectral synthesis code and state-of-the-art stellar population models to simultaneously fit SDSS spectra and GALEX photometry for a sample of 3453 $z < 0.1$ galaxies with $NUV-r > 5$ and classified as elliptical by Galaxy Zoo. We reproduce $FUV$ magnitudes for 80 per cent of UV upturn galaxies selected using criteria from the literature, suggesting that additional stellar population ingredients such as binaries and extreme horizontal branch stars may have a limited contribution to the UV upturn. The addition of ultraviolet data leads to a broadening of the distributions of mean stellar ages, metallicities and attenuation. Stellar populations younger than $1\,$Gyr are required to reproduce the ultraviolet emission in 17 per cent of our sample. These systems represent 43 per cent of the sample at $5<NUV-r<5.5$ and span the same stellar mass range as other ETGs in our sample. ETGs with young stellar components have larger $H\alpha$ equivalent widths ($W_{H\alpha}$) and larger dust attenuation. Emission line ratios and $W_{H\alpha}$ indicate that the ionising source in these systems is a mixture of young and old stellar populations. Their young stellar populations are metal-poor, especially for high-mass galaxies, indicating recent star formation associated with rejuvenation events triggered by external processes, such as minor mergers.