Aims. To better understand the radiation environment in low Earth orbit (LEO), the analysis of in-situ observations of a variety of particles, at different atmospheric heights, and in a wide range of energies, is needed. Methods. We present an analysis of energetic particles, indirectly detected by the Large Yield RAdiometer (LYRA) instrument on board ESA's Project for On-board Autonomy 2 (PROBA2) satellite as background signal. Combining Energetic Particle Telescope (EPT) observations with LYRA data for an overlapping period of time, we identified these particles as electrons with an energy range of 2 to 8 MeV. Results. The observed events are strongly correlated to geo-magnetic activity and appear even during modest disturbances. They are also well confined geographically within the L=4-6 McIlwain zone, which makes it possible to identify their source. Conclusions. Although highly energetic particles are commonly perturbing data acquisition of space instruments, we show in this work that ultra-relativistic electrons with energies in the range of 2-8 MeV are detected only at high latitudes, while not present in the South Atlantic Anomaly region.