The X-ray quasi-periodic oscillation (QPO) at $\sim 2.6 \times 10^{-4}$ Hz in RE J1034+396 has been robustly observed in the 0.2-10 keV band of one $\sim 90$ ks XMM-Newton observation, but was not detected in subsequent observations. Here, we investigate the power spectral density (PSD) of 8 archival XMM-Newton observations of RE J1034+396, and search for the presence of QPOs in three energy bands: soft (0.3-0.8 keV); hard (1.0-4.0 keV) and total (0.2-10.0 keV). We find a significant detection of a QPO feature in the hard-band PSD of 5 low flux/spectrally-harder observations. The QPO frequency has remained persistent at $\sim 2.6 \times 10^{-4}$ Hz over $\sim 4$ years of observations, though it is no longer detected in the soft band, except in the original observation. This result increases the duration where the QPO is present by a factor of three (now $\sim 250$ ks), allowing for a better understanding of the QPO phenomenon observed in both Active Galactic Nuclei (AGN) and black hole X-ray binaries (BHBs).