Ultralight bosons and axion-like particles appear naturally in different scenarios and could solve some long-standing puzzles. Their detection is challenging, and all direct methods hinge on unknown couplings to the Standard Model of particle physics. However, the universal coupling to gravity provides model-independent signatures for these fields. We explore here the superradiant instability of spinning black holes triggered in the presence of such fields. The instability taps angular momentum from and limits the maximum spin of astrophysical black holes. We compute, for the first time, the spectrum of the most unstable modes of a massive vector (Proca) field for generic black-hole spin and Proca mass. The observed stability of the inner disk of stellar-mass black holes can be used to derive \emph{direct} constraints on the mass of dark photons in the mass range $ 10^{-13}\,{\rm eV}\lesssim m_V \lesssim 3\times 10^{-12}\,{\rm eV}$. By including also higher azimuthal modes, similar constraints apply to axion-like particles in the mass range $6\times10^{-13}\,{\rm eV}\lesssim m_{\rm ALP} \lesssim 10^{-11}\, {\rm eV}$. Likewise, mass and spin distributions of supermassive BHs --~as measured through continuum fitting, K$\alpha$ iron line, or with the future space-based gravitational-wave detector LISA~-- imply indirect bounds in the mass range approximately $10^{-19}\,{\rm eV}\lesssim m_V, m_{\rm ALP} \lesssim 10^{-13}\, {\rm eV}$, for both axion-like particles and dark photons. Overall, superradiance allows to explore a region of approximately $8$ orders of magnitude in the mass of ultralight bosons.