The formation of black holes inevitably affects the distribution of dark and baryonic matter in their vicinity, leading to an enhancement of the dark matter density, called spike, and if dark matter is made of WIMPs, to a strong enhancement of the dark matter annihilation rate. Spikes at the center of galaxies like the Milky Way are efficiently disrupted by baryonic processes, but mini-spikes can form and survive undisturbed at the center of dwarf spheroidal galaxies. We show that Fermi LAT satellite data allow to set very stringent limits on the existence of mini-spikes in dwarf galaxies: for thermal WIMPs with mass between 100 GeV and 1 TeV, we obtain a maximum black hole mass between 100 and 1000 $M_\odot$, ruling out black holes masses extrapolated from the M-{\sigma} relationship in a large region of the parameter space. We also performed Monte Carlo simulations of merger histories of black holes in dwarf spheroidals in a scenario where black holes form from the direct collapse of primordial gas in early halos, and found that this specific formation scenario is incompatible at the 84% CL with dark matter being in the form of thermal WIMPs.