We analyze two regions of the quiet Sun (35.6 x 35.6 Mm^2) observed at high spatial resolution (~100 km) in polarized light by the IMaX spectropolarimeter onboard the Sunrise balloon. We identify 497 small-scale (~400 km) magnetic loops, appearing at an effective rate of 0.25 loop h^{-1} arcsec^{-2}; further, we argue that this number and rate are underestimated by ~30%. However, we find that these small dipoles do not appear uniformly on the solar surface: their spatial distribution is rather filamentary and clumpy, creating dead calm areas, characterized by a very low magnetic signal and a lack of organized loop-like structures at the detection level of our instruments, that cannot be explained as just statistical fluctuations of a Poisson spatial process. We argue that this is an intrinsic characteristic of the mechanism that generates the magnetic fields in the very quiet Sun. The spatio-temporal coherences and the clumpy structure of the phenomenon suggest a recurrent, intermittent mechanism for the generation of magnetic fields in the quietest areas of the Sun.