We study the decoherence process associated with the scattering of stochastic gravitational waves. We discuss the case of macroscopic systems, such as the planetary motion of the Moon around the Earth, for which gravitational scattering is found to dominate decoherence though it has a negligible influence on damping. This contrast is due to the very high effective temperature of the background of gravitational waves in our galactic environment.