Ground-based observations of the Earthshine, i.e., the light scattered by Earth to the Moon, and then reflected back to Earth, simulate space observations of our planet and represent a powerful benchmark for the studies of Earth-like planets. Earthshine spectra are strongly linearly polarized, owing to scattering by molecules and small particles in the atmosphere of the Earth and surface reflection, and may allow us to measure global atmospheric and surface properties of planet Earth. Aims. We aim to interpret already published spectropolarimetric observations of the Earthshine by comparing them with new radiative transfer model simulations including a fully realistic three-dimensional (3D) surface-atmosphere model for planet Earth. We used the highly advanced Monte Carlo radiative transfer model MYSTIC to simulate polarized radiative transfer in the atmosphere of the Earth without approximations regarding the geometry, taking into account the polarization from surface reflection and multiple scattering by molecules, aerosol particles, cloud droplets, and ice crystals. We have shown that Earth spectropolarimetry is highly sensitive to all these input parameters, and we have presented simulations of a fully realistic Earth atmosphere-surface model including 3D cloud fields and two-dimensional (2D) surface property maps. Our modeling results show that scattering in high ice water clouds and reflection from the ocean surface are crucial to explain the continuum polarization at longer wavelengths as has been reported in Earthshine observations taken at the Very Large Telescope in 2011 (3.8 % and 6.6 % at 800 nm, depending on which part of Earth was visible from the Moon at the time of the observations). We found that the relatively high degree of polarization of 6.6 % can be attributed to light reflected by the ocean surface in the sunglint region.