Mergers of compact binaries (of a neutron star and another neutron star or a black hole, NSMs) are suggested to be the promising astrophysical site of the r-process. While the average coalescence timescale of NSMs appears to be > 100 Myr, most of previous chemical evolution models indicate that the observed early appearance and large dispersion of [r/Fe] in Galactic halo stars at [Fe/H] < -2.5 favors shorter coalescence times of 1-10 Myr. We argue that this is not the case for the models assuming the formation of the Galactic halo from clustering of subhalos with different star formation histories as suggested by Ishimaru et al. (2015). We present a stochastic chemical evolution model of the subhalos, in which the site of the r-process is assumed to be mainly NSMs with a coalescence timescale of 100 Myr. In view of the scarcity of NSMs, their occurrence in each subhalo is computed with a Monte Carlo method. Our results show that the less massive subhalos evolve at lower metallicities and generate highly r-process-enhanced stars. An assembly of these subhalos leaves behind the large star-to-star scatters of [r/Fe] in the Galactic halo as observed. However, the observed scatters of [Sr/Ba] at low metallicities indicate the presence of an additional site that partially contributes to the enrichment of light neutron-capture elements such as Sr. The high enhancements of [r/Fe] at low metallicities found in our low-mass subhalo models also qualitatively reproduce the abundance signatures of the stars in the recently discovered ultra-faint dwarf galaxy Reticulum II. Therefore, our results suggest NSMs as the dominant sources of r-process elements in the Galactic halo.