We show how the angular clustering between gravitational-wave standard sirens and galaxies with known redshifts allows an inference of the Hubble constant, regardless of whether the host galaxies of any of these sirens are present in the galaxy catalog. We demonstrate this for the first time with realistic simulations of gravitational-wave signals from binary black holes in a three-detector network with Advanced LIGO and Advanced Virgo sensitivities. We show that with such a network, the cross-correlation technique can be used to infer the Hubble parameter with a precision of less than 10% (2%) at 90% confidence with 50 (500) sources, even with a 100% incomplete catalog, which does not contain the hosts of any of the gravitational-wave events. We compare our method with the current state-of-the-art techniques used for the inference of the Hubble parameter from real data. We argue that, if the clustering information is not used explicitly, the inference of $H_0$ from real data is expected to be prior-dominated.