We derive the dense core structure and the water abundance in four massive star-forming regions which may help understand the earliest stages of massive star formation. We present Herschel-HIFI observations of the para-H2O 1_11-0_00 and 2_02-1_11 and the para-H2-18O 1_11-0_00 transitions. The envelope contribution to the line profiles is separated from contributions by outflows and foreground clouds. The envelope contribution is modelled using Monte-Carlo radiative transfer codes for dust and molecular lines (MC3D and RATRAN), with the water abundance and the turbulent velocity width as free parameters. While the outflows are mostly seen in emission in high-J lines, envelopes are seen in absorption in ground-state lines, which are almost saturated. The derived water abundances range from 5E-10 to 4E-8 in the outer envelopes. We detect cold clouds surrounding the protostar envelope, thanks to the very high quality of the Herschel-HIFI data and the unique ability of water to probe them. Several foreground clouds are also detected along the line of sight. The low H2O abundances in massive dense cores are in accordance with the expectation that high densities and low temperatures lead to freeze-out of water on dust grains. The spread in abundance values is not clearly linked to physical properties of the sources.