Orbital variation in reflected starlight from exoplanets could eventually be used to detect surface oceans. Exoplanets with rough surfaces, or dominated by atmospheric Rayleigh scattering, should reach peak brightness in full phase, orbital longitude = 180 deg, whereas ocean planets with transparent atmospheres should reach peak brightness in crescent phase near OL = 30 deg. Application of Fresnel theory to a planet with no atmosphere covered by a calm ocean predicts a peak polarization fraction of 1 at OL = 74 deg; however, our model shows that clouds, wind-driven waves, aerosols, absorption, and Rayleigh scattering in the atmosphere and within the water column, dilute the polarization fraction and shift the peak to other OLs. Observing at longer wavelengths reduces the obfuscation of the water polarization signature by Rayleigh scattering but does not mitigate the other effects. Planets with thick Rayleigh scattering atmospheres reach peak polarization near OL = 90 deg, but clouds and Lambertian surface scattering dilute and shift this peak to smaller OL. A shifted Rayleigh peak might be mistaken for a water signature unless data from multiple wavelength bands are available. Our calculations suggest that polarization alone may not positively identify the presence of an ocean under an Earth-like atmosphere; however polarization adds another dimension which can be used, in combination with unpolarized orbital light curves and contrast ratios, to detect extrasolar oceans, atmospheric water aerosols, and water clouds. Additionally, the presence and direction of the polarization vector could be used to determine planet association with the star, and constrain orbit inclination.