The White Mountain Polarimeter (WMPol) is a dedicated ground-based microwave telescope and receiver system for observing polarization of the Cosmic Microwave Background. WMPol is located at an altitude of 3880 meters on a plateau in the White Mountains of Eastern California, USA, at the Barcroft Facility of the University of California White Mountain Research Station. Presented here is a description of the instrument and the data collected during April through October 2004. We set an upper limit on $E$-mode polarization of 14 $\mu\mathrm{K}$ (95% confidence limit) in the multipole range $170<\ell<240$. This result was obtained with 422 hours of observations of a 3 $\mathrm{deg}^2$ sky area about the North Celestial Pole, using a 42 GHz polarimeter. This upper limit is consistent with $EE$ polarization predicted from a standard $\Lambda$-CDM concordance model.