Complex-valued and widely linear modelling of time series signals are widespread and found in many applications. However, existing models and analysis techniques are usually restricted to signals observed in discrete time. In this paper we introduce a widely linear version of the complex Ornstein-Uhlenbeck (OU) process. This is a continuous-time process which generalises the standard complex-valued OU process such that signals generated from the process contain elliptical oscillations, as opposed to circular oscillations, when viewed in the complex plane. We determine properties of the widely linear complex OU process, including the conditions for stationarity, and the geometrical structure of the elliptical oscillations. We derive the analytical form of the power spectral density function, which then provides an efficient procedure for parameter inference using the Whittle likelihood. We apply the process to measure periodic and elliptical properties of Earth's polar motion, including that of the Chandler wobble, for which the standard complex OU process was originally proposed.