The dynamics of Mars' obliquity are believed to be chaotic, and the historical ~3.5 Gyr (late-Hesperian onward) obliquity probability density function (PDF) is high uncertain and cannot be inferred from direct simulation alone. Obliquity is also a strong control on post-Noachian Martian climate, enhancing the potential for equatorial ice/snow melting and runoff at high obliquities (> 40{\deg}) and enhancing the potential for desiccation of deep aquifers at low obliquities (< 25{\deg}). We developed a new technique using the orientations of elliptical craters to constrain the true late-Hesperian-onward obliquity PDF. To do so, we developed a forward model of the effect of obliquity on elliptic crater orientations using ensembles of simulated Mars impactors and ~3.5 Gyr-long Mars obliquity simulations. In our model, the inclinations and speeds of Mars crossing objects bias the preferred orientation of elliptic craters which are formed by low-angle impacts. Comparison of our simulation predictions with a validated database of elliptic crater orientations allowed us to invert for best-fitting obliquity history. We found that since the onset of the late-Hesperian, Mars' mean obliquity was likely low, between ~10{\deg} and ~30{\deg}, and the fraction of time spent at high obliquities > 40{\deg} was likely < 20%.