Nitrogen is a major nutrient for all life on Earth and could plausibly play a similar role in extraterrestrial biospheres. The major reservoir of nitrogen at Earth's surface is atmospheric N2, but recent studies have proposed that the size of this reservoir may have fluctuated significantly over the course of Earth's history with particularly low levels in the Neoarchean - presumably as a result of biological activity. We used a biogeochemical box model to test which conditions are necessary to cause large swings in atmospheric N2 pressure. Parameters for our model are constrained by observations of modern Earth and reconstructions of biomass burial and oxidative weathering in deep time. A 1-D climate model was used to model potential effects on atmospheric climate. In a second set of tests, we perturbed our box model to investigate which parameters have the greatest impact on the evolution of atmospheric pN2 and consider possible implications for nitrogen cycling on other planets. Our results suggest that (a) a high rate of biomass burial would have been needed in the Archean to draw down atmospheric pN2 to less than half modern levels, (b) the resulting effect on temperature could probably have been compensated by increasing solar luminosity and a mild increase in pCO2, and (c) atmospheric oxygenation could have initiated a stepwise pN2 rebound through oxidative weathering. In general, life appears to be necessary for significant atmospheric pN2 swings on Earth-like planets. Our results further support the idea that an exoplanetary atmosphere rich in both N2 and O2 is a signature of an oxygen-producing biosphere.