The shift-symmetric coupling of a pseudo-scalar particle driving inflation to gauge fields provides a unique way of probing cosmic inflation. We show for an SU(2) gauge group how a classical isotropic background gauge field develops from the standard quantum mechanical vacuum in the far past. Over the course of inflation, the theory dynamically evolves from an approximately abelian regime into an inherently non-abelian regime, with distinct predictions for the scalar and tensor power spectra. The latter regime closely resembles a setup known as chromo-natural inflation, although our main focus here is on a new part of the parameter space which has received little attention so far. For single-field slow roll inflation models, large scales may exit the horizon in the abelian regime, ensuring agreement with the observations of the anisotropies in the cosmic microwave background, whereas smaller scales experience the non-abelian effects. This results in a strong enhancement of the stochastic gravitational wave background at small scales, e.g. at frequencies accessible with ground-based interferometers. For the scalar power spectrum, a similar enhancement arises due to non-linear contributions.