We present new constraints on anisotropic birefringence of the cosmic microwave background polarization using two seasons of data from the Atacama Cosmology Telescope covering $456$ square degrees of sky. The birefringence power spectrum, measured using a curved-sky quadratic estimator, is consistent with zero. Our results provide the tightest current constraint on birefringence over a range of angular scales between $5$ arcminutes and $9$ degrees. We improve previous upper limits on the amplitude of a scale-invariant birefringence power spectrum by a factor of between $2$ and $3$. Assuming a nearly-massless axion field during inflation, our result is equivalent to a $2\,\sigma$ upper limit on the Chern-Simons coupling constant between axions and photons of $g_{\alpha\gamma}<4.0\times 10^{-2}/H_I$ where $H_I$ is the inflationary Hubble scale.