The detection of long-lived radionuclides through ultra-sensitive single atom counting via accelerator mass spectrometry (AMS) offers opportunities for precise measurements of neutron capture cross sections, e.g. for nuclear astrophysics. The technique represents a truly complementary approach, completely independent of previous experimental methods. The potential of this technique is highlighted at the example of the $^{54}$Fe($n, \gamma$)$^{55}$Fe reaction. Following a series of irradiations with neutrons from cold and thermal to keV energies, the produced long-lived $^{55}$Fe nuclei ($t_{1/2}=2.744(9)$ yr) were analyzed at the Vienna Environmental Research Accelerator (VERA). A reproducibility of about 1% could be achieved for the detection of $^{55}$Fe, yielding cross section uncertainties of less than 3%. Thus, the new data can serve as anchor points to time-of-flight experiments. We report significantly improved neutron capture cross sections at thermal energy ($\sigma_{th}=2.30\pm0.07$ b) as well as for a quasi-Maxwellian spectrum of $kT=25$ keV ($\sigma=30.3\pm1.2$ mb) and for $E_n=481\pm53$ keV ($\sigma= 6.01\pm0.23$ mb). The new experimental cross sections have been used to deduce improved Maxwellian average cross sections in the temperature regime of the common $s$-process scenarios. The astrophysical impact is discussed using stellar models for low-mass AGB stars.