Because supernova remnants are short lived, studies of neutron star X-ray binaries within supernova remnants probe the earliest stages in the life of accreting neutron stars. However, such objects are exceedingly rare: none were known to exist in our Galaxy. We report the discovery of the natal supernova remnant of the accreting neutron star Circinus X-1, which places an upper limit of t < 4, 600 years on its age, making it the youngest known X-ray binary and a unique tool to study accretion, neutron star evolution, and core collapse supernovae. This discovery is based on a deep 2009 Chandra X-ray observation and new radio observations of Circinus X-1. Circinus X-1 produces type I X-ray bursts on the surface of the neutron star, indicating that the magnetic field of the neutron star is small. Thus, the young age implies either that neutron stars can be born with low magnetic fields or that they can rapidly become de-magnetized by accretion. Circinus X-1 is a microquasar, creating relativistic jets which were thought to power the arcminute scale radio nebula surrounding the source. Instead, this nebula can now be attributed to non-thermal synchrotron emission from the forward shock of the supernova remnant. The young age is consistent with the observed rapid orbital evolution and the highly eccentric orbit of the system and offers the chance to test the physics of post-supernova orbital evolution in X-ray binaries in detail for the first time.