The fundamentally different isotopic compositions of non-carbonaceous (NC) and carbonaceous (CC) meteorites reveal the presence of two distinct reservoirs in the solar protoplanetary disk that were likely separated by Jupiter. However, the extent of material exchange between these reservoirs, and how this affected the composition of the inner disk are not known. Here we show that NC meteorites display broadly correlated isotopic variations for Mo, Ti, Cr, and Ni, indicating the addition of isotopically distinct material to the inner disk. The added material resembles bulk CC meteorites and Ca-Al-rich inclusions in terms of its enrichment in neutron-rich isotopes, but unlike the latter materials is also enriched in s-process nuclides. The comparison of the isotopic composition of NC meteorites with the accretion ages of their parent bodies reveals that the isotopic variations within the inner disk do not reflect a continuous compositional change through the addition of CC dust, indicating an efficient separation of the NC and CC reservoirs and limited exchange of material between the inner and outer disk. Instead, the isotopic variations among NC meteorites more likely record a rapidly changing composition of the disk during infall from the Sun's parental molecular cloud, where each planetesimal locks the instant composition of the disk when it forms. A corollary of this model is that late-formed planetesimals in the inner disk predominantly accreted from secondary dust that was produced by collisions among pre-existing NC planetesimals.