We explore the evolution of the cold gas (molecular and neutral hydrogen) of elliptical galaxies and merger remnants ordered into a time sequence on the basis of spectroscopic age estimates. We find that the fraction of cold gas in early merger remnants decreases significantly for ~1-2 Gyr, but subsequent evolution toward evolved elliptical systems sees very little change. This trend can be attributed to an initial gas depletion by strong star-formation which subsequently declines to quiescent rates. This explanation is consistent with the merger picture for the formation of elliptical galaxies. We also explore the relation between HI-to-H2 mass ratio and spectroscopic galaxy age, but find no evidence for a statistically significant trend. This suggests little net HI to H2 conversion for the systems in the present sample.