Infrared surveys provide essential insights on galaxy evolution. If near-IR studies suggest mild evolution of stellar populations with cosmic time, indications of a substantial evolution have been seen in the far-IR, although the available information is largely insufficient to delineate precise evolutionary properties. A consistent picture encompassing all the currently available data may be obtained assuming that dust extinction hides the early evolutionary phases of spheroidal galaxies in the optical band, while the corresponding dust re-radiation in the far-IR may have been, during the early evolutionary phases, orders of magnitude larger than today. Hyperluminous IRAS galaxies might be extreme examples of this situation. Additional indications that at least some spheroidal galaxies may have been very dusty during their early evolution are provided by recent data on high redshift radio galaxies and quasars. Galaxies are the likely dominant contributors to the IR background. However, in the framework of unified models for Active Galactic Nuclei, a large number of nuclei hidden by a dusty torus may be expected. Implications for the IR background are discussed.