Using the semi-analytic models based on the Millennium simulation, we trace back the evolution of the luminosity function of galaxies residing in progenitors of groups classified by the magnitude gap at redshift zero. We determine the luminosity function of galaxies within $ 0.25R_{200}, 0.5R_{200} $, and $R_{200}$ for galaxy groups/clusters. The bright end of the galaxy luminosity function of fossil groups shows a significant evolution with redshift, with changes in $M^*$ by $\sim$ 1-2 mag between $z\sim0.5$ and $z=0$ (for the central $0.5R_{200}$), suggesting that the formation of the most luminous galaxy in a fossil group has had a significant impact on the $M^{*}$ galaxies e.g. it is formed as a result of multiple mergers of the $ M^{*} $ galaxies within the last $\sim5$ Gyr. In contrast, the slope of the faint end, $\alpha$, of the luminosity function shows no considerable redshift evolution and the number of dwarf galaxies in the fossil groups exhibits no evolution, unlike in non-fossil groups where it grows by $\sim25-42\%$ towards low redshifts. In agreement with previous studies, we also show that fossil groups accumulate most of their halo mass earlier than non-fossil groups. Selecting the fossils at a redshift of 1 and tracing them to a redshift 0, we show that $80\%$ of the fossil groups ($10^{13} M_{\odot} h^{-1}<M_{200}<10^{14} M_{\odot} h^{-1}$) will lose their large magnitude gaps. However, about $40\%$ of fossil clusters ($M_{200}>10^{14} M_{\odot} h^{-1}$) will retain their large gaps.