We measure the evolution of the specific star formation rate (sSFR = SFR / Mstellar) between redshift 4 and 6 to investigate the previous reports of "constant" sSFR at z>2. We obtain photometry on a large sample of galaxies at z~4-6 located in the GOODS-S field that have high quality imaging from HST and Spitzer. We have derived stellar masses and star formation rates (SFRs) through stellar population modeling of their spectral energy distributions (SEDs). We estimate the dust extinction from the observed UV colors. In the SED fitting process we have studied the effects of assuming a star formation history (SFH) both with constant SFR and one where the SFR rises exponentially with time. The latter SFH is chosen to match the observed evolution of the UV luminosity function. We find that neither the mean SFRs nor the mean stellar masses change significantly when the rising SFR (RSF) model is assumed instead of the constant SFR model. When focusing on galaxies with Mstar ~ 5x10^9 Msun, we find that the sSFR evolves weakly with redshift (sSFR(z) \propto (1+z)^(0.6+/-0.1) Gyr^-1), consistent with previous results and with recent estimates of the sSFR at z~2-3 using similar assumptions. We have also investigated the impact of optical emission lines on our results. We estimate that the contribution of emission lines to the rest-frame optical fluxes is only modest at z~4 and 5 but it could reach ~50% at z~6. When emission lines of this strength are taken into account, the sSFR shows somewhat higher values at high redshifts, according to the relation sSFR(z) \propto (1+z)^(1.0+/-0.1) Gyr^-1, i.e., ~2.3x higher at z~6 than at z~2. However, the observed evolution is substantially weaker than that found at z<2 or that expected from current models (which corresponds to sSFR(z) \propto (1+z)^(2.5) Gyr^-1). -abridged-