We investigate the time average mean square displacement $\overline{\delta^2}(x(t))=\int_0^{t-\Delta}[x(t^\prime+\Delta)-x(t^\prime)]^2 dt^\prime/(t-\Delta)$ for fractional Brownian and Langevin motion. Unlike the previously investigated continuous time random walk model $\overline{\delta^2}$ converges to the ensemble average $<x^2 > \sim t^{2 H}$ in the long measurement time limit. The convergence to ergodic behavior is however slow, and surprisingly the Hurst exponent $H=3/4$ marks the critical point of the speed of convergence. When $H<3/4$, the ergodicity breaking parameter ${EB} = {Var} (\overline{\delta^2}) / < \overline{\delta^2} >^2\sim k(H) \cdot\Delta\cdot t^{-1}$, when $H=3/4$, ${EB} \sim (9/16)(\ln t) \cdot\Delta \cdot t^{-1}$, and when $3/4<H <1, {EB} \sim k(H)\Delta^{4-4H} t^{4H-4}$. In the ballistic limit $H \to 1$ ergodicity is broken and ${EB} \sim 2$. The critical point $H=3/4$ is marked by the divergence of the coefficient $k(H)$. Fractional Brownian motion as a model for recent experiments of sub-diffusion of mRNA in the cell is briefly discussed and comparison with the continuous time random walk model is made.