We give an assessment of the significance of various known effects which may produce genuine fluctuations of star positions comparable to or larger than Gaia's measurement noise, and which thus may limit the ultimately reachable precision of the determination of parallaxes and proper motions. Stellar granulation is found to be no problem except for a small number of cool supergiants. It will be a serious problem for e.g. Mira star parallaxes. Star spots are hard to assess quantitatively. They, too, may be problematic for supergiants and very cool giants. Binarity is a major problem for the astrometry of bright stars. It will significantly impair the kinematics of stellar aggregates with small velocity dispersion. It will also produce a small proportion of grossly wrong parallaxes. Gravitational microlensing will not be a problem, although thousands of highly significant microlensing events will be detected by Gaia. Gravitational macrolensing of quasars will add some noise to their (otherwise ``zero'') proper motions.