We report results of our precise reduction of the high signal-to-noise VRI observations of the optical afterglow of the gamma-ray burst GRB 020813 obtained by Gladders & Hall with the Magellan 6.5-m telescope 3.9-4.9 hours after the burst. These observations are very well fitted locally by a power-law curve, providing the tightest constraints yet on how smooth the afterglows can be in some cases: the rms deviations range from 0.005 mag (0.5%) for the R-band to 0.007 mag for the I-band, only marginally larger than the rms scatter for nearby non-variable stars. This scatter is a factor of several smaller then the smallest reported rms of 0.02 mag for GRB 990510 (Stanek et al). These observations are in strong contrast to those of afterglows of GRB 011211 and GRB 021004, for which large > 10% variability has been observed on timescales from ~20 minutes to several hours. If interstellar medium (ISM) density fluctuations near the GRB are indeed causing the bumps and wiggles observed in some bursts, very uniform regions of ISM near some bursts must be present as well. This result also constrains the intrinsic smoothness of the afterglow itself.