The brightness of gamma-ray burst (GRB) afterglows and their occurrence in young, blue galaxies make them excellent probes to study star forming regions in the distant Universe. We here elucidate dust extinction properties in the early Universe through the analysis of the afterglows of all known z > 6 GRBs: GRB 090423, 080913 and 050904, at z = 8.2, 6.69, and 6.295, respectively. We gather all available optical and near-infrared photometry, spectroscopy and X-ray data to construct spectral energy distributions (SEDs) at multiple epochs. We then fit the SEDs at all epochs with a dust-attenuated power-law or broken power-law. We find no evidence for dust extinction in GRB 050904 and GRB 090423, with possible evidence for a low level of extinction in GRB 080913. We compare the high redshift GRBs to a sample of lower redshift GRB extinctions and find a lack of even moderately extinguished events (AV ~ 0.3) above z > 4. In spite of the biased selection and small number statistics, this result hints at a decrease in dust content in star-forming environments at high redshifts.