White dwarf WD 1145+017 is orbited by several clouds of dust, possibly emanating from actively disintegrating bodies. These dust clouds reveal themselves through deep, broad, and evolving transits in the star's light curve. Here, we report two epochs of multi-wavelength photometric observations of WD 1145+017, including several filters in the optical, K$_\mathrm{s}$ and 4.5 $\mu$m bands in 2016 and 2017. The observed transit depths are different at these wavelengths. However, after correcting for excess dust emission at K$_\mathrm{s}$ and 4.5 $\mu$m, we find the transit depths for the white dwarf itself are the same at all wavelengths, at least to within the observational uncertainties of $\sim$5%-10%. From this surprising result, and under the assumption of low optical depth dust clouds, we conclude that there is a deficit of small particles (with radii $s \lesssim$ 1.5 $\mu$m) in the transiting material. We propose a model wherein only large particles can survive the high equilibrium temperature environment corresponding to 4.5 hr orbital periods around WD 1145+017, while small particles sublimate rapidly. In addition, we evaluate dust models that are permitted by our measurements of infrared emission.