The Kepler mission revealed a class of planets known as ''super-puffs,'' with masses only a few times larger than Earth's but radii larger than Neptune, giving them very low mean densities. All three of the known planets orbiting the young solar-type star Kepler 51 are super-puffs. The Kepler 51 system thereby provides an opportunity for a comparative study of the structures and atmospheres of this mysterious class of planets, which may provide clues about their formation and evolution. We observed two transits each of Kepler 51b and 51d with the Wide Field Camera 3 (WFC3) on the Hubble Space Telescope. Combining new WFC3 transit times with re-analyzed Kepler data and updated stellar parameters, we confirmed that all three planets have densities lower than 0.1 g/cm$^{3}$. We measured the WFC3 transmission spectra to be featureless between 1.15 and 1.63 $\mu$m, ruling out any variations greater than 0.6 scale heights (assuming a H/He dominated atmosphere), thus showing no significant water absorption features. We interpreted the flat spectra as the result of a high-altitude aerosol layer (pressure $<$3 mbar) on each planet. Adding this new result to the collection of flat spectra that have been observed for other sub-Neptune planets, we find support for one of the two hypotheses introduced by Crossfield and Kreidberg (2017), that planets with cooler equilibrium temperatures have more high-altitude aerosols. We strongly disfavor their other hypothesis that the H/He mass fraction drives the appearance of large amplitude transmission features.