Surface roughness is an important metric in understanding how the geologic history of an asteroid affects its small-scale topography and it provides an additional means to quantitatively compare one asteroid with another. In this study, we report the first detailed global surface roughness maps of 25143 Itokawa at horizontal scales from 8--32~m. Comparison of the spatial distribution of the surface roughness of Itokawa with 433 Eros, the other asteroid for which this kind of analysis has been possible, indicates that the two asteroids are dominated by different geologic processes. On Itokawa, the surface roughness reflects the results of down-slope activity that moves fine grained material into geopotential lows and leaves large blocks in geopotential highs. On 433 Eros, the surface roughness is controlled by geologically-recent large impact craters. In addition, large longitudinal spatial variations of surface roughness could impact the role of YORP on Itokawa.