Foreground removal techniques for CMB analyses make specific assumptions about the properties of foregrounds in temperature and in polarization. By investigating the statistics of foreground components more understanding about the degree to which these assumptions are valid can be obtained. In this work we investigate $E$- and $B$-mode maps of the two strongest polarized foregrounds, synchrotron and thermal dust emission, with regards to their similarity with Gaussian processes, their spectral variations and cross-correlations. We perform tests in patches of $\sim3.7^\circ$ size collectively covering the full sky and find most of them to conform with their Gaussian expectation according to the statistics in use. Correlations exhibit distinct differences in $E$- and $B$-mode signals which point towards necessities in foreground removal methods. We discuss potential consequences and possible further directions.