We test the statistical isotropy (SI) of the $E$-mode polarization of the cosmic microwave background (CMB) radiation observed by the Planck satellite using two statistics, namely, the contour Minkowski Tensor (CMT) and the Directional statistic ($\mathcal{D}$ statistic). The parameter $\alpha$ obtained from the CMT provides information of the alignment of structures and can be used to infer statistical properties such as Gaussianity and SI of random fields. The $\mathcal{D}$ statistic is based on detecting preferred directionality shown by vectors defined by the field. These two tests are complementary to each other in terms of sensitivity at different angular scales. The CMT is sensitive towards small-scale information present in the CMB map while $\mathcal{D}$ statistic is more sensitive at large-scales. We compute $\alpha$ and $\mathcal{D}$ statistic for the observed $E$-mode of CMB polarization, focusing on the SMICA maps, and compare with the values calculated using FFP10 SMICA simulations which contain both CMB and noise. We find good agreement between the observed data and simulations. Further, in order to specifically analyze the CMB signal in the data, we compare the values of the two statistics obtained from the observed Planck data with the values obtained from isotropic simulations having the same power spectrum, and from SMICA noise simulations. We find no statistically significant deviation from SI using the $\alpha$ parameter. From $\mathcal{D}$ statistic we find that the data shows slight deviation from SI at large angular scales.