Primordial gravitational waves leave a characteristic imprint on the cosmic microwave background (CMB) in the form of $B$-mode polarization. Photons are also deflected by large scale gravitational waves which intervene between the source screen and our telescopes, resulting in curl-type gravitational lensing. Gravitational waves present at the epoch of reionization contribute to both effects, thereby leading to a non-vanishing cross-correlation between $B$-mode polarization and curl lensing of the CMB. Observing such a cross correlation would be very strong evidence that an observation of $B$-mode polarization was due to the presence of large scale gravitational waves, as opposed to astrophysical foregrounds or experimental systematic effects. We study the cross-correlation across a wide range of source redshifts and show that a post-SKA experiment aimed to map out the 21-cm sky between $15 \leq z \leq 30$ could rule out non-zero cross-correlation at high significance for $r \geq 0.01$.