The polarization of the atmosphere has been a long-standing concern for ground-based experiments targeting cosmic microwave background (CMB) polarization. Ice crystals in upper tropospheric clouds scatter thermal radiation from the ground and produce a horizontally-polarized signal. We report the detailed analysis of the cloud signal using a ground-based CMB experiment, POLARBEAR, located at the Atacama desert in Chile and observing at 150 GHz. We observe horizontally-polarized temporal increases of low-frequency fluctuations ("polarized bursts," hereafter) of $\lesssim$0.1 K when clouds appear in a webcam monitoring the telescope and the sky. The hypothesis of no correlation between polarized bursts and clouds is rejected with $>$24$\sigma$ statistical significance using three years of data. We consider many other possibilities including instrumental and environmental effects, and find no other reasons other than clouds that can explain the data better. We also discuss the impact of the cloud polarization on future ground-based CMB polarization experiments.