We present the discovery and detailed observations of the radio galaxy 1243+036 (=4C 03.24) at a redshift of $z=3.57$. The most spectacular feature of 1243+036 is the presence of a Ly$\alpha$ halo of luminosity $\sim10^{44.5}$ ergs s$^{-1}$ which extends over$\sim20''$ (135 kpc). The narrow band imaging and the high resolution spectroscopy show that the Ly$\alpha$ gas has three distinct components: (i) gas with a high velocity dispersion (1550 km s$^{-1}$ FWHM) located inside the radio structure, (ii) enhanced Ly$\alpha$ emission blue shifted by 1100 km s$^{-1}$ at the location of the strong bend in the radio jet and (iii) Ly$\alpha$ emission extending out well beyond the radio lobes. This emission has a low velocity dispersion (250 km s$^{-1}$ FWHM) and a velocity gradient of 450 km s$^{-1}$ over the extent of the emission, indicative of large scale rotation. On the basis of these obserserations various mechanisms for the origin and kinematics of the Ly$\alpha$ halo are discussed.