Glitches -- sudden increases in spin rate -- are observed in many pulsars. One mechanism advanced to explain glitches in the youngest pulsars is that they are caused by a starquake, a sudden rearrangement of the crust of the neutron star. Starquakes have the potential to excite some of the oscillation modes of the neutron star, which means that they are of interest as a source of gravitational waves. These oscillations could also have an impact on radio emission. In this paper we make upper estimates of the amplitude of the oscillations produced by a starquake, and the corresponding gravitational wave emission. We then develop a more detailed framework for calculating the oscillations excited by the starquake, using a toy model of a solid, incompressible star where all strain is lost instantaneously from the star at the glitch. For this toy model, we give plots of the amplitudes of the modes excited, as the shear modulus and rotation rate of the star are varied. We find that for our specific model, the largest excitation is generally of a mode similar to the $f$-mode of an incompressible fluid star, but that other modes of the star are excited to a significant degree over small parameter ranges of the rotation rate.