We investigate strong nonlinear damping effects which occur during high amplitude oscillations of neutron stars, and the gravitational waves they produce. For this, we use a general relativistic nonlinear hydrodynamics code in conjunction with a fixed spacetime (Cowling approximation) and a polytropic equation of state (EOS). Gravitational waves are estimated using the quadrupole formula. Our main interest are l=m=2 f modes subject to the CFS (Chandrasekhar, Friedman, Schutz) instability, but we also investigate axisymmetric and quasiradial modes. We study various models to determine the influence of rotation rate and EOS. We find that axisymmetric oscillations at high amplitudes are predominantly damped by shock formation, while the nonaxisymmetric f modes are mainly damped by wave breaking and, for rapidly rotating models, coupling to nonaxisymmetric inertial modes. From the observed nonlinear damping, we derive upper limits for the saturation amplitude of CFS-unstable f modes. Finally, we estimate that the corresponding gravitational waves for an oscillation amplitude at the upper limit should be detectable with the advanced LIGO and VIRGO interferometers at distances above 10 MPc. This strongly depends on the stellar model, in particular on the mode frequency.