Selection of starbursts through either deep narrow band imaging of redshifted Ly-alpha emitters, or localisation of host galaxies of gamma-ray bursts both give access to starburst galaxies that are significantly fainter than what is currently available from selection techniques based on the continuum (such as Lyman-break selection). We here present results from a survey for Ly-alpha emitters at z=3, conducted at the European Southern Observatory's Very Large Telescope. Furthermore, we briefly describe the properties of host galaxies of gamma-ray bursts at z>2. The majority of both Ly-alpha and gamma-ray burst selected starbursts are fainter than the flux limit of the Lyman-break galaxy sample, suggesting that a significant fraction of the integrated star formation at z~3 is located in galaxies at the faint end of the luminosity function.