During the past few years, first observations of starburst galaxies at >GeV energies could be made with the Fermi Gamma-ray Space Telescope (GeV range) and Imaging Air Cherenkov Telescopes (TeV range). The two nearest starbursts, M82 and NGC253 were detected, and most recently, the detection of two starburst-Seyfert composites (NGC1068 and NGC4945) were reported. The emission for the two starbursts is best explained by hadronic interactions, and thus providing a first, unique opportunity to study the role of cosmic rays in galaxies. In this paper, the role of cosmic rays for the non-thermal component of galaxies is reviewed by discussing the entire non-thermal frequency range from radio emission to TeV energies. In particular, the interpretation of radio emission arising from electron synchrotron radiation is predicted to be correlated to TeV emission coming from interactions of accelerated hadrons. This is observed for the few objects known at TeV energies, but the correlation needs to be established with significantly higher statistics. An outlook of the possibility of tracing cosmic rays with molecular ions is given.