The euclidean path integral remains, in spite of its familiar problems, an important approach to quantum gravity. One of its most striking and obscure features is the appearance of gravitational instantons or wormholes. These renormalize all terms in the Lagrangian and cause a number of puzzles or even deep inconsistencies, related to the possibility of nucleation of "baby universes". In this review, we revisit the early controversies surrounding these issues as well as some of the more recent discussions of the phenomenological relevance of gravitational instantons. In particular, wormholes are expected to break the shift symmetries of axions or Goldstone bosons non-perturbatively. This can be relevant to large-field inflation and connects to arguments made on the basis of the Weak Gravity or Swampland conjectures. It can also affect Goldstone bosons which are of physical interest in the context of the strong CP problem or as dark matter.