Remote sensing observations suffer significant limitations when used to study the bulk atmospheric composition of the giant planets of our solar system. This impacts our knowledge of the formation of these planets and the physics of their atmospheres. A remarkable example of the superiority of in situ probe measurements was illustrated by the exploration of Jupiter, where key measurements such as the determination of the noble gases' abundances and the precise measurement of the helium mixing ratio were only made available through in situ measurements by the Galileo probe. Here we describe the main scientific goals to be addressed by the future in situ exploration of Saturn, Uranus, and Neptune, placing the Galileo probe exploration of Jupiter in a broader context. An atmospheric entry probe targeting the 10-bar level would yield insight into two broad themes: i) the formation history of the giant planets and that of the Solar System, and ii) the processes at play in planetary atmospheres. The probe would descend under parachute to measure composition, structure, and dynamics, with data returned to Earth using a Carrier Relay Spacecraft as a relay station. An atmospheric probe could represent a significant ESA contribution to a future NASA New Frontiers or flagship mission to be launched toward Saturn, Uranus, and/or Neptune.