Binding of few-electron systems in two-dimensional potential cavities in the presence of an external magnetic field is studied with the exact diagonalization approach. We demonstrate that for shallow cavities the few-electron system becomes bound only under the application of a strong magnetic field. The critical value of the depth of the cavity allowing the formation of a bound state decreases with magnetic field in a non-smooth fashion, due to the increasing angular momentum of the first bound state. In the high magnetic field limit the binding energies and the critical values for the depth of the potential cavity allowing the formation of a bound system tend to the classical values.