An inverse cascade - energy transfer to progressively larger scales - is a salient feature of two-dimensional turbulence. If the cascade reaches the system scale, it creates a coherent flow expected to have the largest available scale and conform with the symmetries of the domain. In a doubly periodic rectangle, the mean flow with zero total momentum was therefore believed to be unidirectional, with two jets along the short side; while for an aspect ratio close to unity, a vortex dipole was expected. Using direct numerical simulations, we show that in fact neither the box symmetry is respected nor the largest scale is realized: the flow is never purely unidirectional since the inverse cascade produces coherent vortices, whose number and relative motion are determined by the aspect ratio. This spontaneous symmetry breaking is closely related to the hierarchy of averaging times. Long-time averaging restores translational invariance due to vortex wandering along one direction, and gives jets whose profile, however, can be deduced neither from the largest-available-scale argument, nor from the often employed maximum-entropy principle or quasi-linear approximation.