The universal character of the gravitational interaction provided by the equivalence principle motivates a geometrical description of gravity. The standard formulation of General Relativity \`a la Einstein attributes gravity to the spacetime curvature, to which we have grown accustomed. However, this perception has masked the fact that two alternative, though equivalent, formulations of General Relativity in flat spacetimes exist, where gravity can be fully ascribed either to torsion or to non-metricity. The latter allows a simpler geometrical formulation of General Relativity that is oblivious to the affine spacetime structure. Generalisations along this line permit to generate teleparallel and symmetric teleparallel theories of gravity with exceptional properties. In this work we explore modified gravity theories based on non-linear extensions of the non-metricity scalar. After presenting some general properties and briefly studying some interesting background cosmologies (including accelerating solutions with relevance for inflation and dark energy), we analyse the behaviour of the cosmological perturbations. Tensor perturbations feature a re-scaling of the corresponding Newton's constant, while vector perturbations do not contribute in the absence of vector sources. In the scalar sector we find two additional propagating modes, hinting that $f(Q)$ theories introduce, at least, two additional degrees of freedom. These scalar modes disappear around maximally symmetric backgrounds because of the appearance of an accidental residual gauge symmetry corresponding to a restricted diffeomorphism. We finally discuss the potential strong coupling problems of these maximally symmetric backgrounds caused by the discontinuity in the number of propagating modes.