This work investigates a particular class of artefacts, or ghost sources, in radio interferometric images. Earlier observations with (and simulations of) the Westerbork Synthesis Radio Telescope (WSRT) suggested that these were due to calibration with incomplete sky models. A theoretical framework is derived that validates this suggestion, and provides predictions of ghost formation in a two-source scenario. The predictions are found to accurately match the result of simulations, and qualitatively reproduce the ghosts previously seen in observational data. The theory also provides explanations for many previously puzzling features of these artefacts (regular geometry, PSF-like sidelobes, seeming independence on model flux), and shows that the observed phenomenon of flux suppression affecting unmodelled sources is due to the same mechanism. We demonstrate that this ghost formation mechanism is a fundamental feature of calibration, and exhibits a particularly strong and localized signature due to array redundancy. To some extent this mechanism will affect all observations (including those with non-redundant arrays), though in most cases the ghosts remain hidden below the noise or masked by other instrumental artefacts. The implications of such errors on future deep observations are discussed.