Understanding the composition of raw materials that formed the Earth is a crucial step towards understanding the formation of terrestrial planets and their bulk composition. Calcium is the fifth most abundant element in terrestrial planets and, therefore, is a key element with which to trace planetary composition. However, in order to use Ca isotopes as a tracer of Earth's accretion history, it is first necessary to understand the isotopic behavior of Ca during the earliest stages of planetary formation. Chondrites are some of the oldest materials of the Solar System, and the study of their isotopic composition enables understanding of how and in what conditions the Solar System formed. Here we present Ca isotope data for a suite of bulk chondrites as well as Allende (CV) chondrules. We show that most groups of carbonaceous chondrites (CV, CI, CR and CM) are significantly enriched in the lighter Ca isotopes ($\delta^{44/40}Ca$ = +0.1 to +0.93 permill) compared with bulk silicate Earth ($\delta^{44/40}Ca$ = +1.05 $\pm$ 0.04 permill, Huang et al., 2010) or Mars, while enstatite chondrites are indistinguishable from Earth in Ca isotope composition ($\delta^{44/40}Ca$ = +0.91 to +1.06 permill). Chondrules from Allende are enriched in the heavier isotopes of Ca compared to the bulk and the matrix of the meteorite ($\delta^{44/40}Ca$ = +1.00 to +1.21 permill). This implies that Earth and Mars have Ca isotope compositions that are distinct from most carbonaceous chondrites but that may be like chondrules. This Ca isotopic similarity between Earth, Mars, and chondrules is permissive of recent dynamical models of planetary formation that propose a chondrule-rich accretion model for planetary embryos.