We investigate asymmetric nuclear matter with two- and three-nucleon interactions based on chiral effective field theory, where three-body forces are fit only to light nuclei. Focusing on neutron-rich matter, we calculate the energy for different proton fractions and include estimates of the theoretical uncertainty. We use our ab-initio results to test the quadratic expansion around symmetric matter with the symmetry energy term, and confirm its validity for highly asymmetric systems. Our calculations are in remarkable agreement with an empirical parametrization for the energy density. These findings are very useful for astrophysical applications and for developing new equations of state.