We systematically study the effects of varying the starting redshift z_i for cosmological simulations in the highly non-linear regime. Our primary focus lies with the (individual) properties of dark matter halos -- namely the mass, spin, triaxiality, and concentration -- where we find that even substantial variations in z_i leave only a small imprint, at least for the probed mass range M \in [10^{10}, 10^{13}] Msun/h and when investigated at redshift z=0. We further compare simulations started by using the standard Zel'dovich approximation to runs based upon initial conditions produced with second order Lagrangian perturbation theory. Here we observe the same phenomenon, i.e. that differences in the studied (internal) properties of dark matter haloes are practically undetectable. These findings are (for the probed mass range) in agreement with other work in the literature. We therefore conclude that the commonly used technique for setting up cosmological simulations leads to stable results at redshift z=0 for the mass, the spin parameter, the triaxiality, and the concentration of dark matter haloes.