We calculate the force that pins vortices in the neutron superfluid to nuclei in the inner crust of rotating neutron stars, relying on a detailed microscopic description of both the vortex radial profile and the inner crust nuclear structure. The contribution to the pinning energy from pair condensation is estimated in the local density approximation with realistic nucleon-nucleon interactions. The kinetic contribution, not consistently included in previous approaches, is evaluated in the same approximation, and found to be relevant. The vortex-nucleus interaction turns out to be attractive for stellar desities greater than $\sim 10^13 g/cm^3$. In this region, we find values for the pinning force which are almost one order of magnitude lower than the ones obtained so far. This has direct consequences on the critical velocity differences for depinning.