We study the post-Newtonian dynamics of black hole binaries in Einstein-scalar-Gauss-Bonnet gravity theories. To this aim we build static, spherically symmetric black hole solutions at fourth order in the Gauss-Bonnet coupling $\alpha$. We then "skeletonize" these solutions by reducing them to point particles with scalar field-dependent masses, showing that this procedure amounts to fixing the Wald entropy of the black holes during their slow inspiral. The cosmological value of the scalar field plays a crucial role in the dynamics of the binary. We compute the two-body Lagrangian at first post-Newtonian order and show that no regularization procedure is needed to obtain the Gauss-Bonnet contributions to the fields, which are finite. We illustrate the power of our approach by Pad\'e-resumming the so-called "sensitivities," which measure the coupling of the skeletonized body to the scalar field, for some specific theories of interest.