The tomographic Alcock-Paczynski (AP) method can result in tight cosmological constraints by using small and intermediate clustering scales of the large scale structure (LSS) of the galaxy distribution. By focusing on the redshift dependence, the AP distortion can be distinguished from the distortions produced by the redshift space distortions (RSD). In this work, we combine the tomographic AP method with other recent observational datasets of SNIa+BAO+CMB+$H_0$ to reconstruct the dark energy equation-of-state $w$ in a non-parametric form. The result favors a dynamical DE at $z\lesssim1$, and shows a mild deviation ($\lesssim2\sigma$) from $w=-1$ at $z=0.5-0.7$. We find the addition of the AP method improves the low redshift ($z\lesssim0.7$) constraint by $\sim50\%$.