We discuss the cooling history of the low-mass, helium core white dwarfs in short orbital period millisecond pulsars PSR J0751+1807 and PSR J1012+5307. The revised cooling age estimated by Alberts et al. agrees with the age estimation for PSR J1012+5307, removing the discrepancy between the spin-down age and the cooling age. However, if we accept this model then the helium white dwarf in the binary pulsar system PSR J0751+1807 must be much hotter than is observed. We propose that this discrepancy may be resolved if, after detachment of the secondary star from its Roche lobe in PSR J0751+1807, the star loses its hydrogen envelope due to pulsar irradiation. When hydrogen burning stops, the white dwarf will cool down much more quickly than in the case of a thick hydrogen envelope with a hydrogen burning shell. We discuss several possibilities to explain different cooling histories of white dwarfs in both systems.