To better understand how planets form, it is important to study planet occurrence rates as a function of stellar mass. However, estimating masses of field stars is often difficult. Over the past decade, a controversy has arisen about the inferred occurrence rate of gas-giant planets around evolved intermediate-mass stars -- the so-called `retired A-stars'. The high masses of these red-giant planet hosts, derived using spectroscopic information and stellar evolution models, have been called into question. Here we address the controversy by determining the masses of eight evolved planet-hosting stars using asteroseismology. We compare the masses with spectroscopic-based masses from the Exoplanet Orbit Database that were previously adopted to infer properties of the exoplanets and their hosts. We find a significant one-sided offset between the two sets of masses for stars with spectroscopic masses above roughly 1.6Msun, suggestive of an average 15--20% overestimate of the adopted spectroscopic-based masses. The only star in our sample well below this mass limit is also the only one not showing this offset. Finally, we note that the scatter across literature values of spectroscopic-based masses often exceed their formal uncertainties, making it comparable to the offset we report here.