With this paper, we release accurate photometric redshifts for 1692 counterparts to Chandra sources in the central square degree of the COSMOS field. The availability of a large training set of spectroscopic redshifts that extends to faint magnitudes enabled photometric redshifts comparable to the highest quality results presently available for normal galaxies. We demonstrate that morphologically extended, faint X-ray sources without optical variability are more accurately described by a library of normal galaxies (corrected for emission lines) than by AGN-dominated templates, even if these sources have AGN-like X-ray luminosities. Preselecting the library on the bases of the source properties allowed us to reach an accuracy sigma_(Delta z/(1+z_spec)) \sim0.015 with a fraction of outliers of 5.8% for the entire Chandra-COSMOS sample. In addition, we release revised photometric redshifts for the 1735 optical counterparts of the XMM-detected sources over the entire 2 sq. deg.of COSMOS. For 248 sources, our updated photometric redshift differs from the previous release by Delta z>0.2. These changes are predominantly due to the inclusion of newly available deep H-band photometry H_AB=24 mag. We illustrate once again the importance of a spectroscopic training sample and how an assumption about the nature of a source together with the number and the depth of the available bands influence the accuracy of the photometric redshifts determined for AGN. These considerations should be kept in mind when defining the observational strategies of upcoming large surveys targeting AGN, such as eROSITA at X-ray energies and ASKAP/EMU in the radio band.