We present infrared, optical, and X-ray data of 48 X-ray bright, optically dull AGNs in the COSMOS field. These objects exhibit the X-ray luminosity of an active galactic nucleus (AGN) but lack broad and narrow emission lines in their optical spectrum. We show that despite the lack of optical emission lines, most of these optically dull AGNs are not well-described by a typical passive red galaxy spectrum: instead they exhibit weak but significant blue emission like an unobscured AGN. Photometric observations over several years additionally show significant variability in the blue emission of four optically dull AGNs. The nature of the blue and infrared emission suggest that the optically inactive appearance of these AGNs cannot be caused by obscuration intrinsic to the AGNs. Instead, up to ~70% of optically dull AGNs are diluted by their hosts, with bright or simply edge-on hosts lying preferentially within the spectroscopic aperture. The remaining ~30% of optically dull AGNs have anomalously high f_x/f_o ratios and are intrinsically weak, not obscured, in the optical. These optically dull AGNs are best described as a weakly accreting AGN with a truncated accretion disk from a radiatively inefficient accretion flow.