We present a systematic analysis of archival HST WFPC2 ``Association'' data sets that correlate with the Chandra positions of a set of 44 ultra-luminous X-ray sources (ULXs) of nearby galaxies. We have improved the Chandra-HST relative astrometry whenever possible. Disparate numbers of potential ULX counterparts are found, and in some cases none are found. The lack of or low number of counterparts in some cases may be due to insufficient depth in the WFPC2 images. Particularly in late-type galaxies, the HST image in the ULX region was often complex or crowded. We therefore address various scenarios for the nature of the ULX since it is not known which, if any, of the sources found are true counterparts. The optical luminosities of the sources are typically in the range 10^4-6 L_sun. In several cases color information is available, with the colors roughly tending to be more red in early-type galaxies. This suggests that, in general, the (potential) counterparts found in early-type galaxies are likely to be older stellar populations, and are probably globular clusters. Several early-type galaxy counterparts have blue colors, which may be due to younger stellar populations in the host galaxies, however these could also be background sources. In spiral galaxies the sources may also be due to localized structure in the disks rather than bound stellar systems. Alternatively some of the counterparts in late-type galaxies may be isolated supergiant stars. The observed X-ray/optical flux ratio is diluted by the optical emission of the cluster in cases where the system is an X-ray binary in a cluster, particularly in the case of a low-mass X-ray binaries in old cluster. (abridged)