The effect of progenitor metallicity on Type Ia Supernovae (SNe Ia) has important cosmological implications due to the need for these standardizable candles to be compared across large spans of cosmic time in which the progenitor stars might have different properties. Theoretical models have come to different conclusions as to the wavelength range impacted by metallicity differences, leading to differing interpretations of the growing sample of UV observations. Recent work has claimed a correlation between the mid-UV flux of SNe Ia measured from Swift grism spectra and the gas-phase metallicities measured for their host galaxies. Here we examine UV photometry for the same objects. We find no significant correlations between the UV-optical colors (or UV/optical count rate ratios) of the SNe Ia and the host galaxy properties of mass or metallicity. The lack of a significant correlation with host galaxy metallicity implies another physical difference other than progenitor metallicity dominates the UV flux differences. We are no longer limited by a lack of UV observations. Rather, understanding the existing observations requires improved theoretical models and a larger parameter space of physical differences.