We compare ultraviolet (UV) and optical colors of a sample of 29 type Ia supernovae (SNe Ia) observed with the Swift satellite's UltraViolet Optical Telescope (UVOT) with theoretical models of an asymmetric explosion viewed from different angles from Kasen & Plewa. This includes mid-UV (1600-2700 Angstroms; uvw2 and uvm2) and near-UV (2700-4000 Angstroms; uvw1 and u) filters. We find the observed colors to be much redder than the model predictions, and that these offsets are unlikely to be caused by dust reddening. We confirm previous results that high-velocity SNe Ia have red UV-optical colors. When correcting the colors for dust reddening by assuming a constant b-v color we find no correlation between the uvw1-v or u-v colors and the ejecta velocities for 25 SNe Ia with published velocities and/or spectra. When assuming an optical color-velocity relation, a correlation of 2 and 3.6 sigma is found for uvw1-v and u-v. However, we find that the correlation is driven by the reddening correction and can be reproduced with random colors which are corrected for reddening. The significance of a correlation between the UV colors and the velocity is thus dependent on the assumed slope of the optical color-velocity relation. After such a correction, the uvw1-v versus velocity slope is shallower than that predicted by the models and offset to redder colors. A significant scatter still remains in the uvw1-v colors including a large spread at low velocities. This demonstrates that the NUV-blue/red spread is not caused solely by the photospheric velocity. The uvm2-uvw1 colors also show a large dispersion which is uncorrelated with the velocity.