We investigate the rest-frame UV morphologies of a large sample of Lyman-a emitters (LAEs) from z~2 to z~6, selected in a uniform way with 16 different narrow- and medium-bands over the full COSMOS field. We use 3045 LAEs with HST coverage in a stacking analysis and find that they have M_UV~-20, below M*_UV at these redshifts. We also focus our analysis on a subsample of 780 individual galaxies with i_AB<25 for which GALFIT converges for 429 of them. The individual median size (re~1 kpc), ellipticities (slightly elongated with (b/a)~0.45), S\'ersic index (disk-like with n<2) and light concentration (comparable to that of disk or irregular galaxies, with C~2.7) of LAEs show mild evolution from z~2 to z~6. LAEs with the highest rest-frame equivalent widths (EW) are the smallest/most compact (re~0.8 kpc, compared to re~1.5 kpc for the lower EW LAEs). When stacking our samples in bins of fixed Lya luminosity and Lya EW we find evidence for redshift evolution in n and C, but not in galaxy sizes. The evolution seems to be stronger for LAEs with 25<EW<100 {\AA}. When compared to other SFGs, LAEs are found to be smaller at all redshifts. The difference between the two populations changes with redshift, from a factor of ~1 at z>5 to SFGs being a factor of ~2-4 larger than LAEs for z<2. This means that at the highest redshifts, where typical sizes approach those of LAEs, the fraction of galaxies showing Lya in emission (and with a high Lya escape fraction) should be much higher, consistent with observations.