We describe the structural and kinematic properties of the first compact stellar systems discovered by the AIMSS project. These spectroscopically confirmed objects have sizes ($\sim$6 $<$ R$_{\rm e}$ [pc] $<$ 500) and masses ($\sim$2$\times$10$^{6}$ $<$ M$_*$/M$_\odot$ $<$ 6$\times$10$^{9}$) spanning the range of massive globular clusters (GCs), ultra compact dwarfs (UCDs) and compact elliptical galaxies (cEs), completely filling the gap between star clusters and galaxies. Several objects are close analogues to the prototypical cE, M32. These objects, which are more massive than previously discovered UCDs of the same size, further call into question the existence of a tight mass-size trend for compact stellar systems, while simultaneously strengthening the case for a universal "zone of avoidance" for dynamically hot stellar systems in the mass-size plane. Overall, we argue that there are two classes of compact stellar systems: 1) massive star clusters and 2) a population closely related to galaxies. Our data provide indications for a further division of the galaxy-type UCD/cE population into two groups, one population that we associate with objects formed by the stripping of nucleated dwarf galaxies, and a second population that formed through the stripping of bulged galaxies or are lower-mass analogues of classical ellipticals. We find compact stellar systems around galaxies in low to high density environments, demonstrating that the physical processes responsible for forming them do not only operate in the densest clusters.