We test the ability of strong lensing data to constrain the size of a central core in the dark matter halos of galaxy clusters, using Abell 611 as a prototype. Using simulated data, we show that modeling a cluster halo with ellipticity in the gravitational potential can bias the inferred mass and concentration, which may bias the inferred central density when weak lensing or X-ray data are added. We also the highlight the possibility for spurious constraints on the core size if the radial density profile is different from the assumed model. These systematics can be ameliorated if central images are present in the data. Applying our methodology to Abell 611 and imposing a reasonable prior on the stellar mass-to-light ratio restricts the core size to be less than about 4 kpc, with a minimum reduced $\chi^2$ of 0.28 for 0."2 positional errors. Such small cores imply a constraint on the dark matter self-interaction cross section of the order of $0.1\ \mathrm{cm^2/g}$ at relative velocities of about 1500 km/s.