Evidence for dark matter self-interactions has recently been reported based on the observation of a spatial offset between the dark matter halo and the stars in a galaxy in the cluster Abell 3827. Interpreting the offset as due to dark matter self-interactions leads to a cross section measurement of sigma_DM/m ~ (1-1.5) cm^2/g, where m is the mass of the dark matter particle. We use this observation to constrain singlet scalar dark matter coupled to the Standard Model and to two-Higgs-doublet models. We show that the most natural scenario in this class of models is very light dark matter, below about 0.1 GeV, whose relic abundance is set by freeze-in, i.e., by slow production of dark matter in the early universe via extremely tiny interactions with the Higgs boson, never reaching thermal equilibrium. We also show that the dark matter abundance can be established through the usual thermal freeze-out mechanism in the singlet scalar extension of the Yukawa-aligned two-Higgs-doublet model, but that it requires rather severe fine tuning of the singlet scalar mass.