We present a detailed study of the early phases of the peculiar supernova 2011ay based on BVRI photometry obtained at Konkoly Observatory, Hungary, and optical spectra taken with the Hobby-Eberly Telescope at McDonald Observatory, Texas. The spectral analysis carried out with SYN++ and SYNAPPS confirms that SN 2011ay belongs to the recently defined class of SNe Iax, which is also supported by the properties of its light and color curves. The estimated photospheric temperature around maximum light, T_{phot} ~8,000 K, is lower than in most Type Ia SNe, which results in the appearance of strong Fe II features in the spectra of SN 2011ay, even during the early phases. We also show that strong blending with metal features (those of Ti II, Fe II, Co II) makes the direct analysis of the broad spectral features very difficult, and this may be true for all SNe Iax. We find two alternative spectrum models that both describe the observed spectra adequately, but their photospheric velocities differ by at least 3,000 km/s. The quasi-bolometric light curve of SN~2011ay has been assembled by integrating the UV-optical spectral energy distributions. Fitting a modified Arnett-model to L_{bol}(t), the moment of explosion and other physical parameters, i.e. the rise time to maximum, the ^{56}Ni mass and the total ejecta mass are estimated as t_{rise} ~14 +/-1 days, M_{Ni} ~0.22 +/- 0.01 M_{sol} and M_{ej} ~0.8 M_{sol}, respectively.