Large area catalogs of galaxy clusters constructed from ROSAT All Sky Survey provide the base for our knowledge on the population of clusters thanks to the long-term multiwavelength efforts on their follow-up. Advent of large area photometric surveys superseding in depth previous all-sky data allows us to revisit the construction of X-ray cluster catalogs, extending the study to lower cluster masses and to higher redshifts and to provide the modelling of the selection function. We perform a wavelet detection of X-ray sources and make extensive simulations of the detection of clusters in the RASS data. We assign an optical richness to each of the 24,788 detected X-ray sources in the 10,382 square degrees of SDSS BOSS area, using redMaPPer version 5.2. We name this survey COnstrain Dark Energy with X-ray (CODEX) clusters. We show that there is no obvious separation of sources on galaxy clusters and AGN, based on distribution of systems on their richness. This is a combination of increasing number of galaxy groups and their selection as identification of an X-ray sources either by chance or due to groups hosting an AGN. To clean the sample, we use a cut on the optical richness at the level corresponding to the 10\% completeness of the survey and include it into the modelling of cluster selection function. We present the X-ray catalog extending to a redshift of 0.6 down to X-ray fluxes of $10^{-13}$ ergs s$^{-1}$ cm$^{-2}$. We provide the modelling of the sample selection and discuss the redshift evolution of the high end of the X-ray luminosity function (XLF). Our results on $z<0.3$ XLF are in agreement with previous studies, while we provide new constraints on the $0.3<z<0.6$ XLF. We find a lack of strong redshift evolution of the XLF, provide exact modeling of the effect of low number statistics and AGN contamination and present the resulting constraints on the flat $\Lambda$CDM.