The cosmic dawn and epoch of reionization mark the time period in the universe when stars, galaxies, and blackhole seeds first formed and the intergalactic medium changed from neutral to an ionized one. Despite substantial progress with multi-wavelength observations, astrophysical process during this time period remain some of the least understood with large uncertainties on our existing models of galaxy, blackhole, and structure formation. This white paper outlines the current state of knowledge and anticipated scientific outcomes with ground and space-based astronomical facilities in the 2020s. We then propose a number of scientific goals and objectives for new facilities in late 2020s to mid 2030s that will lead to definitive measurements of key astrophysical processes in the epoch of reionization and cosmic dawn.