The Pulsar Search Collaboratory (PSC) is a collaboration between the Green Bank Observatory and West Virginia University, funded by the National Science Foundation. The PSC program is currently expanding nationwide and engages high school students, teachers, and undergraduate mentors in real-world research by searching for pulsars in data collected with the 100-m Green Bank Telescope. In the process, students learn about observational radio astronomy, radio frequency interference, pulsar timing, and data analysis procedures. The primary goals of the PSC are to stimulate student interest in Science, Technology, Engineering, and Mathematics (STEM) careers, to prepare teachers in implementing authentic research with students by training them within a professional scientific community, and to promote student use of information technologies through online activities and workshops. In this paper, we provide an overview of pulsar science and the data analysis students undertake, as well as a general overview of the program. We then discuss evaluation data collected from participants through a series of survey questions to determine if the program's initial goals were met. The program had a positive impact on the students according to multiple measures, in particular, on their understanding of the nature of scientific inquiry and motivation to pursue STEM career paths.