Evaluation Design

The evaluation design for this project involves both qualitative and quantitative methods, with dual goals of yielding evidence of the impact of the GK-12 project and formative evaluation information to highlight successes and areas for improvement. Data collection tools have been selected or adapted from existing instruments with evidence of reliability and validity. RMC Research works with the principal investigators and GK-12 staff through meetings and other activities to support the use of evaluation data in long-term planning and guidance. RMC Research also provides mid-year and annual reports with recommendations to support continuous improvement. A final summative report documents progress toward program goals over the course of the project, highlights effective strategies, and provides recommendations for sustaining, improving, and replicating project activities.

A number of protocols are utilized to collect quantitative and qualitative information. Interviews and/or focus groups are conducted with middle school teachers and students, graduate Residents, faculty mentors, and project directors for an analysis of program progress and impacts on all involved groups. Pre/post surveys are utilized with graduate Residents and middle school lead teachers to document change on a range of outcome variables and other moderators expected to influence results. These include changes in understanding of the K-12 environment and ideas related to the role of faculty and professionals in education and outreach. Observations are conducted in classrooms to determine evidence of transfer of research, mathematics, and science content activities into practice, and levels of student engagement and Resident-teacher collaboration. Evaluation of the international component includes observations of videoconferencing and multidisciplinary meetings and interviews with participating Residents. Management team meetings and Residents' meetings are periodically observed for a better understanding of project directions and decision making. Finally, document analysis involves an examination of Residents' journal writing and lesson plans to gain an understanding of the progress made in implementing project goals and activities.