Rising Above a Gathering Storm of Data: Triangulating Findings Through Formative and Summative Evaluation Methodologies

Author: Sonya Martin

The evaluation of teacher development programs, need valid and scalable measures of teaching practice. We have developed and validated the Science Lesson Plan Analysis Instrument (SLPAI) for quantitative evaluation of teacher-generated multi-day lesson plans. This presentation introduces the SLPAI as a complement to surveys and classroom observation, and demonstrates its use in two pilot studies. The SLPAI was developed by adaptation from a general lesson plan review protocol from an established document (Kahle & Scantlebury, 2006), literature (Bransford, et al, 1999; Wiggins & McTighe, 2001; Brickhouse, 1990; Chinn & Malhotra, 2002; Crowther, Lederman, & Lederman, 2005) and policy documents (NCR, 1996), and results from an in-program pilot implementation to refine items, modify scoring mechanisms, and to improve inter-rater reliability. The SLPAI was used formatively, to measure the teaching practices of incoming program cohorts and to tailor program instruction in a degree-granting MSP funded program for in-service science teachers. The SLPAI was also used to track changes in teaching practice and pedagogical knowledge of teacher participants over time, providing summative evidence of program effectiveness.

Context of the work to be presented:
This presentation shares evaluation findings of an MSP Institute for in-service science teachers. Study participants were part of a Masters degree-granting program, which spans three summers and the two intervening academic years, and requires completion of eight science content courses and two science education courses. The purpose of this presentation is to share findings from the development and validation of the SLPAI for quantitative evaluation of teacher-submitted multi-day lesson plans. The This presentation demonstrates our use of the SLPAI in a pilot study examining teacher change in a program for middle grades science teachers and high school chemistry teachers which seeks to strengthen teacher content knowledge and pedagogical content knowledge to improve science instruction in grades 5-12 classrooms.