Examining Teacher and Student Outcomes in the Math in the Middle Institute Partnership
The evaluation uses a quasi-experimental design that compares outcomes for M2 lead teachers and their students to those of teachers and students in a comparison group. Pre- and post-test mathematics achievement data were collected for all Grade 5-8 students in Lincoln Public Schools during the period of the evaluation. Scale scores on the Metropolitan Achievement Test were provided for mathematics procedures, mathematics concepts and problem solving, and total math. Raw scores for district-developed criterion-referenced tests were provided in the areas of algebra, computation, data analysis, geometry, measurement, and numeration. Individual student identifiers were provided so student data could be linked across years. A series of regression analyses was conducted to examine the relationship between teacher participation in M2 and student achievement outcomes. Specifically, the impact of participation in M2 on spring student achievement data was examined, using several variables to control for prior achievement, differences among students, and school effects.
M2 participants completed the Learning Mathematics for Teaching: Survey of Content Knowledge for Teaching Middle School Mathematics (Hill & Ball, 2004) as part of their participation in the M2 Institute. The instrument was administered to each cohort of teachers at their first M2 meeting, 2 years later at the completion of their participation in the program, and for a 1 year follow up after their participation. Scale scores were created based on item responses and repeated measures ANOVA was used to assess change in scale scores over time.
A teacher survey was administered to each M2 participant when s/he was first accepted to participate and in the spring of each subsequent year for a total of three administrations. Surveys were also administered to comparison teachers in the Lincoln Public Schools. The teacher survey includes questions about professional development participation; readiness to teach math topics; ratings of capacity to serve as an instructional leader; philosophy of math teaching and learning; math curriculum and instruction; ratings of learning community, teacher collaboration, collegiality and academic climate; and teacher background information. Teachers to whom M2 Institute participants serve as mentors were also asked to respond to survey items that address the roles of lead teachers, mentorship relationships and the quality of their leadership (e.g., training, peer coaching and observation activities). To examine change over time for survey responses of participants, repeated measures analysis of variance (ANOVA), along with post-hoc analyses (least significant difference tests), was used.
The research team is using data collected as part of the evaluation to further explore impact on student achievement outcomes. Various challenges have limited the extent to which impact on student achievement could be explored, particularly in small rural districts. Challenges included lack of a uniform state assessment across districts in Nebraska, differences in test administration timing, and difficulty accessing individual level student data that are linked to classroom teachers. The research team is exploring methods for integrating disparate assessments to better explore impact on achievement and increase the reliability and generalizability of findings.
