Improving Mathematics Discourse Among Students: Results and Lessons Learned from a 4-Year Study
The table below identifies the primary research and evaluation questions relating to mathematics discourse, the data sources for the information, and some of the analyses performed using the data collected.
Evaluation Question | Data Sources | Possible Analyses |
Primary Research Question | ||
Can student achievement in mathematics be significantly improved by increasing the quantity and quality of meaningful mathematical discourse in mathematics classrooms? | Classroom Observations, Implementation Rubric Ratings and Student Assessment Data | Analyze the relationship between the classroom observation mathematical
discourse index of the teachers participating in the research study and the mathematics
achievement of their students. |
Project Impacts |
||
To what extent has the project increased the percentage of students who demonstrate proficiency on the Oregon State Mathematics Assessments for Grades 3, 5, 8, and 10 and district mathematics assessments in all participating schools? | Student Assessment Data | Analysis of student mathematics achievement
as a function of the level of school implementation of key aspects of the OMLI
project. |
What impact has the project had on the teaching practices of the participating teachers? | Teacher and Principal Survey |
Analyze the mathematical discourse, classroom practices, and pedagogical preparedness indices for change over time for all participating teachers. |
To what extent has the project increased the quantity and quality of meaningful mathematical discourse between teachers and students and among students in the classrooms of the participating K-12 schools? | Teacher and Principal Survey, and Classroom Observations | Analyze the discourse,
classroom practices, and pedagogical preparedness indices for change over time for all
participating teachers. |
To what extent has the professional development prepared the teacher leaders for their leadership role in terms of both content knowledge preparedness and pedagogical knowledge? | Professional Development Database, Content Knowledge Assessment of Teachers, Teacher and Principal Survey | Review the professional development participation and content knowledge
assessment data to determine the extent to which the teacher leaders are meeting their
professional development obligations and the extent to which they are gaining
mathematics content knowledge. |
Data Collection tools and Instruments
Professional Development Database-RMC Research developed a comprehensive online database that contains information about the professional development participation of all participating teachers (http://www.rmccorp.com/).
Implementation Rubrics-With input from the TDG site visit staff, RMC Research developed a 13-trait scoring rubric for the site visiting staff to use to rate the level of OMLI practices implementation in each school as of the end of the 2006-2007 and 2007-2008 school years. Traits measured using the rubric include quality and implementation of the action plan, leadership quality of the SLT, school/district policies and practices, school priority of mathematics, scope of professional development activities, use of professional leadership tasks and protocols, and focus on discourse.
Student Assessment Data-The school is the primary unit of change for the OMLI project. Thus the evaluation examines trends in school-level student performance on the mathematics portion of the standardized state assessment and compares the performance of the schools participating in the OMLI project to statewide averages. Student achievement is also analyzed as a function of the level of implementation of key elements promoted through the OMLI professional development.
OMLI Classroom Observation Protocol-RMC Research worked closely with the OMLI leadership to develop a classroom observation protocol that was used to gather quantitative data about the quality and quantity of mathematical discourse observed among students during a lesson observation. One aspect of the observation protocol involved a taxonomy of discourse type that observers used to classify each incident of mathematical discourse observed among the students. The taxonomy level correspond to the level of cognitive demand associated with each type of discourse; answering, stating, or sharing represents the lowest cognitive demand, whereas justification and generalization represents the highest cognitive demand. Quantity of discourse was measured by the frequency of each type of discourse. The quality of discourse was measured by level of cognitive demand predominantly observed.
Teacher & Principal Survey-RMC Research, in cooperation with the OMLI leadership, administered surveys to mathematics teachers and principals in participating schools in the OMLI project in spring 2005, winter 2006, winter 2007, and winter 2008. The survey consists of a number of scales calculated from responses to clusters of item. Scales were tested for reliability and address constructs such as classroom practices, student discourse, traditional practices, content preparedness, pedagogical preparedness, and school climate.
Content Knowledge Assessment of SLT Teachers-At the conclusion of each summer institute, OMLI staff administered a postsurvey of mathematics content knowledge to all SLT teachers. The presurvey was administered at the beginning of the 2005 summer institute or at the beginning of the first summer institute a teacher attended. The surveys, developed by OMLI staff at Oregon State University, comprised a series of mathematics problems developed and tested at The Study of Instructional Improvement and The Learning Mathematics for Teaching Project at the University of Michigan.
