Eric.ed.gov – How Non-Linearity and Grade-Level Differences Complicate the Validation of Observation Protocols

Teacher evaluation is currently a major policy issue at all levels of the K-12 system driven in large part by current US Department of Education requirements. The main objective of this study is to explore the patterns of relationship between observational scores and value-added measures of teacher performance in math classrooms and the variation in these relationships across grade levels. While the MET analyses used a single composite score consisting of a simple average of the eight component scores of the protocol, in our work we treated each component separately since each measures a separately definable aspect of classroom practice. Specifically, across all the components, the authors pose the following questions: (1) Do the relationships between observation scores of math teachers and their value-added scores tend to be non-linear?; (2) Is there a difference in the relationships associated with developmental changes as indicated by grade levels (four through eight)?; and (3) To what extent the observed changes are associated with increasing departmentalization (in upper elementary and middle school)? The relationship between classroom observation instruments and teacher effectiveness is often presumed to be a simple linear mapping. The authors find that the relationship is complex: it is typically non-linear and the shape of this relationship appears to evolve with the grade level progression. Careful analysis of such relationships will improve understanding of teacher effectiveness and result in more accurate teacher evaluation systems. Better evaluation systems will in turn generate more useful administrative data for use in effectiveness research. Tables and figures are appended.