Eric.ed.gov – Due Diligence and the Evaluation of Teachers: A Review of the Value-Added Analysis Underlying the Effectiveness Rankings of Los Angeles Unified School District Teachers by the “Los Angeles Times”

On August 14, 2010, the “Los Angeles Times” published the results of a statistical analysis of student test data to provide information about elementary schools and teachers in the Los Angeles Unified School District (LAUSD). The analysis, covering the period from 2003 to 2009, was put forward as an evaluation of the effects of schools and their teachers on the performance of students taking the reading and math portions of the California Standardized Test. In Los Angeles, teachers were classified into one of five levels of “effectiveness” for their teaching in reading, math and a composite of the two. The decision by the “L.A. Times” to make these results publicly available at a dedicated web site, and to publish an extensive front page story that contrasted–by name–teachers who had been rated by their level of effectiveness was promptly criticized by many as a public “shaming” of teachers. The purpose of the present report is to evaluate the validity of the ratings themselves, not to weigh in on the wisdom of the decision by the “L.A. Times” to publish teacher effectiveness ratings. The value-added analysis of elementary school teachers in the LAUSD was conducted by Richard Buddin, a senior economist at the RAND Corporation. As part of his analysis, Buddin produced a white paper entitled “How Effective are Los Angeles Elementary Teachers and Schools?” The authors’ first objective is to provide a critical review of the analysis and conclusions reached by Buddin. The authors conduct this review by evaluating whether the evidence presented in Buddin’s white paper supports the high-stakes use of value-added estimates to classify teachers as effective or ineffective. They also attempt to replicate Buddin’s empirical findings through an independent re-analysis of the same LAUSD data. Their second objective is to scrutinize a premise of Buddin’s analysis that was unexamined in his white paper: that he has successfully isolated the effects of teachers on their students’ achievement. To this end they present the results from the kind of “sensitivity analysis” that one should expect as due diligence any time a value-added model is being proposed as a principal means of evaluating teachers. They highlight especially those cases where the sensitivity analysis leads to substantively different inferences than those suggested on the basis of Buddin’s white paper. Technical details is appended. (Contains 7 tables, 3 figures and 41 notes.)