Using Value Added Assessments for Teacher Evaluations Seriously Flawed

The Washington Post article this morning entitled "Study: Error Rates High When Student Test Scores Used to Evaluate Teachers" should cause policymakers to pause in their push to tie teacher evaluations to test scores. The actual study entitled "Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains" published by the US Department of Education's National Center for Education Evaluation and Regional Assistance can be downloaded here for those who want to see the actual study. While trying to sort through all of the statistics, one thing is clear, there is enough room for error for a teacher to be improperly dismissed from a job based on test scores. In the hands of a careless and not-very-conscientious school administrator, teachers could be fired or be kept from receiving tenure unfairly.

One point the study makes clear is this:

"Our results strongly support the notion that policymakers must carefully consider system error rates in designing and implementing teacher performance measurement systems that are based on value-added models."

As states have rushed to satisfy Secretary Arne Duncan to qualify for the Race to the Top funding, I doubt very much "careful consideration" has occurred. While some states like Colorado have given themselves time to develop such a performance evaluation system, I fear that others will try to find a way to cobble together a system using existing testing instruments and teacher evaluation systems. The problem with such cobbling, you end up with a product that might satisfy the letter of the law or some state board policy, but the product itself is simply inferior at best and downright hurtful to education at its worst. Policymakers from Secretary Duncan down to the state levels need to understand that when it comes to using testing, in any shape or form, for measuring teacher performance, there is no room for cobbling. Fairness must rule the day.

The study also warns that it is important for policymakers to consider error rates when using value-added estimates for making decisions regarding tenure and hiring-firing decisions.

"Consideration of error rates is especially important when evaluating whether and how to use value-added assessments for making high-stakes decisions regarding teachers (such as tenure and firing decisions.)"

As states push into this uncharted territory of using test scores to assess teacher performance, it is important that educators keep a watchful eye on the teacher performance evaluation systems developed by their own states. They need to understand how those instruments are used in evaluations and know their limitations. Even though the study goes on to say that "teacher value-added estimates in a given year are fairly strong predictors of subsequent-year academic outcomes in teacher's classes," I'm not sure anyone would want to be dismissed or not receive tenure on "fairly strong evidence." That threshold for evaluation is too weak. The study goes on to talk about the importance of "mitigating these error effects" if value-added assessments are used in teacher evaluations. It is the mitigation measures that state policymakers create that are most worrisome.

As an administrator and education advocate, my problems with the whole idea of using test scores in any way to evaluate teachers is the question of fairness. I do not want the job of deciding the future employment of a teacher based on "fairly strong evidence." There is too much danger of being unfair. People who write policy for administrators to implement do not see the effects of that policy firsthand, unless they leave their offices and venture to the front-lines of implementation. My fear is that in the rush to satisfy politicians like Secretary Arne Duncan, we implement a system that has unfairness built in, or it gives unscrupulous administrators the means to be unfair.

This is simply another example of how our current US Department of Education under Secretary Arne Duncan is pushing a reform measure that looks good on paper, and perhaps looks good from the eyes of non-educator, but in implementation it may cause downright harm to our Great American Education System.