Merit pay: “It simply did not do much of anything.”

Today, Vanderbilt University’s National Center on Performance Incentives released a report from a three-year controlled experiment on merit pay in the Nashville schools. Teachers in the treatment group, from fifth through eighth grades, stood to earn an extra $15,000 a year if their students improved enough on math tests. Teachers in the control group could not earn bonuses.

With one exception researchers could not fully explain—fifth grade—they found that students of teachers eligible for bonuses performed no better than other students. The teachers in one group did not approach instruction differently from those in the other, and where they did, it did not affect student achievement. Scores went up for both groups, “a trend that is probably due to some combination of increasing familiarity with a criterion-referenced test introduced in 2004 and to an intense, high-profile effort to improve test scores to avoid NCLB sanctions.”

Are you surprised? I am not. Presuming that merit pay alone would elevate student achievement makes sense if you assume teachers have a hidden trove of skills and effort they are not unloosing on their students only because they lack the proper incentives to do so.

Can we predict what happens now in the edublogosphere? Opponents of merit pay will use this study to further dump on Race to the Top. Supporters will say that it doesn’t matter, because there are other reasons for merit pay (people good at their professions should be paid more, it might attract better teachers—Rick Hess critiques the study in this regard here). Or they will emphasize the limitations of the study. Or both. The researchers, who are proponents for making performance incentives work, emphasize the soundness of the research, while pointing out that they tested only “a particular model of incentive pay.” Others, they wrote, might be more successful.

A more detailed analysis is forthcoming from the center. For now, some survey results: Teachers said that the prospect of bonuses didn’t change their behavior much, because they were already doing the best they could. While they supported merit pay and found no fault with the structure of this particular system, they didn’t think student scores showed whether they were teaching effectively.

What does this mean for reporters? As always, it is important to be clear about what the research does and doesn’t say; we cannot really say that merit pay “works.” Nor can we say that it does not—because we have not yet clearly defined what “works” means.