The Firing Line

In a contentious meeting last Thursday, Houston’s school board stepped out on to a cutting edge that few if any large urban districts have dared tread: using student test scores as the basis to fire teachers.

The new policy, passed in a 7-0 vote with two members absent but almost 1,000 angry teachers in attendance, represented the culmination of years of trial-and-error attempts in the state’s largest school district to connect test scores to pay and job security. The Houston Independent School District already uses the data to dole out some $40 million annually in teacher bonuses, ranging from token amounts to more than $13,000. HISD now moves into the even more controversial territory of basing terminations on so-called “value-added” data, developed by educational statistician William Sanders, which purports to measure all students' improvement in one school year’s time — regardless of their academic starting point — and amalgamate all the scores into one grade for a teacher or teachers.

Such ideas have long been advocated by corporate observers and conservative politicians but have been beaten back by teachers unions that have fought to protect job security and pay scales based strictly on seniority and degrees earned. Students’ performance, the unions argue, is a reflection of a thousand immeasurable variables, chiefly the educational backgrounds and family lives of students, and teachers shouldn’t be blamed for the myriad failures of parents or society as a whole. But the union position has been gaining traction in recent years, not just in Houston and Texas but nationally, too. Now, even the Obama administration, so often regarded as a union lap dog, pushes merit pay for teachers (as well as charter schools, the other burr in the union saddle).

Though the new policy will doubtless bring the issue of teacher performance to a head, the trouble at HISD started long before last week. Setting the stage for the vote was the most recent board elections, when five of nine members' seats were up. Historically, the district’s teachers union — which has vehemently opposed linking test scores to merit pay and negative evaluations — has held great sway in those elections. But this time it got blanked: Two board members opposed by the union drew no challengers, while in two other races, candidates who were endorsed and financially backed by the union lost.

“It gave the other members of the board an awakening that what we’re doing is the right thing,” crows Greg Meyers, the president of HISD’s Board of Education and one of those who ran unopposed. “It gave them courage. … I’d say this board of education, in this district, is without a doubt the most data-driven district, not just in the state, but in the country.”

Gayle Fallon, the longtime president of the Houston Federation of Teachers, sees such pronouncements as the naive hubris of politicians who have lost touch with the both inherent complexities of schooling and the fundamentals of management. “In all my change-theory classes — which every administrator has to take, but unfortunately school boards members don’t — it said the change that is implemented top-down will not be effective. The only change that will be effective involves the people being changed,” Fallon says. “There’s a lot of ways to do accountability, but they’re looking for a silver bullet. They’ve caught on to a couple of pat phrases, like ‘data-driven.’ Sorry, there’s more to teaching than that.”

At the crux of both arguments is whether the data is really that good. Can any one measure — even a complex blend of finely tuned super-data — really assess the value of the intimate relationship between teacher and student? Can a database peak behind thousands of closed classroom doors?

And that’s another fight: The data system in Houston remains a secret. Sanders, a senior research fellow with the University of North Carolina system and an employee of software giant SAS, invented “value-added” data, which now has a wide following among reformers nationwide. How does it work? That’s proprietary corporate information, which the company and HISD have refused to share with the public and the union, which Fallon says has filed three unsuccessful open-records requests for the formula. The union is currently devising a legal strategy to force release of the formula and possibly a court challenge of the entire system of firing teachers based on test scores.

Meyers: The no-excuses revolution

Meyers, elected to school board in 2005, has been working since then with various board colleagues to pioneer the use of value-added data to incentivize excellent teaching — and now, to purge the districts of bad teachers.

Traditionally, public school teachers almost never get fired except for unethical or criminal behavior — merely failing to do the job gets tolerated for years at the expense of students, he and others argue. When all the angry teachers showed up at last Thursday’s meeting to protest the new order, their voices were matched by those of parents and business leaders who voiced full-throated support for what he views as a simple yet revolutionary strategy: Use data to make sure all students get the effective teachers they deserve and to weed out those who can’t hack it. For the first time, test score data has reached a level of sophistication that such decisions can be made fairly and confidently. Value-added data takes the mystery out of how a particular teacher performed.

Consider, for instance, two scenarios. One ninth-grade teacher has students who enter her math class working, on average, at a third-grade level. She raises them to an average of the sixth-grade level — and gets a big bonus for boosting them three grade levels in one school year, even though they remain below average. Another ninth-grade teacher starts with stronger students, working on a seventh-grade level. But by year’s end, they show no improvement on state and national tests. This happens three years straight, despite extra help and mandatory training for the teacher — and so he gets terminated. This occurs even though his students, in raw terms, perform at a higher level than those of first teacher, who raked in the bonus.

And so HISD can eliminate the knee-jerk excuse: that the student’s low performance isn’t the teacher’s fault, because he arrived well behind, bringing problems at home. Value-added data measures purely what a teacher did — despite the challenges.

“I truly believe every parent expects their child to grow academically at a minimum of one school year every year,” Meyers says. “This is not an effort to fire teachers, or to go on a witch hunt to get rid of teachers. … We can identify some material weaknesses to help them in their careers.”

But if a teacher hasn’t produced adequate student growth after three years — despite planned “rich and profound” training efforts — “then maybe it’s time we sit down and talk to them and say it’s a time for career change,” Meyers says. 

Fallon: Beware of the “black box”

Fallon, the president of the teachers union since the early 1980s, has never seen so many teachers so angry as they left last week’s meeting. The ultimate results of the board’s new policy, she predicts, will be that the best teachers will avoid difficult teaching situations — the exact opposite of what districts want them to do — or bail out of Houston ISD entirely. More and more, Houston teachers live in the surrounding suburbs because of high-priced real estate in the city, and many might decide for the relatively easy life of suburban schooling. “They’re going to hurt themselves on recruitment badly. Teachers are not gamblers,” she says. “Why would they drive through a couple of districts to come here and by evaluated by a system they don’t perceive is fair?”

The argument for value-added data is both seductive but dangerous, Fallon asserts. “There’s an interesting study out of Arizona State that says the biggest problem with these value-added systems is that they have intuitive appeal to people. They just kind of on the surface make sense. The problem is, the devil’s in the details,” Fallon says. “That was the same study that said the particular model we used, developed by Bill Sanders, I think they referred to it as ‘opaque,’” meaning confusing and lacking in transparency. A second study, by RAND, concludes that the research is “currently insufficient to support the use of (value-added models) for high-stakes decisions about individual schools or teachers." Meanwhile, the district and SAS, the company that sells the data system, won’t let anyone peak behind the curtain to even see how value-added scores get assigned to teachers.

“This formula was designed and implemented in a black box,” Fallon says. “How are you spending state money on something that people can’t see?”

Fallon attacks the two basic premises of the would-be reformers: that bad teachers never get fired and are union-protected, and that any data system alone should be used to evaluate teachers and mete out rewards and consequences. Fallon concedes that only a handful of teachers get formally “fired,” and almost never on purely academic grounds. But informal terminations happen all the time — and with the help of the union.

“I probably take more people out of the classroom than higher administration staff in any given year. We can say things to them their bosses can’t,” she says. “We can sit them down and say,  ‘Let’s have a frank discussion about your ability.’ … If you’re doing a lousy job in most professions, you know it. Let me tell you, if you’re in classroom and you’re doing a lousy job, your life is beyond a living hell.”