Searching for the Green Bay Packers of Ed Schools

If you asked which NFL team was best right now, you'd get some debate. The Packers have the best case. But if you asked if the Packers are waaay better than the average NFL team, you'd get no debate. Yes.

Researchers are trying if there's a "Packers" in teacher prep. That is, an Ed School or alternative program anywhere in the USA that is obviously way better than the others in the state. The thinking goes like this. The NFL is a copycat league. Perhaps if there were some clear standouts -- based on evidence, not reputation -- other teacher prep programs could copy. Then the USA could produce more good teachers. All the land would rejoice.

Alas, to date, the researchers don't have a Packers. The best teacher prep program is only a tiny bit better than an average teacher prep program, in the handful of studies ever conducted on this question. Here's how such a study works.

*Examine both traditional universities or non-traditional programs that train teachers.

*Track down their alums in a state database.

*See whose alums are teachers which generate larger-than-normal standardized-test growth (value-add). I.e., how much progress is made by kids who are taught by Orin, an alum of Leafy State? What about similar kids who are taught by Laura, from Madre U? It's a decidedly limited approach, but the only empirical one out there.

But before delving into the new study, from Louisiana, let's look back at some old studies.

1. A 2009 study in New York City examined alums from 20 teacher prep programs. (Susanna Loeb and Pam Grossman and their 3 colleagues list the programs anonymously, by number instead of university name).

The #1 program (listed as "7" in the graph, possibly an homage to George Constanza) was about 0.07 standard deviations better than the average. Good job, Institution 7.

But the big picture here: most programs are not that far apart. They're not identical. There are differences. But the differences are not huge.

The authors essentially say that you'd much rather hire a top 25% grad from the worst Ed School than a low 25% teacher from the best Ed School. That there's much more variation within each teacher prep program than among them.

That's not true in everything. Some institutions are way better than others. For example, my guess is if you ran a management consulting business, you'd probably rather hire a bottom 25% grad of Harvard Business School than a top 25% grad from University Of Phoenix's online MBA.

2. Dan Goldhaber just did a similar study of Ed Schools in Washington State, which you can read about here. The top school was about 0.05 standard deviations better than the average.

So: same story. Like in New York City, the Ed Schools and alternative teacher prep programs in Washington State are all clustered closely together. There are programs like Walla Walla that produce slightly better teachers, but there's not an obvious program that is the standout.

3. Okay. That was context. The New Study is from Louisiana.

Here's a table which lists several teacher prep programs, and then how math teachers did.

Ah. So: New Teacher Project and University of New Orleans were the top alternative and traditional programs, respectively. They had mean scores of "5.0" and "2.1" respectively.

But what is the meaning of those numbers? We need the "mean scores" converted to standard deviations, so they're like the other studies.

I couldn't find a "key" in the link. But one of the co-authors, Kristin Gansle, kindly explained:

The mean as reported here refers to the number of points on the state achievement test in that particular content that new teachers from a teacher preparation program add to the expected value for each student. The mean on those tests is approximately 300 and the standard deviation is approximately 50.

Gotcha. So New Teacher Project teachers were 5.0 points out of 50, or 0.10 standard deviations, "better than the average teacher." And UNO alum teachers were 2.1 points out of 50, or 0.04 standard deviations, "better." Congrats to both.

Big picture, the Louisiana numbers look a lot like NYC and Washington State. By comparison, some of the top charter schools in the nation -- remember the average charter is not so hot, but there are standouts) -- are generating gains that are 0.40+ standard deviations above average. That effect size is so high, it literally "shocked" the economists who studied those charters ("We had to recheck our calculations, we couldn't believe it" I recall Tom Kane saying at the 2009 unveiling of one such paper.) Schools like Edward Brooke and Excel and Roxbury Prep are Green Bay Packers caliber, where most people (including many in the local school district) acknowledge they are waaaay above average.

So the search continues: is there an Ed School or teacher prep program whose graduates clearly blow away those of everyone else? Is anyone ready to lay a claim to being the Green Bay Packers?