Ranking Ed Schools

Duke beat Carolina last night. We're ranked 5th in the nation. Or 7th. It depends who you ask. There are 4 major ranking systems. The ranking is almost entirely based on outputs: wins and losses. There's a little bit of personal judgment involved.

Ed Schools get ranked, too. How? This is a recurring theme of this blog.

Joe at KIPP DC and Nick at New Visions sent me this NY Times article.

Now U.S. News & World Report is planning to give A through F grades to more than 1,000 teachers’ colleges, and many of the schools are unhappy, marching to the principal’s office to complain the system is unfair.

Here is the letter from concerned deans.

Disclosure: I'm on NCTQ's advisory board. I have seen some of the early drafts of grading system. Here is how US News describes the effort.

Now I blogged about the US News existing method a year ago. I wrote:

But look at their methodology. Almost none of it explores any factors that affect undergrad and masters programs for teachers.

Instead, it’s all about doctoral students. Most of whom have nothing to do with k-12 teaching. Inputs like: # of doctoral degrees granted. GRE scores of doctoral students only — no GRE data of the teachers they’re training. Ratio of faculty to doctoral students, but nothing about ratio of faculty to those preparing to be schoolteachers.

Then there are deans' and superintendents' ratings. Do you think a single dean in America could accurately describe any difference between teacher prep at, say, Penn State and Ohio State (in front of recent grads of those programs)? No way!

Same with superintendents. Most supes don’t hire any teachers. Principals do. Why do supes even participate?

So of course I'm glad to see US News is going to change the ratings. But are they changing in the right way? Here's the pushback:

“Nobody’s against being evaluated or having good reliable information available to the public about how we can prepare better teachers,” said Mary Brabeck, dean of New York University’s school of education. “But what will we know if everybody uses the same textbook? What will that tell us about how you prepare highly effective teachers?”

Sharon P. Robinson, president of the American Association of Colleges for Teacher Education, said the ratings were focusing on superficial “inputs” rather than “outcomes,” like how well teacher graduates perform in the classroom.

For the record, I'm with Ms. Robinson in her emphasis. Outcomes outcome outcomes.

Meanwhile, Kate Walsh of NCTQ says:

“What they want us to do is hold off until a perfect assessment is in place."

Kate may have a point. If we examine the recent history of K-12 standardized testing linked to accountability in 1990s and 2000s, many opponents at first opposed any sort of standardized accountability. Then they realized the public wanted it.

So a new strand was "Yes, of course I want assessment and accountability, but these tests won't do it."

So if you knew their true beliefs -- i.e., what they had been arguing until they realized they were losing -- you knew they wanted no accountability, not just "a better test," and it was pure stall mode.

So some opponents of MCAS-like tests just want to stall -- they want no standardized tests.

Others, like me, see many flaws with the details of MCAS questioning and scoring -- but vastly prefer it to nothing.

* * * *

My thoughts on this dust-up:

1. The status quo is that US News already ranks Ed Schools (the whole thing, not just the teacher prep part). This will continue under the new plan.

2. Ed Schools with good ratings right now promote their US News ranking. For example, it's the lead sentence in U Conn's website.

The University of Connecticut's Neag School of Education is not only the #1 public graduate school of education in the Northeast and the East Coast, it is ranked the 16th best public graduate school of education in the U.S.

We'd expect that, right?

I do that. For example, the Boston Globe rates schools based on MCAS scores. Our charter school does well on MCAS. So we promote that, just like UConn. Our charter school does worse on SAT scores. So we do not promote that. If the Globe suddenly changed, we'd probably freak out a bit.

3). Any change to a rating system creates winners and losers. US News rates high schools, too. (Disclosure: I worked on that project a little bit).

In 2008, we were a Top 100 high school in the nation. Then they changed their methodology. Now we're not a Top 100 high school. (We are on Newsweek's ratings, though). We're not gold medal anymore. We're "silver."

There's an inside-baseball question within charter world on whether high schools with lots of native Spanish speakers are at an advantage. You could get all the seniors to take Advanced Placement Spanish, and your school's ratings would rise quite a bit.

Charters on the East Coast, like ours, have mostly black students who are not native Spanish speakers, though. So we can't easily do that.

Would we? If a) we had a high proportion of kids who were native Spanish speakers, and b) the US News ratings really affected our bottom line (which they don't)? Sheesh. I have to admit, we'd talk about it.

A more meaningful way for us to improve the actual lives of kids, and our ratings too, would be to improve our teaching and tutoring so that kids did better on the AP exams they already take in calculus and English and so forth. That's the work we're honestly trying to do.

But F*&#&$ (Rahm!) that is hard work. Sometimes I think we're getting traction. Sometimes I think it's quicksand and we're getting nowhere.

3. The letter from the Ed School deans at least appears to be quite responsible. It surprised me in a sense. They seem so open to legitimate evaluation of outputs. When you're citing Tom Kane, you're open to real number crunching. There's nothing mushy about his work. They wrote:

We offer to work with U.S. News and Secretary Duncan to develop a methodologically rich outcome-focused study of all teacher preparation programs. Performance assessments that have been developed over the last decade (See Tom Kane’s MET project funded by the Gates Foundation and new statistical methods like Value Added Modeling can lead to robust program assessment.

Race to the Top states are implementing multiple measures of effective teaching, and other states and teacher education programs are voluntarily doing the same. All of these efforts indicate that robust, valid, and useful assessments of the outcomes, in terms of what aspiring teachers know and can do, and the impact these candidates have on student achievement, is possible.

Hell yeah! Sign me up. Let's do that. Just delete "know and can do" (that's still input), and roll with "impact these candidates have on student achievement" (output).

The reason I fear inputs so much to drive ratings, even better inputs, is that we may disagree.

Last night Dick Vitale, the commentator, was asked why Duke's stadium has such a powerful home court advantage. He blathered some stupid stuff about the tradition and program success.

But he's wrong. Many other programs (Kentucky, UNC, UCLA) have the same things. But their home court advantage is lower. I disagree with Dick about the critical input to Duke's home court advantage. I believe Duke is the only stadium I know where students get all the best seats. Not fat cats who calmly watch the games and say: rah. Students in the best seats. So they go nuts the whole game, and their energy radiates upward, making the entire 2 hours noisier.

I want a world where we can disagree about important inputs, because what counts are the outputs, and that's how we're ranked.

4. So let's review:

a. Status Quo: US News rates Ed Schools. I have written that they use silly input measures.

b. NCTQ / US News: proposing to rate Ed Schools on what seem like smarter input measures -- but still input measures.

My understanding is they'll add output measures as they become available, but they won't wait.

c. Opposition to the change which probably includes:

c1. Legitimate opposition from those who want only legit output measurement -- how good are the teachers as measured by learning gains by the kids they ultimately teach?

c2. Red herring opposition from those who want status quo. "Let's cloud the issue by offering to study the issue, but in truth we'll make the perfect the enemy of the good, thereby stalling."

I dunno. I smell an opportunity here to work with "legitimate opposition" deans somehow.

And indeed NCTQ did a webinar yesterday with 450 folks to discuss the issue. I wonder how that went.