New Orleans CREDO

1. Charter critics say many charters are not good. They are correct.

One of the most-cited studies by charter opponents was done by a Stanford researcher named Macke Raymond. It is called the CREDO study. Here, for example, is Diane Ravitch:

Some fact-checking is in order, and the place to start is with the film’s (Waiting for Superman) quiet acknowledgment that only one in five charter schools is able to get the “amazing results” that it celebrates. Nothing more is said about this astonishing statistic.

It is drawn from a national study of charter schools by Stanford economist Margaret Raymond (the wife of Hanushek). Known as the CREDO study, it evaluated student progress on math tests in half the nation’s five thousand charter schools and concluded that 17 percent were superior to a matched traditional public school; 37 percent were worse than the public school; and the remaining 46 percent had academic gains no different from that of a similar public school. The proportion of charters that get amazing results is far smaller than 17 percent.

I agree, by the way, with Ravitch on this point. While I have some technical quibbles with the study, the big picture -- that many charters are not good -- is correct.

Ravitch was right to ask of KIPP:

What I want to say to KIPP, because I really really admire what you are doing. You have an excellect reputation, you get great results. Thousands of new charters will be created in the wake of your success. But your results are not typical. Warn President Obama and Secretary Duncan.... that the wonderful results you get are unusual they are not typical of the charter sector.

I do that. I try to warn state policymakers when they visit. And with my nanoscopic megaphone (miniphone?). Like here:

Nationally, charter critics have a legitimate point in terms of quality. It’s mixed.

One interesting question is -- if nationally charters are not great, what are the regions which seem best?

Studies of this ilk get less attention. For example if you google "stanford study charter schools raymond" you get 315,000 results. But if you google "harvard study charter schools kane" you get 143,000 results. The Harvard study found Boston charters did unusually well.

A recent MIT study found that among urban charters in Massachusetts, the first point of departure seems to be whether the school has a "no excuses" approach to school culture. A year ago I pitched in some ideas for a forthcoming Roland Fryer study on the same question in NYC: what makes some charters work.

2. New Orleans

Today there's a new regional study out. I hope charter critics today praise the teachers and leaders working in New Orleans. The same scholar (Raymond), using her exact same methodology of which critics approve, found that New Orleans charter schools are doing unusually well.

From the Times Picayune today:

Roughly half of the New Orleans charter schools that have produced enough test scores to measure are improving student performance in reading or math at a significantly faster rate than competing traditional schools, according to an analysis by Stanford University researchers. Another 12 of the 52 charter schools included in the latest analysis performed no better than their traditional peers, and 12 schools lagged significantly behind. The figures come from Stanford's Center for Research on Education Outcomes, or CREDO, which has been drafted by the nonprofit New Schools for New Orleans to help decide which charter schools should receive grant money to expand with additional campuses.

Let's review.

The national charter numbers were: 17% better, 46% same, 37% worse. This study has been "approved" and repeated by many charter critics.

The NOLA charter numbers were: 50% better, 25% same, 25% worse. Using the same researcher and methodology. Which is still imperfect. But at least apples to apples.

Note that New Schools For New Orleans (which supports our small teacher coaching experiment) doesn't think their charter schools are near as good as they need to be. To their credit. They believe the kids could do much better academically. And their mission is to make that happen.

3. Analysis

Let's set aside the larger charter debate for a moment. Of course to survive, to even exist as charters, we can't lose this debate. And that includes calling out the fair-minded charter critics at moments like this -- a study they've blessed shows a result that, while doesn't vitiate their anti-charter case (i.e., it's not a national study), it does suggest perhaps they should laud New Orleans and agree that certain regions seem to produce/attract/have unusually strong charters.

But staying focused on the charter "existence" debate obscures the more pressing "insider" question. Let's even set aside what leads to a "good charter." I'm not interested in the 50% "good" versus the 25% "bad."

To my knowledge, nobody has studied this question: among charter schools who have a very strong culture and generally high test scores, what leads to standouts? I.e., if you only studied those charters, what would describe the top few? In New Orleans, for example, what seems to be true of the top 10%?

Our former principal Jorge Miranda gets to explore that question this year. As director of high schools for KIPP national, he has a cohort of 19 to examine. He'll make observations about standouts.

Some argue the key lever, after kids generally behave well and try hard, is curriculum. Others think it's somehow doubling down on character (like grit, persistence, et al). Still others believe the key lever is data linked to much more customized follow-up for each kid. There's an argument, too, to make most of the actual questions posed during each class much tougher (higher-order thinking).

Another view comes from MathTeacher, who works at one of the best charters in Boston (Edward Brooke). He emailed me last night and mused:

At my former charter school 5 years ago (a well-regarded one), weekly early release day was mostly filled with department meetings and grade level meetings and some whole school staff mtgs. We had a couple of formal PD days that were filled with some staff and external sessions re: diversity, writing, etc. It always felt like we were brainstorming how to make the school better (outside the classroom), rather than the teaching better inside the classroom.

At Edward Brooke, however, we have weekly pd sessions that contain a short staff mtg for shoutouts or observations debriefs, a whole school or break out group session on improving some aspect of our teaching vis a vis our teaching rubric, co-planning time and a grade-level (k-5) or department level (6-8) meeting to work on some focus topic we are all working on as teachers.

It's one reason that I love to teach here -- always a focus on getting better as teachers and teams of teachers.

My view is that in schools which are disorderly, that is the first thing to work on, until there is order and effort from (often) reluctant kids. Usually this needs to start with massive proactive teacher outreach to parents, which is very labor intensive and (to my mind) impossible (or simply not worthwhile) to impose upon reluctant teachers who disagree with this strategy.

But how to go to the next level beyond strong school culture, something we're trying to do in our own MATCH schools? We're pondering that, asking teachers, principals, outsiders. Paul's argument is that an unusually powerful (and satisfying to him) lever is to shift improvement focus from whole school to individual teachers -- in a way they (the teachers) describe as meaningful. (Because obviously there's a million ways to try to "force it" and just totally waste everyone's time).

If you've got a bad-ass teacher like Paul F (we send our teacher residents to observe his class) saying "one reason I love this school because I improve here"....that signifies a culture of excellence. And though I've only visited a handful of NOLA charters, I heard that same vibe from teachers when I visited Sci Academy. I'll describe more of our fledgling efforts here at MATCH later in the school year, and welcome your thoughts.