Dear Matt DiCarlo

A friendly open letter to Matt DiCarlo at the Albert Shanker Institute. Dear Matt,

I'm a fan. Keep writing!

I noticed you have a Phd from Cornell. It's freaking cold there. I assume that led you to spend a ton of time in the library, learning stats n stuff. You learned well and do great work with numbers.

I thought your KIPP post was quite good. Two quibbles I'd like to examine here.

First, the big picture of agreement. You write:

We can and should discuss the possibility of unmeasured factors such as peer effects, but it seems unlikely that these factors would come close to explaining away the estimated impacts. In general, KIPP schools are well-run and they do a good job with their students. At this point, arguing otherwise is unsupportable.

At the same time, it’s worth noting that Mathematica’s analysis found that roughly one in three KIPP middle schools did not produce significantly better results in reading, and one in four failed to do so in math (though, in both subjects, virtually none showed significant negative results). Even for KIPP, some “failure” is part of the game. Running schools is difficult.

Agreed. Especially that last sentence. I watch my colleagues work tirelessly to run our schools.

I'm pretty sure the folks at KIPP would agree, too. Now to my 2 quibbles.

1. Money

You write:

KIPP is very expensive, but they may get a worthwhile return on that additional investment. The existing evidence, though still a bit scarce (due mostly to data availability), suggests that the average KIPP school spends substantially more money than comparable regular public schools.

I don't think that's true. I won't dig in extensively here. The 2012 report you cite contends "average spending per pupil was around $12,000 to $14,000 citywide" in the traditional NYC schools. Then it says NYC charters spend more.

The $12k to $14k figure is pretty far from what others say the traditional NYC schools spend.

a. US Census Bureau says "New York City spent $19,000 per student."

b. The Citizens Budget Commission says New York City spends $20,276.

c. And here's back and forth between the IBO and a charter advocacy group.

There are fair questions here. We could dig in on them, of exactly what should count as spending per pupil both for districts, and what should count for KIPP. Each type of school has some sort of central office expenses.

Moreover, KIPP does attract an unusual type of teacher. There is almost surely a limited supply of those folks. And that could go to some reasonable call for "more money," to expand that labor supply in various ways. When Steve Brill considered this point in his excellent book Class Warfare, he threw up his hands. Essentially, the high-performing charter used equivalent cash to buy more labor, because it was attracting some workaholic teachers who were discounting themselves. So the kids did, in fact, get "more inputs." Bottom line, I agree with you on the idea of examining "true cost."

2. You write:

KIPP’s approach only works for some students, but these are among the students who need the most help to catch up. Through 8-9 hour days and summer school, KIPP adds the regular public school equivalent of about 3-4 months to the school calendar. In addition, exceedingly rigid disciplinary policies and parental contracts are used to enforce high-bar academic and behavioral standards.

I respectfully differ that "exceedingly rigid" describes discipline that I've seen in various KIPP schools. It's totally a fair question.

I'm trying to imagine the falsifiable experiment or study that would allow us to empirically analyze discipline in the classroom. Do you have ideas?

And I say "examine the classroom" -- not the code of conduct as written (because often there is distance between what is written and what actually happens), but the lived experience of the kids (which is what I assume you care about).

Here's my idea.

a. Get an agreed-upon "neutral" scholar.

b. Find a group of schools -- some traditional but with a dress code, some Catholic, some KIPP, some low-performing charters.

c. Shoot videotape of actual classes at random. Make sure there are no identifying characteristics (like a cross, a KIPP poster, etc).

d. Have folks interested in the KIPP and charter debate, pro and con, watch video with school identifications "blind" -- no idea what they're looking at -- and rate what they see.

Is it a productive classroom? Is it joyful? Is it "exceedingly rigid?"

And so forth. Then try to predict which classrooms are in which type of schools.

Do you think this would be a useful experiment?

Here is a second idea. This goes to concern about whether kids are "forced out."

a. Get an agreed-upon "neutral" scholar.

b. Find a group of schools -- some traditional, some KIPP, some other charter schools, like the UFT-run school in New York City.

c. Create a list of all of their departed students.

d. Interview departed students at random. Or their parents. Possibly this could be done in the presence of a former teacher or principal, who might also comment. Provide incentives to be able to conduct these interviews. Roland Fryer, I believe, was able to interview students who lost the admission lottery to Harlem Childrens Zone schools this way.

e. We'd ask: Why did you leave School X? Did the teachers and principal try to entice you to stay? What do you think of your new school, School Y? That sort of thing.

I think this survey work would shed light on the concern that charter students are commonly "forced out." I also think KIPP would fare very well in this type of study.

The interviewers could and should include some real charter skeptics. For example, Paul Hoss wrote a comment on your blog along these lines. I've read many of his other comments on edu-blogs. I think he is a smart educator. He remarks that he's read the "forced-out" meme about KIPP. That suggests he hasn't had a chance to decide for himself one way or other by talking to kids who've left KIPP and other schools. I'd nominate him as the perfect sort of guy to include in a study like this. I'd predict he'd end up converted, and conclude that the most typical KIPP story is teachers and a principal who desperately try to get a kid to STAY, but who chooses to leave in search of lax academics in a traditional school.

*

Why do I belabor how you and others characterize discipline?

We both want to learn why some charters, like KIPP, tend to out-perform other charters.

And school culture, I believe, is at the heart of why some charters do well and others don't. It's certainly not the whole story but it's an essential component. We need precision here or we won't learn any lessons.

I think KIPP and similar no excuses schools tend (imperfectly) to create a fairly positive, warm, and disciplined culture. I admire them. I think other charter schools often have exactly the same discipline policies as KIPP and don't achieve that culture to the degree that KIPP, and by extension, don't achieve the academic gains.

If we're going to understand the no excuses charter school, and here is a scholar who has done just that, I'd encourage you to really dig in on this issue.

*

One more thing. A friend -- a union leader -- says he gets hit by "both sides" whenever he makes what he sees as a reasonable, centrist, evidence-driven statement....ed reformers attack him as half empty, some of his members call him weak and appeasing. It's safer to stay in a single tribe. You're to be commended for trying to call em the way you see em.