Harlem Charters: Lower Attrition than District Schools

Charter attrition in the news again today, via GothamSchools:

WNYC obtained three years’ worth of discharge data from the city’s Department of Education. We found no exodus from the charters.

In fact, the charters had lower attrition rates, on average, than the regular public schools.

Attrition in charter elementary schools for 2010-11 was 10.8 percent citywide, compared to 14.3 percent in district schools. Middle school attrition was lower in both categories and more comparable, at around 10 percent.

My reaction -- that's not a huge difference. I would not trumpet this finding to say "charter public schools are better than traditional public schools in attrition." Moreover, Gotham has a good story which provides context and complexity.

However, I would trumpet the WNYC finding to say that the common attack on charters is off-target. Yes, charters do lose students, but generally at rates similar to nearby regular schools, and perhaps slightly lower.

KIPP's national report card shows 11% attrition across 100+ schools in 2011, a rate similar to the Harlem schools mentioned above.

Match middle and high school attrition last year was 12% and 11%; our new elementary school was 3%.

I tried to look up attrition for Boston charter and district schools. Alas, Massachusetts doesn't track this data the same way as New York. Nothing for the full 12 month cycle.

But our DOE does have some data. Massachusetts tracks an unusual number called Churn. It's not a perfect number for our purposes. "Churn" is any kid who either arrives or leaves during the school year. Doesn't include summer months.

So I pulled the numbers into a spreadsheet. My usual caveats apply. Anytime I'm playing with a data set, there's a 35% chance I've made a boneheaded mistake. Buyer beware.

1. Churn

The Boston charter elementary schools have low "Churn" compared to district schools.

Same was true of Boston charter middle and high schools. The median charter middle or high, which happened to be our school, was 7% churn. The median district school churn was 31%.

Now is this a feather in the cap for charters? No. Not in my opinion.

Churn is simply what is easily available on our state's website to "sort." Churn isn't, in my opinion, a good statistic for this debate.

Churn rate includes intake. So it's useful to help you understand the challenges that a typical public school teacher is up against: the higher the churn, the tougher it is on the teacher. Teachers in many district schools face a daunting challenge that charter teachers face less frequently -- welcoming new kids in mid-year, and getting them acclimated.

So churn doesn't help much. And to my knowledge, MA doesn't track the sort of "full year" departure rate that WNYC was able to assemble in its report. Now what?

I took a shot at coming up with a new metric. DOE reports Churn. They also report "Intake." That is: the number of new students.

So I've created a new statistic called "During The Year Departure Rate."

During The Year Departure Rate = Churn Rate - Intake Rate.

2. Departure Rate

Here's what I found from the MA state data.

Among Boston middle and high schools:

1a. The median Boston charter had a 3.8% departure rate during the school year.

1b. The median Boston district school had a 5.4% departure rate during the school year.

For K-5 and K-8 schools:

2a. The median Boston charter had a 3% departure rate during the school year.

2b. The median Boston district school had a 4% departure rate during the school year.

In other words, it seems like Boston and Harlem stories are similar. The charters do lose students during the year -- but at slightly lower rates than district schools.

If anyone is interested and wants to check my math, happy to send you the spreadsheets, just email me.

* * *

So if you're scoring at home, and zooming out to big picture:

1. Nationally, does it seem like charter schools, on average, do better than district schools?

Not really.

2. Does it seem like charter schools in some places, like NYC and Boston, generate big learning gains for kids?

Yes.

3. Do Boston and NYC charters serve exactly the same students as district schools?

No.

We do serve disadvantaged, minority kids...so a similar population...but kids arrive to the somewhat charter better off, on average, than at district schools.

Roughly 0.2 standard deviations better in the Boston studies.

Charter advocates err when they imply or say "do better with exactly the same kids."

Charter critics err when they imply or say "totally different kids."

Furthermore, in Boston the "pilot schools" -- district-run unionized schools -- have exactly the same selection bias....kids arrive about 0.2 standard deviations above the district average. That is, it's a school choice thing seemingly, not a charter thing.

4. Do charters have higher attrition than district schools?

Hard to say. But....

In Harlem, per the WNYC report, no. Charters have lower attrition.

In Boston, charters have lower during-the-year attrition than our friends in district schools.

5. What about which kids leave?

Now we really don't know much. Lower-performing kids are probably more likely to leave in any school. But to my knowledge there are no data sets with apples-to-apples comparisons.

For example...

The typical 11th grader who leaves a Boston charter returns to a district school, but with higher MCAS than the other kids in that school, and does well.

The typical 11th grader who leaves a Boston traditional school drops out entirely.

6. But what if the reason that kids make really big gains at handful of charters do well is because the worse-off kids leave?

Ah, that's addressed by a new paper by M.I.T. economist Chris Walters.

He crunches the Boston charters numbers and finds the opposite of a common critique: he shows the biggest test score gains go to the students who were worst-off upon arrival.

Put another way: if Boston charters enroll more worse-off kids, their value-added (gain) scores will actually go higher.

The tricky thing? Scholars and teachers correctly care about gain scores -- it's what the school "controls." But newspapers care about absolute scores.