Pre-K, Charters: Peas in Evidentiary Pod?

1. Gina D My colleague Gina D and I exchanged some emails on pre-K. She'd read about a famous program, called Perry PreSchool Project (pdf). She wondered if I knew about it.

I did. This study gets cited over and over. Why? One reason is that the research design was "random assignment," which is the gold standard. Since there aren't a ton of these experiments, those that exists get a lot of play.

Result?

It found evidence of preschool program effects on children’s readiness for school and their subsequent educational success, economic success in early adulthood, and reduced number of criminal arrests throughout their lives.

Thing is, this was a boutique program. 123 kids. And it was done a long time ago. Launched in 1962! Older than me. That is saying something.

2. State of The Union

Pre-K has been a hot topic these past few weeks. President Obama:

Let's make it a national priority to give every child access to a high-quality early education.

If you're interested in this topic, you should read Sara Mead. She blogs at Edweek.

3. Pre-K evidence and charter evidence

Let's ponder the movement to expand access to pre-K, and the movement to expand charter schools.

Interesting, the worlds rarely overlap. In fact, often the folks who push hard for greater pre-K oppose charters. And while it's not exactly parallel, some of the folks who push hard for more charters are skeptical of pre-K.

I am not saying the evidence base here is identical. But I do think the topography is similar.

a. Some small samples, random assignment, with great outcomes.

b. Some medium sized implementations with good outcomes.

c. Overall, however, the average pre-K program, and the average charter, do not seem to be game-changers for kids. In Boston, yes, game-changer. In Massachusetts, yes, high-poverty charters are game-changers. In USA as a whole across the 6300 charters, I don't think so, although we can't say for sure.

* * *

For more on pre-K evidence, see here.

Tomorrow I'll examine 2 good analysts who have recently examined charter school evidence.

PREVIEW:

The short version is there are 10 gold-standard studies of charters that show good things and zero that show bad things, and those studies are associated with many more actual children than the gold-standard pre-K studies (I know of two, but there may be more). Also there are some randomized studies of pre-K that don't have good outcomes.

So while I do think the charter evidence is stronger than the pre-K evidence on the whole, the evidence at "large scale" for charters is not compelling, or at least not yet.

Now: with 10-0 score on the gold-standard studies, why wouldn't we conclude the average charter is indeed a game changer for the kiddos? We'll dig in manana.

* * * 4. Bonus Thought Experiment About Reasoned Discourse

What would happen if ...

a. You grabbed 20+ studies of charters and pre-K. b. You stripped out any mention of those words. c. You simply called them "Intervention 1, 2, 3, 4" etc. d. You changed the data sets a little so you couldn't tell which study was which, even if you knew the studies off top of your head. and.... e. You got gathered all the big name education advocates, with fiercely competing beliefs, to sit in a room and consider the evidence?

Here is what I think would happen.

Polarity and stridency would go way down.

If you're a big pre-K advocate and hate charters, if someone holds up a piece of evidence and says "X worked in a limited context," you're not going to want to offer a strong opinion.

If you say "See, X works?" then you might have just endorsed charters by accident.

If you say "X is a meaningless study" you might have just undermined your own case for pre-K. And vice-versa.

I've often wanted to do the same "blind discussions" where video is excerpted from real life classrooms from all types of schools, in such a way you can't say what type of school it is, you can only see a real teacher and real kids.

Again, I predict polarity and stridency would go way down.

If a reform advocate says "Wow, that room is amazing," well, it might be teaching from a school labeled "failing" and about to be shut down. If someone says "That class is militaristic and teaching to the test," and it turns out to be a class led by a prominent union leader in a traditional school, well that might not bode so well for some edu-pundits.

Mike Petrilli, I'm handing you the ball to organize this and get everyone to the table. You've got time. Just watch less Caillou.