"Building Evidence Into Education"

Dan Willingham writes:

Ben Goldacre is a British physician and academic, and is the author of Bad Science, an expose of bad medical practice that is based on wrong-headed science. For the last decade he has written a terrific column by the same name for the Guardian.

Goldacre has recently turned his critical scientific eye to educational practices in Britain. He was asked by the British Department for Education to comment on the use of scientific data in education and on the current state of affairs in Britain. You can download the report here.

So I clicked.

It's good!

Goldacre opens with this:

I think there is a huge prize waiting to be claimed by teachers. By collecting better evidence about what works best, and establishing a culture where this evidence is used as a matter of routine, we can improve outcomes for children, and increase professional independence.

This is not an unusual idea. Medicine has leapt forward with evidence based practice, because it’s only by conducting “randomised trials” - fair tests, comparing one treatment against another - that we’ve been able to find out what works best. Outcomes for patients have improved as a result, through thousands of tiny steps forward.

I pause here to emphasize that point, which I think is missing in education research. Study is on the "bigger policy questions," with very little devoted to individual tiny steps -- i.e., teacher choices. But those tiny steps could add up, a la medicine.

Goldacre:

But these gains haven’t been won simply by doing a few individual trials, on a few single topics, in a few hospitals here and there. A change of culture was also required, with more education about evidence for medics, and whole new systems to run trials as a matter of routine, to identify questions that matter to practitioners, to gather evidence on what works best, and then, crucially, to get it read, understood, and put into practice.

I want to persuade you that this revolution could - and should - happen in education. There are many differences between medicine and teaching, but they also have a lot in common. Both involve craft and personal expertise, learnt over years of experience. Both work best when we learn from the experiences of others, and what worked best for them. Every child is different, of course, and every patient is different too; but we are all similar enough that research can help find out which interventions will work best overall, and which strategies should be tried first, second or third, to help everyone achieve the best outcome.

Before we get that far, though, there is a caveat: I’m a doctor. I know that outsiders often try to tell teachers what they should do, and I’m aware this often ends badly. Because of that, there are two things we should be clear on.

I've written before about the need for a particular type of randomized trials in K-12, and a fair amount about RCTs in general, but I'd never come across this story. Goldacre writes:

Archie Cochrane was one of the pioneers of evidence based medicine, and in his autobiography, he describes many battles he had with senior doctors, in glorious detail. In 1971, Cochrane was concerned that Coronary Care Units in hospitals might be no better than home care, which was the standard care for a heart attack at the time (we should remember that this was the early days of managing heart attacks, and the results from this study wouldn’t be applicable today). In fact, he was worried that hospital care might involve a lot of risky procedures that could even, conceivably, make outcomes worse for patients overall.

Because of this, Cochrane tried to set up a randomised trial comparing home care against hospital care, against great resistance from the cardiologists. In fact, the doctors running the new specialist units were so vicious about the very notion of running a trial that when one was finally set up, and the first results were collected, Cochrane decided to play a practical joke. These initial results showed that patients in Coronary Care Units did worse than patients sent home; but Cochrane switched the numbers around, to make it look like patients on CCUs did better.

He showed the cardiologists these results, which reinforced their belief that it was wrong of Cochrane to even dare to try running a randomised trial of whether their specialist units were helpful. The room erupted: “They were vociferous in their abuse: “Archie,” they said “we always thought you were unethical. You must stop this trial at once.” ... I let them have their say for some time, then apologized and gave them the true results, challenging them to say as vehemently, that coronary care units should be stopped immediately. There was dead silence and I felt rather sick because they were, after all, my medical colleagues.