Ask, Ye Receive: More Khan

Good morning. Friday I asked - does anyone know of any randomized evaluations of Khan Academy?

So far, the answer seems to be: no. However, I did learn a lot...

1. Via Sharon at Boston Prep (thanks!), this study will be out in December 2012

SRI International's Center for Technology in Learning (CTL), with funding from the Bill & Melinda Gates Foundation, is studying the adoption of Khan Academy in a wide variety of primary and secondary schools and classrooms in Northern California during the 2011–12 school year. Twenty-one public, charter, and independent schools are participating in the study. CTL's research encompasses

*Case studies to understand how Khan Academy is being used to support math instruction in different school settings *Quasi-experimental designs to measure the potential impacts on student learning *An analysis of the costs of adoption

2. Okay, but what do we know now? Cormac reminded me that our friend Brian Greenberg of Envision (former Match principal intern) had written a non-experimental study of using Khan in their charter school summer academy.

It's excellent. If you have any interest in this topic, I encourage you to read the whole paper. I'd share the best nuggets, but I think the file is somehow encrypted to make pasting difficult, or I'm just dumb. Since I can't think of a good reason why they'd do the former, let's bet on the latter.

3. It's not a study, but a friend at a West Coast CMO described a widespread implementation of technology, which included kids working with Khan. Her summary:

...The kids really like using their laptops (though keeping them on track is sometimes a challenge).

That's exactly what I figure a randomized experiment would account for -- the "net" of the advantages vs. the disadvantages.

4. This is a Slate article about Khan. I link to it because the comments are valuable. If you skip past the whining and technophilia, you get some nuance. Like this from teacher "Dag":

Is the idea of the reverse lecture good. Absolutely and with my AP Calculus students I take advantage of it time from time (I make my own video's for my students and the students have said they appreciate them).

But realize, these are my AP students we are talking about, students who would thrive in pretty much any environment. These students are the ones who will actively seek help when they need to whether its through internet resources to coming in at lunch or after school to get additional help from me.

Unfortunately, the positive work ethic displayed in my Calc class is not prevalent in the lower classes.

I also teach Algebra 2 students, a class with a mixture of motivated to not motivated students. They are good students but if I tell them to go watch a certain video lecture so that tomorrow we can spend the day working on problems about 1/3 to 1/2 might do so. It's not that they are stupid or lazy they just think that they don't need to learn on their own, that they will get the concepts without help. The problem is of course, that what ends up happening is that the next day they will either call me over or try having a friend help them (teach them all the concepts that they should have been exposed to in the video), or simply don't understand the concept.

And these are my Algebra 2 students. In an Algebra 1 class you probably would get next to no students who would bother to watch the video lecturers.

Please don't get me wrong, I would love for the reverse lecture model to work. I am at my best as a teacher, when I am going around helping students individually. I can pinpoint where their issues are and fix them. If I knew for certain that my students were getting a decent introduction into the concepts outside of the classroom I would not hesitate to adopt this style. But it is simply not the case.

...The AP Calculus students that I taught at this previous school were so vastly different than those I currently have. Their precalculus teacher was a sub, they were never taught trigonometry which is vital for Calculus. For many english was not their first language and while they had obviously found a successful way of learning the concepts that does not change the fact that they were at a disadvantage to my current students.

Would having access to a source like Khan's academy help students like these, probably (though many of the student did not have access to a computer) but it would not be a "fix" for the situation they are in by any stretch of the imagination.

Another teacher, Kate, writes:

My students like using Khan and they like going at their own pace. I don't use Khan exclusively, but it is useful for allowing my top students to move forward . It allows me to free up more of my time to spend on the students who really need the help, which is most of them.

I have found since using Khan (last year and this year) that my students check out of algebra because they don't have a strong foundations in arithmetic. I had my students start at 1+1 exercises this year and move forward. This has made a world of difference and we are further ahead, as a class, than ever before. I have found this is the motivating factor. It seems once my students are able to master the basics, it makes algebra easier and they will actually do the work.

5. And this one, from Slashdot....a range of comments about Khan. This one by "Bruce" was interesting:

Everyone hates to have their business made into a commodity, that's simple economics. Once it happens, you have to compete on cost alone and be hyper-efficient to make a buck. You can only stay above that if you have a clear and provable advantage over the commodity version, and such things are difficult to maintain as the quality of the commodity version improves.

Look at this like Wikipedia. There are obvious quality problems, but Wikipedia keeps improving and getting larger, and if you're Microsoft Encarta, there's just no market for you any longer (thus, the first MS product actually killed by Open Source).

The guild apprenticeship system really hated book-learning. Copyists really hated printing. Both of these were previous means to commoditize education. This is just more of the same.

There will be tremendous economic repercussions from the further commoditization of education.