IES gets it 3/4 right

Via Sarah Sparks at Edweek, I saw that IES -- the U.S. Government's education research division -- has a new idea.

The new research program, Partnerships and Collaborations Focused on Problems of Practice or Policy, will incorporate two extant research programs, Evaluation of State and Local Programs and Policies (84.305E) and Researcher-Practitioner Partnerships in Education Research (84.305H) and it will add a third new topic, tentatively called Continuous Improvement Research in Education.

I know, I know. Let me translate. IES wants to give grants of $1.5 million for a new type of research.

Their idea is precisely 3/4 "good."

1. Good idea: They have 3 topics they want investigated, and one of them is the single biggest issue facing high-poverty schools.

Creating a safe, orderly and supportive learning climate for students from preschool through high school. In this case, applicants might develop or adapt a range of practices to be interwoven into general classroom instruction and student-teacher interactions. These practices might also be supported by school-wide policies aimed at establishing uniformly high expectations for student achievement and behavior for all students and engaging parents and community members in positive and helpful ways.


2. Good idea: Tinkering, failing, trying again, tinkering, succeeding, more tinkering.

They're building on what a type of research our friend Tony Bryk, among others, has brought to K-12 from other fields. Sparks writes:

Anthony S. Bryk, an NBES member and the president of the Carnegie Foundation for the Advancement of Teaching, likened continuous-improvement research to the Toyota auto company's famed "total quality management" system, in which any worker can stop the factory line if he spots a problem.

"What is the problem we're trying to solve, what and why is the change we're putting in place, and how will we know if that change is an improvement?" Mr. Bryk said, describing the approach continuous-improvement research takes.

"That's a very different way of thinking about the work," he said. "It's seeing improvement as a learning journey, rather than the way most districts see it as, 'Well, we have this new idea, I have to roll it out fast, implement it at scale, and most of my attention is on crisis intervention.' "

"Learning Journey." "Empirical tinkering." I like that.

3. Good idea: Getting practitioners in on this.

Successful applicants will demonstrate evidence of a deliberate collaboration between researchers and practitioners. Projects under typical IES Development grants can, but are not required to, incorporate these three aspects in their work.

Hey if they can legitimately pull it off, great.

(Much more typical is "fake" collaboration where the researchers know what they want to do, and just want a "practitioner sign off." Nothing wrong with that, but it's just not real collaboration nor genuinely coming from the trenches.)

4. Really bad idea -- "Projects funded under this topic must target systems."

A system is a set of multiple actors and decision makers—teachers, administrators, parents and central office staff—who work together for a common goal. Systems also include the array of policies, practices and tools designed to help teachers and school leaders affect a particular outcome or set of outcomes.

I make my case against this focus in an Education Next essay I wrote last year. IES insists on ignoring the individual teacher. I'd argue that the 3 million individual teachers should be the key subject of research, but I'd settle for IES even allowing the individual teacher to be a secondary or tertiary subject of research.

But no. Instead, IES is funding more studies over the whole system.

My argument:

In July 2011, Bill Gates told the Wall Street Journal, “I believe in innovation and that the way you get innovation is you fund research and you learn the basic facts…. I’m enough of a scientist to want to say, ‘What is it about a great teacher?’”

As a “practitioner” of sorts, I’ve wondered the same thing for 15 years. The K–12 school sector generates little empirical research of any sort. And of this small amount, most is targeted to policymakers and superintendents, and concerns such matters as the effects of class size reduction, charter school attendance, or a merit-pay program for teachers. Why is there virtually no empirical education research meant to be consumed by the nation’s 3 million teachers, answering their questions?

Those 3 million teachers generate about 2 billion hour-long classes per year. We do not know empirically which “teacher moves,” actions that are decided by individual teachers in their classrooms, are most effective at getting students to learn.

Mr. Gates has part of the answer. Money. For 2011, the Microsoft R&D budget is $9.6 billion, out of total revenue in the $60 billion range. The U.S. Department of Education’s Institute of Education Sciences (IES) represents only a fraction of total education research, but its budget gives some perspective: IES spends about $200 million on research compared to more than $600 billion of total K–12 spending. So, 15 percent to upgrade Microsoft, 0.03 percent to upgrade our nation’s schools. And while Microsoft’s research is targeted to the bottom line ($8.6 billion is on cloud computing, the profit center of the future), IES spends almost nothing examining the most important aspect of schools: the decisions and actions that individual teachers control or make.

One IES project is the What Works Clearinghouse (WWC), established in 2002 to provide “a central and trusted source of scientific evidence for what works in education.” The WWC web site lists topic areas like beginning reading, adolescent literacy, high school math, and the like. For each topic, WWC researchers summarize and evaluate the rigor of published studies of products and interventions. One might find on the WWC site evidence on the relative effectiveness of middle-school math curricula or of strategies to encourage girls in science, for example.

But there is almost nothing examining the thousands of moves teachers must decide on and execute every school day. Should I ask for raised hands, or cold-call? Should I give a warning or a detention? Do I require this student to attend my afterschool help session, or make it optional? Should I spend 10 minutes grading each five-paragraph essay, 20 minutes, or just not pay attention to time and work on each until it “feels” done?

After I wrote that, an eminent scholarly friend corrected me on one point:

I agree that more research needs to be done on what you’ve called teacher moves and I think you’ve made some good suggestions for the types of research questions that might interest teachers.

However I regret that you substantially misrepresented the efforts by the Institute of Education Sciences to fund such research by focusing on the What Works Clearinghouse, which has the assigned function of reviewing branded programs that can be adopted/purchased by schools and school districts (school administrators are the audience, not teachers), and ignoring research that is funded through the IES National Center on Education Research. A cursory search of NCER funded grants in topics such as reading & writing, mathematics and science, and special education discloses many that examine the impact of changes in teacher behavior, e.g., recent grants on reading and writing.

I looked through the examples he listed. I didn't find many that I'd describe as studying "teacher moves." We must have a semantic difference.

Many of the grants listed above examine curricula, like XYZ type of vocabulary flash cards.

My reaction is

a) often an individual teacher doesn't decide curriculum, and

b) in any case, curricular decisions aren't representative of the hundreds of decisions a teacher must make each day. They're typically one-time decisions made in summer.

You can follow the link and decide for yourself.

Instead, here is my analogy for examining the IES choice.

Let's say you cared about music performance. You wanted more great live music heard. You had money to spend on research. You studied symphonies, marching bands, music conservatories. Fine.

But < 1% of your research examined individual musicians, choices they make, lessons they take, and how they develop...with the idea of the musicians themselves as the key to your larger goal.

IES's new "tinker" approach to research could easily be applied to individual teachers as the key unit of change. Would love to see their approach become 4/4 correct.

I hope they'll consider at least some of the $1.5 million grants for people specifically to study "individual teacher turnarounds" -- coaching (or other PD) interventions that document a journey from "failing teacher" to "pretty good," or from "pretty good" to "great."

Hmm. I'll try to paste this blog into the comment section at IES's website. If you have ideas, IES would love to hear 'em.

P.S. Just an update to say I got a courteous response from the director of IES.

I think that the millions of “teacher moves” that you describe need to be thought of as a system. Teachers work together and their moves are either helped or hindered by school and district programs and policies. That’s what we’re trying to get at.

And that is where I respectfully disagree, per my Ed Next essay.

Again, my view is: both are valuable -- studying systems (i.e., groups of teachers), and studying individual teacher moves. But there is value in the latter, when almost all federal research dollars are for the former.

Medical researchers study whether eating broccoli cures cancer. Period.

Yes, the body is complex; yes cancer is complex; and yes the health care system is complex. All should be studied.

But individual patients make lots of decisions. Like what to eat. And if it turns out that eating broccoli helps -- irrespective of every other complex system -- it's useful to know!