Thinking in the right direction; just don't put all your faith in RCTs. Ben Goldacre's vision for evidence based learning.

But is it outstanding?

Bethnal Green Academy, a soy latte's throw from both hipster Columbia Road and the surrounding estates, was the venue for the launch of Ben Goldacres's new advocacy project: Building Evidence into Education. Nice looking school; it's got BSF written all over every plane and pane. The livery outside the school shouted every second sentence of the latest Oftsed report. Most pleasingly it was styled in the Star Wars font (or as my kids call it, 'That old film'). 

Hosted by Teach First, the Royal Marines of the teacher profession, we were first treated to an introduction by Dame Gove himself in what was, I thought, a remarkably short set. It was like booking Geoff Capes and asking him to open a jam jar. Goldacre followed;  a passionate and determined thinker and speaker, whose Bad Science series shook me, like Hume did for Kant, from my dogmatic slumber.

It's a familiar saw for him: the need for Randomised Controlled Trials (RCTs) to become more prevalent in the social sphere (here represented by education), as they are in the medical profession. He made an interesting comparison: medical practitioners at first resisted RCTs because they were convinced of their own intuition, until it turned out that intuition often led us down the wrong path; eg using steroids for brain inflammation did more harm than help- the reverse of orthodoxy. Now, so are teachers (resisting RCTs, not harming the brain. Oh wait.)

Goldacre's point is refreshing and disarming: we run trials like this for every pill and potion; why not policies, which in the galactic scale of things, dwarf even pharma for dollar tonnage. And as usual, there is a lot of practical wisdom in what he says. I am delighted that the DfE is looking into ways of conducting research that is robust and real, rather than fanciful and driven by dogma.

So why not have RCTs in the social sciences? One possible objection, the ethical, can be easily dismissed: issues of consent, and issues of knowingly withholding a potentially useful intervention from the control group. But we already introduce dozens of interventions in schools every single day. It isn't as if we aren't already drowning our kids in optimistic hoodoo. Why NOT randomise it once in a while? When was the last time we asked a kid's consent before we popped a Thinking Hat on them, or told them to rub their brain buttons? 

I think there are, however, some serious problems with the use of RCTs in education.

 RCTs aren't the answer to the question 'What really works in schools?' because differences in context can never satisfactorily be ironed out. The causal density of humanity is too high; there are too many factors to establish a reliable protocol that could hope to encompass the variables of the human mind. There are as many factors to juggle as there are grains of sand on the beach. Lazy research is sodden in bias and assumptions, wooly over interpretation and optimism. That said, there is room for quantitative research , some of which will be amenable to RCTs. I've heard social scientists say that the problem is merely one of design; I say the problem is the methodology itself. If you're making quantifiable predictions about the physical world you need to provide unambiguous methods of establishing initial conditions as well as outcomes, possible causal mechanisms, and demonstrate reproducability. That isn't easy.  

Also, the problems of cognitive bias, the Hawthorne Effect, the John Henry Effect, the Pygmalion Effect and so on are enormous obstacles to appreciating when an intervention is effective and when it is not.

This is definitely thinking in the right direction. Goldacres's attempt to lasso the spoon bending of social science and pin it down to some kind of methodological rigour, is exactly what I, and many teachers, want to hear, and I absolutely support his attempts to reinvigorate the debate. But there are many obstacles to this, even before considering motivation.

One is time. I work pretty hard. I don't expect a biscuit, because I bet you do too. I'm also pretty good with my time. And I barely have time to do, well...this kind of thing, and the only reason I do is because I cash in hours of sleep like chips in a casino. Doing research into education is going to be a niche pursuit until workloads go down. Like archaeology in the 19th century, it will remain the hobby of gentlemen.

And let's be honest, most teachers aren't scientists. They're as prone to their pet prejudices and tender perversities as most people. Read the Tooley report if you want to see how badly education research can be conducted even by people who should know better. Now multiply that by 100 and you get, I would say, most of us. Many teachers would love to get involved with what Goldacre sensibly described as driving research.But would research benefit? Not until teachers learn what the scientific method is.

Dating for Nerds

So, what next? From the presentations, and the discussion one thing became apparent: this isn't the launch of anything concrete yet. At this point, the program is advocacy. There was no funding committed, no projects starting tomorrow. From one perspective, it's hard to see many education secretaries getting too worked up about RCTs- the average tenure of the Headmaster-in-chief is about 18 months. RCTs can take years; to be efficient, to be meaningful, they have to be as large and long as possible (as the inspector said to the janitor). Why invest in something that a) won't bear fruit until after you've been moved to The Department of Silly Walks and b) Might disagree with your own pet projects? It's always safer to simply pick research that appears to validate your own objectives.

And yet, and yet. Gove was, at least, there. The Dfe's involvement with Goldacre shows, at least, a symbolic commitment to better research in education. And let's not forget the £125 million DfE funding for the Education Endowment Fund (represented here by Dr Kevan Collins) a Sutton Trust start-up devoted to research, some of which involves RCTs. There might be some will there, but it isn't very full-throated. 

So much in education is an abstract, an artefact of art as much as engineering. We can barely agree as to what educated means. Or learning. Or thinking. Or engagement. Or creativity. These aren't amenable to metrification.How do we study what we cannot catch?

But to sound a more positive note: anything which seeks to firm up the wiliest of educational research is an asset.  Goldacres's profile should help turn a search light upon the relationship between research community and Chalkface warriors. He's absolutely right when he says that there needs to be more communication between the practitioner and the research communities. Teachers make bad researchers, and often, researchers make bad teachers, and the assumptions are apparent in many articles of research I read- where assumptions and biases that would have you laughed out of the physics club are common.

One thing's clear: it's a mess. I'm not so sure, as Mark Keary and Ben Goldacre implied, that we're on the cusp of a Golden Age of research bounty in education. They've been saying that about the social sciences since they were invented, and microscopes and abacuses, which had been so generous in the natural sciences, were turned to the human sphere. We're still waiting for the Industrial Revolution in social science.

One thing teachers need to do, exactly as Goldacre said, is to familiarise themselves with the the principles of science; to arm themselves with at least a basic understanding of what it means to say an intervention is true, or probably true, or probably not. God knows we need to, given the deluge of garbage that we've endured, justified by silly hat research.

So I wrote a book about it. It's called Teacher Proof, and it's out this June. 

Read Dr Becky Allen's take on the same event 

David Didau also has this to say