Wednesday, 5 June 2013

Teacher Proof: Why Educational Research doesn't always mean what it claims, and what you can do.

So, I have a book out.

It's been a long time coming. Since I started teaching, I knew there was something suspicious about what I was being told worked in classrooms, and what actually happened. It started in teacher training, as well-meaning lecturers and reading lists advocated apparently cast-iron guarantees that this method of educating children, or that way of directing behaviour, would be efficient. It continued on DfE sponsored training programs where I was taught how to use NLP, Brain Gym, Learning Styles and soft persuasion techniques akin to hypnosis.

Then I began teaching, guided by mentors who assured me that other contemporary orthodoxies were the way to win hearts and minds. It took me years to realise that thing I could smell was a bunch of rats wearing lab coats. And why should any new teacher question what they are told? Establishment orthodoxies carried the authority of scripture. And often it was justified with a common phrase- ‘the research shows this.’

I remember reading Ben Goldacre’s Bad Science, and being amused and horrified by the cavalier ways in which science could be hijacked by  hustlers. His harrowing of Brain Gym led me to wonder what else, like Descartes, I needed to question. What I discovered led me to write Teacher Proof.

First of all I discovered that a lot of what was considered to be absolute dogma by many teachers, was built on quicksand.  Learning Styles, for example, were almost universally accepted by every teacher who trained me. It was a Damascan epiphany to find out that there was hardly a scrap of evidence to substantiate it, that the serious academic  community had washed its hands of it long ago. But it lingered on, a zombie theory, staggering from classroom to classroom, mauling lesson plans.

Once I had peeled one strip of paper from the wall, I could do nothing else but keep pulling, and see how much came off. Much, much more, it turned out. First of all, I entered the world of pseudo-education, where optimistic internet sites boasted of Olympian gains to made by the adoption of this pill (often Omega 3), that smell (sometimes Lavender, sometimes not) or even this sound (the Mozart Effect, for instance). These, at least, seemed to be obvious pigs in pokes. Other companies sold hats- literally, thinking hats- of various colours, or exercises that promised to boost brain power. But they asked customers to gamble a lot more than a stamp, as Charles Atlas innocently proposed.

Unfortunately, it was often just as bad when I progressed to the realms of alleged propriety; I found that a lot of what was practically contemporary catechism, was merely cant. Group work, three-part lessons, thinking skills, multiple intelligences, hierarchies of thinking like Bloom’s, all- at least to my poor eyes- appeared to rely on opinion and rhetoric as much as data. Delving deeper, I found that this was an affliction that affected the social sciences as badly as the natural sciences- perhaps worse, as natural sciences are at least readily amenable to verification. But any social science- from economics to sociology- is subject to inherent methodological restrictions that makes any claims to predictive or explanatory powers intrinsically difficult.

Which isn’t to say that social science isn’t’ a powerful and urgent device with which to accrue an understanding of the human condition. But merely to require that its claims be interpreted appropriately. It is a very different proposition to claim, for example, that water boils at 100 degrees Celsius at sea level, than it is to say that children learn best in groups. The first can be at least disputed immediately, or not, by testing. The latter requires a plethora of causal factors to be adjusted and  accounted for. And to confound matters further, humans are notoriously hard to fit on a microscope slide. Nor are we always the most reliable of subjects.

Sometimes this was the faulty of those writing the research; sometimes the research was, as Richard Feynman describes, Cargo Cult Science; sometimes the writers appeared to have no idea what the scientific method was, believing it to be some kind of fancy dress with which one clothed a piece of journalism; sometimes allegedly sober pieces of research were simply misinterpreted by a willing media; sometimes it was the teachers themselves that had misappropriated the findings; sometimes it was the policy makers who were hungry for a magic bullet and had already made their minds up about what they were buying.

Whatever the reasons, it was clear: the educational research we were asked to assimilate in schools was often more like magic beans than magic bullets. That’s unhealthy. There are armies of earnest, dedicated professionals working in educational research who are horrified by some of the fantastical or flimsy claims made by the hustlers and their PRs. If educators want to get past this unhealthy  system of intellectual bondage, we need to become more informed about what the research actually says, and what good research actually means; about how hard it is to say anything for certain in education, and when claims can be ignored, and when they should be listened to.

So I wrote Teacher Proof. It’s aimed primarily at people who work in schools, but it’s also for anyone involved in education, research and policy. I am, unashamedly, a teacher. I admit I have entered a world- of educational research- in which I am only a guest. I am aware that in my travels I may be more of a tourist than a native. But I have tried to write as honestly and as plainly as I can, about matters that affect me deeply- the education of children. If I have made any errors- and I’m sure that I have- I welcome correction, and discussion. I can’t shake the feeling that teachers would do well to make research more of their business, get involved, participate in studies, and perhaps even conduct some of their own, with guidance. I’d also like to think that researchers would be well advised to ensure their theories are tested objectively, with an eye to disproving them, in classrooms with meaningful sample sizes. There is a great deal of good that the two communities can do together.

Perhaps then teachers can look forward to hearing the latest research, and run towards it; and researchers can see classrooms not as awkward inconveniences between data sampling and publication. There’s an awful lot of good research out there, but it gets drowned out by the bad.

Good ideas, like decent whisky, need time to settle and mature. I suspect that we need to develop more of a critical faculty to sift the ideal from the merely idealistic. Maybe then we’ll be immune to novelty and fashion in pedagogy. Or, as I call it, Teacher Proof.
 
Buy Teacher Proof HERE

24 comments:

  1. Really looking forward to reading this! V much agree teachers should be making it more of their business. Have just done a quick poll of colleagues asking them to rank the effect of different teaching/school techniques and then shown them how that compares to Hattie's research/effect sizes. Some fascinating reactions - but main take-away was lack of confidence these bright, talented people had talking about academic research.

    ReplyDelete
    Replies
    1. Cheers, Piers. I sense a new taste for good research, and a willingness to be responsible for it, but it's going to take time to work it's way through the system.

      Delete
  2. Tom, here's one you might like to add to your collection. Becta’s "Harnessing Technology, Next Generation Learning, 2008–2014," a hugely influential piece of educational research, contained only one reference in the entire paper to evidence which suggested technology has had a positive educational effect. That reference was to Becta's own previous 2007 technology review and the research that referred to...was itself commissioned by Becta.

    As an ex teacher and researcher, I would advise teachers to ignore anything which hasn't undergone a formal academic process, including peer review, to reach publication, especially from pressure groups with a political axe to grind. The educational world is almost suffocated by this kind of pay-the-piper ideology masquerading as "research."

    ReplyDelete
    Replies
    1. 'As an ex teacher and researcher, I would advise teachers to ignore anything which hasn't undergone a formal academic process, including peer review, to reach publication, especially from pressure groups with a political axe to grind. The educational world is almost suffocated by this kind of pay-the-piper ideology masquerading as "research."'

      Could NOT have said it better myself.

      Delete
  3. Education abounds with sellers of snake oil. About the only thing we know with any degree of certainty is the relation between social class and attainment. Knowledge which few teachers find agreeable as it tends to reduce their role in the attainment of school knowledge.

    ReplyDelete
    Replies
    1. I was very impressed when I read this major research study from the States, which looked at books in the home across 27 countries over 20 years, as a predictor of educational performance.
      http://www.unr.edu/nevada-today/news/2010/books-in-the-home-as-important-as-parents-education-level

      Delete
  4. One of the big problems with this is that management & inspectors love it. It gives them something to check & ensure the teachers are doing - Plenty of room in all this for Continuing Professional Development.
    Managers can show the inspectors they are 'improving the teaching 'in their schools/colleges/universities/etc. by setting all this stuff up.
    If you don't fully buy into all this stuff, forget promotion, too. Can't have teachers that rock the boat...

    ReplyDelete
  5. Tom: do you know if there are plans to publish the book for the Kindle or other e-book format?

    ReplyDelete
    Replies
    1. Yes, should be out in groovy electric form in a week or two. Could be earlier, or so my publisher says......

      Delete
    2. Many thanks! Not that I don't prefer Real Books, but with about 8000 books currently at home and shelves in my classroom bending under the weight of the things, my Kindle is a life-saver.

      Delete
    3. Good: there's a lot of this kind of problematic stuff floating around the EFL world; kindle makes it easier to get hold of internationally.

      Delete
  6. Good luck with the book Tom. I'll be interested to see if you reference Hattie in your book, and if any reference is positive or negative. I don't know of a greater body of work regarding educational research than "Visible Learning".

    This period of enlightenment in education is starting to really take hold in the UK. Of course it started back in the 1970s with work from Creemers and Hattie. As with many improvements in education, it takes about 40 years to come to fruition!

    ReplyDelete
  7. It's a tricky one this - I wonder if one of the core problems is the lack of communication between universities (research departments) and schools. I suppose I should put cards on the table - I am an academic and a researcher and I have been saying the things you say in your post above. I'm pretty sure most of my academic colleagues have been too we know about the snake oil or NLP, VAK, Mozart effect etc.. and we are well aware of the limitations of social science research and the complexities in employing the physical science methods and RCTs (randomised control trials) in the school setting [Ben Goldacre also admits this]. Media often wants simple answers and books with titles "teaching the buggers to behave" get rather bigger circulations that papers on "impact effect on the relationships between pupil and teacher in classroom discipline".

    So, I come back to this first premise. Perhaps those of us in the research field need to learn about good ways to communicate about our research - including ways to make the methodological limitations clear and understandable but the media also needs to play its part in not wanting to over simplify.

    And then there are the politicians ... what can you do with them they struggle to read or listen to research unless it fits and ideology and do not want to think about longtitudinal studies when "quick fixes" can be applied. This has been exacerbated by the current political incumbents as they have rubbish academia in both the persons (professors of education) and the idea (by supporting school based training over research led training) and in the non-support of teachers doing research level study (compared to, for example, physicians).

    So from my perspective you are saying nothing new here Tom, but if you can get some of these messages out this would be great and I shall plug along with the courses which include neuromyths (you mention a couple of these), research methods, and yes the scientific method and its place in qualitative and quantitative research.

    ReplyDelete
  8. Just re-read my entry above - apologies for the typos. I was trying it on the train, obviously not conductive to hitting all the correct keys!

    ReplyDelete
  9. All of that's right. I was already too long in the job to be distracted by learning styles, faulty brain science etc. But one thing I've learned (starting with my MEd in the 70s, whcn an MEd degree was rather more academic than they now are) is to try to look beyond what research evidence says and question, in detail, how it was obtained. (The same question needs to be asked of attainment data in schools but that's a different story) I really doubt, for example, whether it's actually possible to run a proper medical-style RCT in a school, or across schools, for all the reasons that are surely pretty clear to anyone who works in one. And it you try to make it clinical by taking it out of school, or too heavily controlling the context, then you also risk invalidating it. Comparisons with medicine are simply not on. I tried to raise this on twitter a month or so ago, but twitter is the wrong arena, and I couldn't get the point across. If there really are still teachers who believe myths, then all power to Tom's elbow. I just want to say, as I used to say (still do) to children and young people -- 'But exactly how do we know that?'

    ReplyDelete
  10. Great read - hilarious at times! Loved the section on Ken Robinson who I've blogged about on educationandstuff.wordpress.com

    Greg

    ReplyDelete
  11. Thanks for this. In Ontario we're being Damien Cooper'ed to death right down, and he doesn't even try to offer studies to support his claims, just a whole lot of anecdotes and weak analogies. I've complained about him at length here, and here, and about Ken Robinson's schtick and on Marc Prensky's desire to have kids play video games in class. I hope your book gets the following it deserves because these people are seriously wasting our time!

    ReplyDelete
  12. Any chance of getting an e-book version?

    ReplyDelete
  13. Tom Bennett, hi!

    Are you by any chance related to Barrie Bennett, author of "Beyond Monet"? "Beyond Monet" has been the bible of "cooperative learning" for many educrats and eduserfs!

    What an irony it would be ...

    ReplyDelete
  14. Fab book. And better still, seriously funny. I am currently writing a text that leaves out all the pseudoscientific twaddle on concentrates on the solid stuff. Only thing if argue with is that your description of Vygotsky is more like the bowdlerised piffle that people attribute than what young Lev actually said. He didn't hold with recycling ignorance but said that people learn from working with more competent others. This has been reduced to 'let the kiddies chat'.

    ReplyDelete
  15. Oh and Howard Gardner doesn't qualify as a member of the intelligence researching community. He is, as we'd say in Australia a bower bird, someone who collects shiny things. He knows very little about intelligence theory or he'd know that from th earliest days theorists were divided about whether there is a g or whether intelligence is actually multifaceted. His idea isn't even original on that score. He writes about anything that takes his fancy and isn't inhibited by not knowing a great deal about the latest shiny thing.

    ReplyDelete
  16. There is very little time and scope for teachers to do real research in the UK classrooms. Mainly because the original aim of teaching is not about the pupils learning. But really to be day keeper of prospective voters borned to present voters. Teachers should be researchers clearly, but needs time to carry out these research instead of writing reports and meeting heads targets driven by league tables or heads ego. However, I do believe there are and that learners have different learning styles.

    ReplyDelete
  17. Oh fantastic. Can't wait to read this (on Kindle hopefully). On my PGCE course, we had a session on teaching that is backed up with research. Imagine my shock when I realised that the entire rest of the course had no academic support whatsoever. I switched off at that point and merely went through the motions to pass the course. Such a shame.
    However, what I would say is that if it works for your group of learners, then shouldn't you just use whatever technique you like? You can tell what works in your classroom for your students, no matter what any book or research tells you. For example, orthodoxy tells us that children like working in groups. Some of mine hate it and it produces terrible results. I learnt this by sad experience

    ReplyDelete
  18. Tom - In another article you wrote yous said "I now have so called educational neuroscience right at the top of my hit list. It makes the attactive claim that understanding the brain will do everything from boosting grades and curing ADHD to raising IQ and reversing ageing." I am a PhD student in cognitive science doing some related research, I may be misunderstanding what you said, but I'd love to know more about your sources and what you mean when you say "educational neuroscience". Please feel free to email me! - Ashley Jo Thomas UCI (ashleyjt@uci.edu)

    ReplyDelete