iPads: game changers or money paperweights? New study tells us little

'Computer: activate holo-deck, In the Night Garden sub-routine.'

An interesting and problematic study from Northern Ireland about iPads in early-years settings hit the interweb today. Interesting because it makes some extraordinary claims about their efficacy that, if true and replicable, could revolutionise the way we teach in those settings; and problematic because  that ‘if’ has a lot of heavy lifting to do.

The study ‘Mobile Devices in Early Learning’ was carried out for two years and involved 650 pupils in five Belfast primary schools and five nursery schools.

‘Schools which took part were in some of the most deprived areas of the city.
They were each supplied with sets of iPads for nursery, primary one, primary two and primary three classes.’ (1)

What did they find? Fans of chalk boards and cuneiform look away now:

• The introduction of digital technology has had a positive impact on the development of children's literacy and numeracy skills

• Contrary to initial expectations, principals and teachers report that the use of iPads in the classroom has enhanced children's communication skills

• Children view learning using handheld devices as play and are more highly motivated, enthused and engaged

• Boys appear to be more enthused when using digital technology, particularly when producing pieces of written work (2)

Impressive stuff, and these findings represent prizes we all value: improved gateway skills, engagement, enjoyment, motivation. Game over for sceptics surely? Alas, one Boss-level obstacle remains. Is it true?

The quotes above are taken from a news website, which only describes the authors' findings. But in order to understand if the research findings are robust, and that they flow from the iPad intervention, we need to be able to access methodology, study design, attainment measures and so on. We need to hear a critical voice to contract with the claims. Otherwise we could just say anything. 

Thrilling sub-heading supported by weak evidence in paragraph 14

What’s wrong with reporting like this? In my opinion, it's unhelpful. In fact I think taken as a whole it makes the business of knowing how to educate children harder. Because if we want to make sure that what we do with children in classrooms is useful rather than frivolous, it’s important that claims of efficacy are matched by evidence, and extraordinary claims matched by extraordinary evidence. This project set the Belfast Regeneration Project back £300K, with change back for a Solero- or a teacher’s salary for a decade if you prefer. School budgets are finite systems and getting more finite by the year.

'Sir! This intervention appears to based on weak findings.'

When we report unconfirmed results like this without challenge, the intellectual landscape of education discourse is changed subtly. This news report will be cited somewhere, by someone who wants to bring a cache of iPads into a school, and someone somewhere will say ‘OK’. That’s great if they have the effect they claim, but what if they don’t? At best a waste of money and time. In fact, that’s also the ‘at worst’ scenario, because children- especially children in deprived areas, don’t have second chances, or time for expensive substitutes for teaching time. When we report research without question, it enter the collective psyche as factual: ‘iPads make kids smarter and happier.’ But what if they don’t? And I don’t have skin in this game. I love iPads. But I also loved Tom Hardy’s performance in Taboo, and I’m not using that in any lessons soon because there is no obvious reason for me to do so.

Show me the money

Ok, so go beyond the slightly breathless news report. Where is the research itself?

The article doesn’t link to anything we can look at, so a quick search reveals that this study is:

‘Gray, C., Dunn, J., Moffett, P., & Mitchell, D. (2017). Mobile devices in early learning. Developing the use of portable devices to support young children's learning. Stranmillis University College: A College of The Queen's University of Belfast, 24.05.2017’

To the website, Robin. Over at Stranmillis University College, we find a link to a press release, where one of the report’s author’s makes these claims:

“The study’s findings showed that, in the five participating schools, all of which were located in catchment areas of high social deprivation and academic under-achievement, the introduction of digital technology has had a positive impact on the development of pupil literacy and numeracy skills. And, contrary to initial expectations, principals and teachers also reported that their use had enhanced children’s communication skills, acting as a stimulus for peer to peer and pupil to teacher discussion.” (3)

There’s a link at the bottom of this breathless review, but it doesn’t work- happily the study is elsewhere on the website (4).

Surely here at last we'll find evidence that robustly stands the claim up? Well, in my opinion, it's a bit disappointing. Why?

1. Completely subjective self-reporting: If you were hoping to find some evidence that children's literacy or numeracy had been demonstrably improved in an objective way, you will go home with empty pockets. All the evidence collected in this areas was in the form of semi-structured interviews with teachers, school principals, student focus groups and parental questionnaires. So the teachers (small focus groups from each of the 5 schools and pre-schools) said things like 'I think they've improved their literacy.' How do we know this? How can we separate any gains from normal progress, or progress attributable to other interventions or processes? 

2.  Questionnaire response rate: 27% (after a second push- the first response was 8%), which seems to my mind to be a poor response. We have no way of knowing how representative this is (although I'll suggest 'not very')

3. Possible design biases: schools were selected to participate in this project based on their commitment to the project, their pre-existing use of ICT and iPads in the school, and their commitment to use iPads in the future, as well as a troubling commitment to 'The benefits of developing literacy and numeracy skills to be gained from the use of iPads.' So, to summarise: schools that were enthusiastic about iPads, already used them and believed they had big educational benefits, participated. 'Person who likes x, thinks x is good' isn't so much a research finding as a disappointing maxim in a fortune cookie. 

4. Variable usage: schools used them at different times, with different apps, in different ways, with different children. In some schools they were used more than others. It seems very hard to discern if like is being compared with like. 

5. Funding. This whole program came about because the Belfast Educational & Library Board was awarded a grant from the Belfast Regeneration Office to 'develop an ICT program.' Was there sufficient critical examination of the need to do so in the first place? Every study needs to suspend disbelief in its own utility, and question its own existence.

6. No control group. What is this intervention better than?

Duvet days: no longer a get-out from teaching.

This study' findings may well be found to be correct, and I’m sure that the authors and everyone involved has the best of intentions and conducted themselves with scruples and integrity. That’s not in question. But questions are all we have at this stage. All we are holding in our hands is a fog of grand claims and optimism. Do iPads turn frowns upside down? Do they turn light bulbs on above confused heads? Are they just a novelty or a distraction? We can’t tell, not from this. A day of terrific press is great for the University, but doesn’t help the debate.

Never mind the quality, feel the tech

I’ve looked at a lot of research that often gets used to support positive claims for the utility of tech in the classroom, and often they don’t stand up in court. Some of the most duplicitous research I have read in this area uses proxies of success that are entirely subjective or impossible to substantiate. ‘tech has the potential to do x’ is the same as ‘tech has not done x yet.’ And ‘boys appear to be more enthused when using digital technology’ could be uncharitably responded to with a ‘so what?’ and a ‘oh really?’ and a 'did it take a £500 iPad to do that?'

And that’s important, because schools are poor and kids don’t often get second chances when they come from deprived areas. Universal, free education is one of humanity’s greatest inventions. Wasting that is a sin, and a theft from people with nearly nothing. Who would rob a child, from a family with nothing but debt?  

Other people's children

Public money needs to be spent as carefully as if it were our own. Other people’s children need to be taught as carefully as if they belonged to us. No child should endure the loss of their right to an education, no matter how digitally it is dressed. If iPads and their ilk can bring benefit to the table, then let them demonstrate it in public. Let everyone see how well they work, and if they do, the truth will be unmistakeable. But when claims are made without data that substantiates it appropriately then we have a right to ask if our money is being spent wisely. This matters. Ominously, the report suggest that:

'These findings should inform the future rollout of similar initiatives and will be of interest to practitioners, policy-makers and parents.'

Ireland, I love you. My family migrated from Ireland. I wish you and your beautiful island nothing but fortune and love. For the good of your children, and the wealth of your nation, and the prospect of better things to come, I suggest that you use these findings wisely. Keep your hands away from the cheque books for now and wait until better data supports swapping out precious resources for digital magic beans.

I'll end with a lovely quote from Piaget, which starts the report:

'The principal goal of education is to create men and women who are capable of doing few things, not simply of repeating what other generations have done—men and women who are creative, inventive, and discoverers, who have minds which can be critical, can verify, and not accept everything they are offered (Piaget,1973).'

Be critical? Verify? Not accept everything we're offered? I couldn't agree more. 

(1)  http://www.bbc.co.uk/news/uk-northern-ireland-40021187

(2) Ibid

(3) http://www.stran.ac.uk/informationabout/theuniversitycollege/newsevents/title,756164,en.html

(4) http://www.stran.ac.uk/media/media,756133,en.pdf