The Bones Have Spoken: Is Value-Added Crystal Bollocks?

It's gospel, mate.

NICK GIBB  HAS HIS BALLS OUT TODAY! Calm down, major, his Crystal balls. Today's piñata is the Great Satan of Fischer Family Trust data, and the lesser demons of value-added and predicted grades, and boy am I going to beat the HELL out them.

He's- quite rightly- spoken out against the gamification of league tables, where schools, in an attempt to meet the success criteria dictated to them, put their shoulder to nothing but those criteria. Every teacher knows about this- intervention classes aimed at C/D borderline students; not entering the hopeless for final exams; press-ganging children into high-value BTECs for point score advantage, and so on. It's evil, but perhaps understandable when the stakes for schools are so high; let your slip show on the league tables, and you might as well load all six chambers of your gun with dum-dums and press the muzzle to your temple.

So what's the Funky Gibbon proposing? Stand easy citizens- schools will be exhibiting their Contextual Value Added scores from now on, not unlike a baboon, presenting its ghastly floral undercarriage. Gaze into the abyss, for it cannot be unseen.

Pick a Card...any card

Now I have a problem with CVA. Not me personally- my CVA is, thankfully, bulletproof, fireproof, and susceptible only to Kryptonite, and I only say that so you don't think I'm a bitter victim of its diabolic engines. I just don't think it's that useful. In fact, I think the way it's used, it's corrosive, and actively damages education.

In many ways picking a fight with predictions is an easy task, because I'm attacking the belief that we can tell what is going to happen about things that have not happened yet. Can you see where I might be going with this? Nobody can tell the future, not even with a great big telescope and all the data in the world. Not even then. While science has offered us many sweet meats and shiny trinkets in the fields of the natural world, it has yet to lift its petticoat in any meaningful way in the realm of something more stubbornly unpredictable: us. Human beings resist the reductive powers of physical determinism; we just won't do as we're told. This is why we are human, and not, say, a Meccano Set.

The problem is that schools are- now, at least- very much in the business of predicting the future. Why? Because..well, the simple answer is because Lucifer the Hoofed one rules this world, but that won't get me into the smart edu-blog clubs. The other answer is that we are required on an annual basis, to show that children have made good progress, not merely an A, or a B, but progress; that they have become smarter. The hope is that it's something to do with us. How do we show this? By comparing what they get to some notional projection of what they 'should' have obtained. And that's where the problems start.

If Little Davina or Limoncella start year 7 with a level 5 in English and Maths, then we know that's good- probably better than most of her peers. So you'd hope she'd leave with a bag full of A's at GCSE wouldn't you?

Your data manager, yesterday.

Most schools (and by most, I'm suggesting 100%) use Fischer Family Trust (FFT) or ALIS data to set targets for pupils. What am FFT? They're an organisation that sell data; they take the results of all the children in their data set, and then track the % of those children on, say, level 4 at the end of Key Stage 3, to see how many of them achieve an A or a B or a C at the end of their GCSEs. They then plot statistical projections of likelihood that Davina will get those grades.Sounds simple. On this level, it is. Then the problems start:

Here are SOME of the problems; I'll deal with more in a subsequent post:


1. Nobody but wizards can understand how CVA is calculated. Do you know how many factors are taken into account when constructing the predicted median for a school's grades? Here are, I think, the key ones:

• pupils’ prior attainment
• special educational needs;
• English as an additional language;
• pupil mobility;
• age of pupils;
• an ‘in care’ indicator;
• ethnicity;
• free school meals status; and
• a measure of social deprivation.

These are then thrown into the tombola of certainty, the handle is turned, and the future is born, neat as an egg. When Einstein published his theory of Relativity, it was alleged (probably erroneously) that at that point, only a dozen men and women in the world could fully understand it. The number is even smaller for CVA calculus. It is a done deal; the High Priests have spoken, and we must genuflect to the wisdom of the ancients. I'm not saying they're lying or anything, I'm just saying I have a gut mistrust for any pupil prediction that can be obtained with NO KNOWLEDGE of the pupil whatsoever. This ties into my next point:

2. It's de-professionalises the whole role of the teacher. Excuse me? You want to say what one of my students is probably going to get this year....and you haven't even met them? You say that you DON'T sit in the classroom with them every day, working with them, talking to them, marking their books and correcting their mistakes? Well I do. I know my students. I can see their potential. I have my own data set I base predictions on, and it isn't one I can write down or reduce to an algorithm. It's in my mind, and my gut, and it's part of what being a teacher is about. My contempt for the practise of reducing children to points on a scatter graph is endless and bottomless. It could boil iron.


3. They're NOT predictions. I can't emphasise this one enough. EVEN THE FFT DOESN'T THINK THEY'RE PREDICTIONS. They are wise, the wizards of the Fischer Family. This is a common problem: often, research is published in any field, with sensible, delicate, cautious conclusions, only for the recipients (in this case, politicians, journalists and school leaders) to go OH BOY LOOK AT THESE COOL PROPHECIES WRITTEN BY THE WIZARDS IN THE MOUNTAIN. THESE ARE OMENS FOR SURE LOOK THE BONES HAVE SPOKEN.

Here's what the FFT actually have to say:

'The FFT Data Analysis project produces ESTIMATES of likely attainment. The estimates are calculated for each pupil and, from these, school and LA estimates are calculated. They are called estimates – not predictions or targets – because they provide an estimate of what might happen if your pupils make progress that is line with that of similar pupils in previous years.'

NOT predictions. NOT targets. That's because they're professionals, who are more than aware that we cannot tell the future. They're statistical guesses; they're probability fields; they are the statistical equivalent of saying that Summer will probably be hotter than Winter, but we don't know if it'll rain today or not. The comparison with the weather is appropriate: we know what water is; we know what heat is; we know what pressure is; but the enormous density of causal factors makes weather forecasting impossible for more than a few hours in the future. The Met office gave up long-term forecasts a long time ago. And humans, as I repeatedly point out, are a Hell of a lot more complex than a damp bag of warm gas in Brownian motion. (Most of them- I saw TOWIE once)

'What band should we use, oh omniscient one?'

So when the FFT says that a given child is estimated a B at GCSE, based on prior attainment data, once social and circumstantial factors have been accounted for, what does it mean?

Almost nothing. Almost nothing.

What it means is that many children with similar socio-economic and attainment levels achieved that grade. So what? Most children in Mozart's street didn't grow up to write The Magic Flute, but he did. Most children from Omaha, Nebraska didn't grow up to lead a black consciousness movement, but Malcolm X did. I taught a kid who scraped a C in bottom set RS, who scored a U in AS, and then an A at A2. The human spirit is a genie; it is absurd, noetic, a screaming eagle of ambition and indeterminacy. It is a ghost, a comet, a nuclear furnace of optimism and ambition and impossibility. It is also a disappointment; the anti-life, failure snatched from the jaws of victory. House prices can go up as well as down. That is what makes being alive so glorious and terrifying.

I have a knowledge of my children's predicted grades that approaches telepathy, because I know my subject and I know my kids. But every year I am knocked sideways by kids who exceed my expectations and those who ridicule them. Nobody can predict the future. Guesses are fine, but let's admit that's what they are.

Guesses.

Let's stop pulverising children with our bureaucratic assumptions about their potential. Can you imagine what it must feel like to be told by your teacher that your prediction is a D?

F*ck. That.

You know what my expectation of my children is? An A. For everyone. That's the target I set myself, and if I don't get it, well, I try again next year. I don't cry into my coffee, I just try again.

Here's a thing: what does it even mean to 'aim for a C, or a B'? Have you ever seen a kid revise, and try to get a B? It's nonsense. Kids try as hard as they can/ can be bothered, to get the best grade they can. If you set a child to run 100 metres, and they really bash their guts out on it, can you imagine asking them, 'What speed were you going for?' No. They just run. They just run. Target setting has become the fetish of 21st century teaching. It is another ravenous, ridiculous imported imaginary animal from the paradigm of the market place, where ambitions are plucked from the air- and they are- and called 'predictions', when they should be called 'hopes'.

When I worked for the Great Satan of commerce, every year our units were given sales targets to reach. Meeting them, would ensure we survived. Failing to meet these targets was an assured spell in the cooler. The targets were usually 'Last year's sales, plus 5%'. Brilliant. How long did you think it took for them to come up with that?

Aggregating outta Here

The market has infected the classroom, and I use the word infect carefully. It is a sickness that cuts down children, teachers and learning. It is already absurd that the economic model is predicated on infinite expansion, in a world that is plainly finite; it is doubly absurd to do so in a room full of children, nenulus and spectral a commodity as you can imagine, in an environment where they are learning, an abstract multiplied by an abstract. The Data Oracles pretend they are dealing in beans, when they are counting intangibles. They are trying to catch a fairy tale with an iron claw.

I have no idea what they have in their nets. But it isn't real.

I'm not finished with you, CVA *looks sternly at it*. Keep looking over your shoulder.

UPDATE: In response to some comments to this blog post, I've added another supplementary post to it here. I haven't changed this blog, because I think the discussion makes more sense if I jes' leave it as it is.