What went wrong with the A-level algorithm?

You can enable subtitles (captions) in the video player

So why did the government, through the regulator Ofqual, decide to tweak the A-levels this year? Well, essentially, it was because they thought that teachers had been too optimistic or had inflated the grades and they want to keep A-level grades the same standard every year, the gold standard, as they like to call them. And if they hadn't done that, they looked at those grades that the teachers had submitted and they realised they were inflated very significantly. And it would have meant that A-levels were a different standard this year to other years. Some people think that doesn't matter, but the government thought it did matter. And that's why the computer algorithm got involved.

So how did the algorithm work? Well, in the vast majority of cases, Ofqual, the regulator, took information for each school and each subject in that school from the past, from their performance in the past, and then worked out a distribution of grades. From A star at the top to unclassified at the bottom of what they thought this year's crop of students would get. They adjusted it up or down if this year's students seemed better than previous years based on their GCSE results.

But they then said, right, this school will get, let's say, three A stars, five A's, ten B's, six D's, two E's, and one U. And then they just took the teacher rankings in that subject and said if you rank one you'd get the A star. And if you ranked at the bottom you'd get the U. And the teacher grades for that school would have had no effect whatsoever. It would have been done entirely on the distribution that Ofqual said was the right distribution for that school and the rankings that the teachers had given.

So what's unfair about this process? Well, there were three, I think, big problems with the process. The first is that it meant that the outliers couldn't be accounted for in the system. So if you were the brilliant student in a school that never had a student like you before, you literally couldn't get the top grades because the algorithm would not allow any top grades to be given for that school. And that was monstrously unfair for some of the best pupils in poor-performing schools across the country.

The second unfairness related to small schools in particular. Most of these were private schools. Now there, Ofqual thought that their algorithm just didn't work for people where there were fewer than five or so students doing a particular subject in a particular school. So then it just gave the kids effectively the teacher-assessed grades, which it knew to be optimistic. And you've seen some very small private schools doing that fantastically well in this set of A-level results.

So and also, you see that in some subjects doing well, where generally, whether it's private or state school, there are very small groups of students taking that A-level. So we see big increases in the marks for German, for classical Greek, for music. And these are definitely then unfair compared with students who've taken the big subjects, like maths, the sciences, or English or history.

And the third unfairness was in the rounding method. That meant that even if you were ranked one lower then another student in your school, if there weren't that many grades to be given, and let's say the algorithm had said the second lowest grade would be a B in the school and the lowest grade would be a U, the lowest-ranked people would get that U and the one just above that would get the B, even if they were basically the same. And there was no means of getting around that ranking system. So it meant that the big unfairness was that the grades were imposed on the schools rather than taking account of what the teachers thought and then moderating them up or down a little bit.

So what could Ofqual have done differently? Well, moderation or standardisation isn't a new thing. It happens in education all the time. In primary schools, writing exams, for example, at the age of 11 are moderated every year. Doesn't cause any particular problem because you use the teacher grades and then you use whatever evidence exists to mark them up or down a bit if certain schools have been seen to be over optimistic or pessimistic.

And it would have been perfectly possible for Ofqual to do a very similar system, where it takes the fine grades that teachers assessed for each child and then looked at the whole school together and moderated that school up or down a bit to take account of whether that school was overly optimistic or overly pessimistic. And it could have given schools, effectively, a score they had to reach in advance, so they would know and have the internal arguments about whether they'd been too optimistic or too pessimistic, or whether German had been too optimistic and computer science had been too pessimistic. And they could have that argument when they know the children rather than it all being imposed by an algorithm.

And that would have been seen, I think, much more as a much fairer system. And it would have gone around a lot of the problems, because you could still give the exceptional people in a poorly-performing school the exceptional grades. It's just that the average would have to be in line with that school's average in the past.