Will recruitment ‘gamification’ drive diversity or replicate biases?

From government office workers to private sector management consultants, technology is displacing human interviews in a recruitment trend that is accelerating during the era of coronavirus.

Aspiring British civil servants now answer multiple-choice questions in video scenarios about how to handle difficult co-workers. Would-be McKinsey consultants play complex ecological computer games in which they protect plants from animals by placing rocks, obstacles and predators in the way.

The formats vary but even before the pandemic, a pattern was emerging in graduate recruitment and selection for short-term internships: applications that take longer to complete and growing use of computer-based assessments. The interview is increasingly minimised and deferred to later stages of the process.

Defenders say the trend is a fair and cost-efficient way to recruit a diverse group and handle increasing applications. Critics are concerned that the technology and its underlying assumptions remain unproven.

Amit Joshi, a professor at IMD business school in Switzerland, is researching computer recruitment techniques. “From the perspective of the organisation, there can be really significant gains in efficiency,” he says. “I don’t know if there is a gain in the quality of participants. I’ve heard tons of complaints from [applicants], some of whom have said they will never apply to the company again.”

With cost-cutting a priority as employers adjust to the impact of Covid-19, he says “some organisations are accelerating the use of algorithms, and resistance on the part of employees has been reducing”.

We are trying to open the aperture of who we recruit and the places we recruit from

A survey by the Institute for Student Employers in the UK showed that just 30 per cent of companies used face-to-face interviews in the first stage of graduate recruitment last year. Psychometric tests were used by 59 per cent and gamified assessments by 10 per cent. In a report last month, the institute concluded that “there is a strong indication that online recruitment may become the new normal”.

The adoption of technology in selection is relatively recent, partly reflecting growing competition for jobs. “Volume is definitely a factor,” says Dan Richards, UK and Ireland head of recruitment at EY, the accountancy and consultancy firm.

EY processes 45,000 applications for 1,500 graduate apprenticeships in his region each year. “In all our student selection programmes we have an immersive digital experience,” he adds.

Some employers say computerised approaches help to increase diversity. “We are trying to open the aperture of who we recruit and the places we recruit from,” says Keith McNulty, global director of people, analytics and measurement at McKinsey.

“There’s a danger in selection that you tend to default to the easiest decisions — those at the top universities. We want to give opportunities to people from a wide variety of backgrounds,” he says.

McKinsey has used computer games developed by Imbellus, a US start-up, to test problem-solving skills. Only those who pass the test are selected for interview.

We have a long history of discrimination in hiring. We cannot allow recruitment platforms to simply propagate the past with naive AI

One benefit Mr McNulty sees in the “gamification” of recruitment is that it is more entertaining, easing candidates’ frustration at traditional question-based selection — 90 per cent of applicants who have used the process agree, he says.

Others are concerned about a different type of “gaming” — the extent to which such tests can be anticipated and prepared for, sometimes with the help of a fast-growing breed of specialist consultancies. Critics also argue that technology risks including — and concealing — biases.

Cathy O’Neil, a computer scientist and author of Weapons of Math Destruction, a book critical of AI, says: “We have a long history of discrimination in hiring. We cannot allow recruitment platforms to simply propagate the past with naive AI, which is what would happen by default. Instead we must demand evidence that what they are doing is fair, and how they define fair.”

Late last year, the Electronic Privacy Information Center, a US non-profit organisation, filed a complaint with the Federal Trade Commission to investigate HireVue, a start-up used by many recruiters, for claims of potential bias and inaccuracy in the use of its computerised interview technology. Its software uses facial analysis, among other techniques, to assess eligibility for jobs.

“There are examples of start-ups using cool metrics which may have no or low actual relation to performance — for example facial expressions and tone of voice from video,” says Prof Joshi at IMD. “However, they often ignore real, but harder to measure, metrics.”

Jack Buckley, president and chief scientist at Imbellus, which developed McKinsey’s game, says its cognitive testing approach is based on “bullet proof” science. He stresses that his company’s does not use artificial intelligence. “I’m very sceptical. I think we’ll see a backlash,” he says.

Similarly, EY and the UK Civil Service do not use AI. “All our applications are assessed by a human,” says Mr Richards at EY. “We didn’t go down the AI route. We felt we wanted a blend of smart people and smart machines working together.”

A British government spokesman said the Civil Service’s online tests had been used 1m times since 2017 “to help make sure the candidate experience is engaging rather than more traditional text-based assessments”. The spokesman said the tests had shown “positive diversity outcomes” with “no marked performance difference” in terms of gender, ethnicity, socio-economic background, or university attended.

Critics including Prof Joshi and Ms O’Neil stress that human-based recruitment is also biased, and that it is too early to tell the long-term impact on productivity of recruits hired using technology.

A concern for students at elite universities, which provide a quality filter for human recruiters, is that computer-based approaches seeking different skills will undermine the value of studying hard to achieve high educational standards.

If technology comes to dominate recruitment, the disruption ahead may yet be far greater for employers, applicants and the education system that has traditionally prepared them.