The 21st century’s sexiest job is the data scientist, Harvard Business Review claimed recently. Such employees are “a hybrid of data hacker, analyst, communicator and trusted adviser”, the US business school journal said, pointing out that this combination of skills is extremely rare.
Therein lies the problem for many businesses across the globe. While company managers recognise the benefits that big data can bring, they are struggling to find people with the right skills.
The rush to make use of big data shows no sign of slowing. A survey of 600 companies in the US and UK conducted last year by Accenture, the management consultancy, found that two-thirds had appointed a senior figure to lead data management and analytics in the past 18 months. Even among companies that had not made such an executive appointment, 71 per cent expected to do so in the near future.
Recruitment consultants also report soaring demand for data analytics specialists. Cititec, which specialises in the information technology sector and has offices in London and Amsterdam, says: “In the first six weeks of this year we received as many big data requests as we did for the previous six months. We reckon it will be up 100 per cent on last year, if not more.”
Indeed, big data is fast becoming a crucial specialism for IT recruitment agencies. “We now have a big data specialist – we didn’t have one a year ago,” Cititec adds. “It’s a competitive market, with many companies vying for experienced talent.”
Brian McCarthy, executive director of Accenture’s financial services analytics practice in North America, says global demand across the industry means there is a massive shortage of data analytics skills, especially in the US and the UK.
“Graduates with the right kinds of backgrounds for data scientist – computer science, statistics, machine learning – are coming out of the universities, but they are not coming out in sufficient numbers,” he says.
Mr McCarthy adds that although many companies are turning to contractors – nearly six out of 10 companies report turning to external analysts and consultants, according to Accenture – they are still unable to find the people they need.
The shortfall is such that IT contractors who are experienced data architects or business analysts can command between £500 and £650 a day in the UK, says Cititec. At the most highly qualified end of the profession, PhD-level data scientists have attracted salaries of £95,000.
“[The solution to the problem] is not even offshoring,” Mr McCarthy points out, “because there aren’t enough experienced people in emerging economies like Indian, China and Brazil.”
And the squeeze looks set to continue for a few years.
An Accenture report, Analytics in Action: Breakthroughs and Barriers on the Journey to ROI [return on investment], published this year, forecast that in the US and the UK alone, jobs demanding advanced knowledge in science, technology, engineering and mathematics (STEM) would grow five times as fast as jobs in other occupations by 2018, and four times faster than jobs in information-intensive industries such as financial services.
Emerging economies are producing STEM talent in greater numbers than developing economies, the report said, but it is not enough to meet likely demand across the globe.
Indeed, a survey conducted last year by Accenture’s Institute for High Performance looked at demand for analytics experience in the US, China, India, the UK, Japan Brazil and Singapore. It found that by 2015, with the exception of China, all of these countries would be facing a net shortage of PhD graduates qualified for analytics scientist jobs.
“The US, the UK, Japan, Singapore and Brazil will almost certainly experience significant shortages of this top tier of talent,” Accenture said. “India, with its booming analytics services industry, will also struggle to produce enough new PhDs to fill all the new analytics scientist jobs.”
The US, for example, is expected to create 44 per cent of the new jobs for analytics experts but only 23 per cent of the supply, leading to a shortfall of nearly 32,000 workers.
Only China looks like showing a modest surplus, although Accenture cautioned: “Shortages are also possible in China if demand for analytics accelerates.”
Nevertheless, there is plenty of training available for IT professionals who want to acquire data analytics skills. Courses – both online and class-based – are offered by large IT vendors such as IBM and Oracle, as well as by independent IT training specialists.
Training is also offered by big data entrepreneurs. Cloudera, the provider of software and services based on the Hadoop framework, has its Cloudera University, for example, while competitor MapR has its MapR Academy.
Cititec says: “A decent data analyst should have the requisite skills to move into analytics, perhaps learning or receive training on the job. IT professionals are beginning to be aware that the big data sector is becoming important, and they are asking, how do I get these skills?”
However, Mr McCarthy at Accenture believes retraining IT specialists for the new world of data analytics is rarely as simple as putting existing staff on a course.
“On-the-job training is helping to address the shortage of experienced people to a certain extent,” he says.
“The challenge is that it’s not just a matter of learning new tools – there is also mindset shift required… That’s proving even more challenging for IT organisations, especially if they have people who grew up with the technology of the 1990s and the 2000s. In many cases, it’s easier to bring in new people than to attempt to retrofit an existing workforce.”