The retraction of three papers in prominent medical journals in the space of a week over concerns about data quality has highlighted the perils of rushing to publish scientific studies in response to the coronavirus crisis.
Two journals, the New England Journal of Medicine and The Lancet, in the same week in June withdrew reports on trials of malaria drug hydroxychloroquine as a treatment for the virus. The third, Annals of Internal Medicine, retracted a study into the effectiveness of face masks in blocking viral transmission.
“We had never seen a week like it,” said Ivan Oransky, co-founder of the Retraction Watch database and a scientific integrity campaigner. “Covid-19 has created a perfect storm, exposing all the vulnerabilities in the publishing system.”
Since the start of what has become the worst virus pandemic for a century, scientists from microbiology to mathematics have rushed to study every aspect of the disease — and push their findings out in scientific literature.
Data collected by Airfinity, a UK-based analytics company working for the life sciences industry, show Covid-19 papers were being pumped out at an ever-increasing rate. As of June 4, some 23,000 had appeared in just a few months, of which 18,300 were published by academic journals and 4,700 were on preprint servers such as medRxiv and bioRxiv, which post papers online without peer review.
“Our analysis indicates that journals, including the top tier, have applied different scientific standards for what they deem fit to publish on Covid-19 than in other therapy areas,” said Rasmus Bech Hansen, Airfinity chief executive.
He said that both the Lancet and NEJM, regarded as the best medical journals, were publishing far more observational studies and far fewer randomised clinical trials — the gold standard of clinical research — about Covid-19 than other diseases.
Retraction Watch lists 15 Covid-19 papers that have been retracted since the start of February, two temporary retractions and two “expressions of concern” by journal editors. “It is not a leap to say that the rush to publish is having an impact on standards,” said Mr Oransky, vice-president of editorial at Medscape, a website offering medical information for doctors.
Walid Gellad, an associate professor of medicine at University of Pittsburgh, was among the first to notice inadequacies in data used in the withdrawn Lancet and NEJM studies, which was provided by Surgisphere, a little-known US-based company. He too views their retraction as a symptom of a wider problem.
“There is a difference between being rushed and being fast. There has definitely been a rush [among] authors, editors, journalists and everyone” since the pandemic began, Prof Gellad said. “Everyone wants to help, but sometimes getting people to slow down is the biggest help when they are in a hurry.”
Mr Hansen said that, under normal conditions, the time between a paper being submitted and its publication was typically three to four months. “Now we are down to two to three weeks for Covid-19 papers,” he said.
Every scientist engaged in Covid-19 research to whom the Financial Times has spoken recently has experienced pressure from the rush to publish — as an author, reviewer or journal editor.
“We’re all struggling to get our papers out as fast as we can,” said Azra Ghani, professor of infectious disease epidemiology at Imperial College London. “It is quite a challenge to deal with normal day-to-day science when you’re working under such pressure.”
A particular issue is the peer review process, in which journals send out papers before publication to scientists in the same field, who are asked to assess the quality of the research, spot errors and suggest improvements.
“I get about 10 requests a day to review articles — and all I can actually do is four per month,” said Thomas Hartung, toxicology professor at Johns Hopkins University. He said he had four promised reviews to complete “but getting my own paper out has to take precedence”.
Jeremy Sanders, chemistry professor at Cambridge university, agreed. “When reviewers are rushing to get their own work out, it is not surprising that some work gets through the process that is not of sufficient quality. Some skimping is inevitable in the current atmosphere when everyone is working on Covid at breakneck speed.”
He added: “It’s forcing publishers to speed up their processes, which might result in more papers that are shoddy or worse.”
Prof Hartung has also experienced the problem from the other side, as an academic journal editor. “I sometimes have to ask 30 people to find two reviewers,” he said.
Peter Openshaw, professor of experimental medicine at Imperial College London, and a journal editor, said his current workload was “absolutely immense”.
“The time pressure on referees does not allow us to drill down to the level of detail that we would like when looking at data appendices and figures,” he said.
Mr Hansen, the Airfinity chief executive, said there was a simple way to relieve the pressure that was driving the rush-to-publish pressure — do less.
“Good science, including peer review, takes time and journals have a responsibility to slow things down,” he said.
Get alerts on Coronavirus pandemic when a new story is published