Nearly 10 years ago, an Italian court handed down six-month suspended prison sentences to three senior Google executives. Their crime: failing to prevent the internet company from hosting a user’s video showing an autistic child being bullied. The verdict was seen as an outlier, and later overturned.

Now it seems everyone wants to get in on the act. Australia reacted to the live-streaming of the Christchurch massacre that left 50 people dead in March with a new law that could see internet executives locked up if they do not act quickly enough to remove violent content.

The UK has also just floated the idea of imposing criminal penalties. It came as part of a sweeping proposal this week to make all companies that carry user-generated content and communications responsible for everything on their sites.

The UK’s attempt to put itself at the front of the pack in legislating a safer internet has prompted inevitable warnings about the threat to civil liberties online. It “might fall afoul of any reasonable human rights standard for free speech”, Alex Stamos, Facebook’s former chief information security officer and a professor at Stanford University, said this week. 

A police officer patrols at a cordon near a mosque in central Christchurch, New Zealand, Friday, March 15, 2019. Multiple people were killed in mass shootings at two mosques full of worshippers attending Friday prayers on what the prime minister called "one of New Zealand's darkest days," as authorities detained four people and defused explosive devices in what appeared to be a carefully planned attack. (AP Photo/Mark Baker)
After the massacre in Christchurch, New Zealand, Australia drafted a new law that lays greater responsibility on internet executives © AP

Yet there is no doubting that the fight against hateful content and misinformation online has taken a sharp turn. Facebook and Google — owners of the biggest platforms for user-generated content — are suddenly being forced to adapt to a new reality. For years, they sheltered behind broad immunities that left them with no liability for content, provided they acted to remove illegal material promptly when notified.

The change in mood has been profound — as evidenced by comments from Bob Iger, the chief executive officer of Walt Disney. On Wednesday, according to Variety, he told an awards ceremony that “Hitler would have loved social media”. It was, he added, “the most powerful marketing tool an extremist could ever hope for”.

In some ways, it is surprising that the reckoning did not come sooner. As one early Google executive says of the company’s success in avoiding most forms of national content regulation for so long: “We thought we had 10 years — it turned out we had 20.”

In an influential 2006 book, co-author Tim Wu, a Columbia Law School professor, predicted the internet would inevitably fragment into nationally regulated networks as governments acted to protect their citizens online, imposing their own legal and cultural norms.

It took longer than expected, he now says, because of the dominance Google and Facebook have had over so many national markets, enabling them to set international norms that many governments were hesitant to challenge.

Two things have combined to change the mood. First, the powerful visceral reactions to tragedies such as Christchurch and the suicide of British teenager Molly Russell, whose father said she viewed Instagram images of self-harm.

Terrorist content and other forms of hate speech, along with child endangerment, are the issues that have galvanised governments to take action. But weariness with election-related misinformation and “fake news” has continued to spread around the world.

The other big change has been a collapse in the value of the internet companies’ political capital. That includes in Washington, on which they could once count for support. 

elizabeth warren facebook ad
An Elizabeth Warren Facebook advert calling for the break-up of Facebook

As they expanded around the world, Google and Facebook initially found natural allies on the left of US politics, donating heavily to Democratic candidates and enlisting their support on issues such as net neutrality. But the recent series of scandals, from the promulgation of fake news, to large data breaches, to Russian election interference, have created a rift in that alliance.

It is now Democrats in Congress who are pushing hardest for tough new data privacy rules, including Elizabeth Warren, the senator and presidential candidate, who has called for the break-up of companies such as Google and Facebook. In response, companies have begun to cultivate allies on the right, partnering with people such as rightwing donor Charles Koch in their fight to resist the toughest forms of regulation. 

This effort is struggling to gain traction, however. Many Republicans, already suspicious of the new tech companies, with their liberal Californian roots, now accuse them of censoring conservative content. 

If countries such as the UK feel newly emboldened, then it could have far-reaching consequences. This week’s proposals could have an impact on attitudes far beyond Britain’s shores, warns Michael Posner, professor of ethics and finance at NYU’s Stern School of Business and a former Obama administration official. “I think it will lead to each government deciding what is illegitimate or not,” he says. “And you end up with what Iranians have called the ‘halal internet’ — where [each country] decides what is kosher or not.”

Employees work inside the "War Room" ahead of Brazil's runoff election at Facebook Inc. headquarters in Menlo Park, California, U.S., on Wednesday, Oct. 17, 2018. Even if it's not a full solution, the War Room is symbolic of Facebook's work to assuage public concern about the fake accounts, misinformation and foreign interference that cloud discussion about elections on its site. Photographer: David Paul Morris/Bloomberg
Facebook's 'war room' ahead of Brazil's runoff election, a sign the company is responding to public concerns © Bloomberg

For some, greater intervention by governments in online content is inevitable, and even welcome. Mr Wu, for one, argues that it all “depends on the legitimacy of the underlying government. If people think their democratically elected leaders should act”, then intervention is valid — even if that leads to some limitations on speech.

However, Mr Posner says internet companies need to act quickly to set new rules, rather than give less democratic governments an excuse to step in. “There are too many governments that are authoritarian in nature,” he says. 

Russia and Singapore have each introduced legislation in recent weeks to clamp down on “fake news” and misinformation — moves critics say are an excuse to extend government control and stamp out speech they do not like.

Sensing how quickly the winds have changed, internet companies have been shifting their ground. Mark Zuckerberg, caused a stir last month when he called for government regulation. Top of his list of issues to be addressed was harmful content — “hate speech, terrorist propaganda and more”. 

“Internet companies should be accountable for enforcing standards on harmful content,” he wrote. “Regulation could set baselines for what is prohibited and require companies to build systems for keeping harmful content to a bare minimum.”

Alex Stamos, Facebook’s former chief information security officer, has warned that increased regulation could threaten free speech © Getty

It appeared to be a surprising about-turn, given how hard the internet companies have lobbied to be excused any formal role in policing content on their sites. But some critics question how serious the Facebook co-founder is about fighting back against noxious online content, and argue instead that he is trying to shift responsibility.

“It’s designed to lock in the current business model and transfer the blame to governments,” says Roger McNamee, a former adviser to Mr Zuckerberg who has become one of the main voices in Silicon Valley calling for a reckoning.

Some warn that the action Mr Zuckerberg advocates could have unintended consequences. “Facebook is looking to get out from under all these political controversies,” says John Samples, vice-president at the Cato Institute, a libertarian think-tank. “But the danger is that if you have regulations that require 20,000 content moderations, almost no one is going to be able to do that, except an extremely large global undertaking.”

One of the key questions is whether advertisers will continue to stay loyal to services such as YouTube and Facebook. In 2017, some advertisers turned off their YouTube advertising spending over fears they might be placed next to extremist or inappropriate content. YouTube responded by announcing policy changes, algorithm updates and content takedowns, and most of the advertising dollars have flooded back.

There have been fresh stirrings of discontent this year. On Thursday, Procter & Gamble chief brand officer Marc Pritchard bemoaned a lack of “brand safety” for advertisers on social media and warned that he would only partner with platforms that “promote civility”.

Whatever their motivation, the companies have been racing to convince politicians and others that they are serious about taking action. This week Facebook outlined its latest product tweaks in a bid to remove harmful material and prevent “borderline content” from spreading so easily on its network. 

But critics question how far the online platforms will be able — or willing — to go. The vast amount of material posted on their sites threatens to overwhelm even the large numbers of people they have hired to police content.

The fight against noxious content will also be made harder by initiatives such as Facebook’s promise to encrypt its messaging, says Mr Stamos. “You can’t moderate content unless you see it. You can’t find bad guys unless you are collecting data about them,” he says. “I think this is the conflict that is going to be the biggest over the next few years.”

Ultimately, though, it may come down to a matter of dollars and cents. Critics such as Mr McNamee say the online platforms were designed to stir up strong reactions in their users: the most controversial content plays a key role in provoking the powerful responses from users that enable them to model and predict human behaviour.

“The problem with hate speech is that it is fundamental to the business model of Google and Facebook,” he says. 

Get alerts on Social Media when a new story is published

Copyright The Financial Times Limited 2019. All rights reserved.
Reuse this content (opens in new window)

Follow the topics in this article