The ballyhoo around big and open data raises some pressing questions. Are we all puppets in a wired world? Does the growing reach of online surveillance and censorship pose a threat to the future of democracy? And, while having fun online, did we unwillingly give too much of ourselves away?
There aren’t many people better placed to talk about these topics than Sir Nigel Shadbolt, the 57-year-old professor of artificial intelligence, computer and web science who is credited with kick-starting the revolution in public data sharing in the UK. “We are at a tipping point,” says Shadbolt. “There is a terrible asymmetry in our lives: online, we generate tons of information about ourselves but have virtually no control over it. That needs to change.”
It is just over one year since Shadbolt and Tim Berners-Lee, inventor of the world wide web, co-founded the Open Data Institute (ODI). The east London-based organisation, whose motto is “knowledge for everyone”, hopes to spark an open data culture among businesses that has economic, environmental and social benefits.
We are speeding through undulating countryside in Shadbolt’s Maserati sports car (“My third,” he admits sheepishly, “I’m a bit of petrol head”) towards his cottage in Lymington, in the New Forest district of Hampshire, southeast England. No sooner have I fastened my seatbelt, Shadbolt is reflecting on the erosion of traditional political party lines, beyond left and right. “It might be better to support a party on the basis of how open or closed they are, in terms of making data available,” he says.
“In the past few years, most people have all too readily equated the internet with freedom and prosperity. But recently that has given way to a more realistic assessment of what it’s good for and who benefits. Now we are at the stage where we need to have a debate about civil liberties and how far we should allow the state to intervene on our behalf on the grounds of things like national and personal security. Rightly so, the debate around the NSA has lit a fuse.”
Five years ago, Shadbolt co-authored The Spy in the Coffee Machine: The End of Privacy as we Know It, an alarming treatise outlining how technology has eroded privacy and continues to do so. “It turned out to be very prescient,” he says. Have things got worse? “The trouble is that organisations such as the NSA are now in possession of such powerful technologies they can do things at a scale they weren’t able to before.”
We drive down a narrow country lane and park outside an attractive 1820s yellow-brick cottage. Six terracotta barley-twist chimneys thrust up towards the blue winter sky. “They are a bit self-aggrandising but I love them,” says Shadbolt. Two gardeners work on the manicured front garden.
“People say I’m mad not to develop this land,” says Shadbolt, leading me into the half-acre wooded area. “But we get all sorts of wildlife here: deer come through from the nearby forest; there are foxes, birds, badgers. Of course, none of this is good news for my flowerbeds.”
Shadbolt moved into the Grade II-listed house with his wife Bev Saunders – “aka Lady Shadbolt” – and their two teenage children at the turn of the millennium.
“These forest cottages are very cosy but they have very little space upstairs. The kids have had to get used to growing up in fairly small bedrooms. That’s why they are so good on the ‘second home,’ ” he says, referring to his 10-metre sailing boat moored in the Solent, the passage of water separating the Isle of Wight from the English mainland.
Shadbolt was born in London but raised in the Peak District village of Ashford-in-the-Water, living a “bucolic existence” until he left home for university in Newcastle. He wanted to study artificial intelligence (AI) but there were no undergraduate degrees in the subject back then. He took a PhD in AI at Edinburgh University in the 1970s. “We thought it would be impossible to build a computer system that would be able to beat a world chess champion,” he says. “We couldn’t have been more wrong.”
Although the AI department at Edinburgh was “excellent”, according to Shadbolt, it was in danger of closing down because of the 1973 James Lighthill report, “which claimed AI was all nonsense. There was such a lack of respect for the subject back then.”
Shadbolt is, he says, “as old as the space race. I grew up with Mercury, Gemini and Apollo” – the first, second and third space flight programmes led by Nasa, the US space agency – “and lots of the astronauts on those missions were my heroes.”
Many of those heroes are now up on his walls. “I don’t collect stamps, I collect all this,” he says, pointing to a black-and-white photograph of Apollo 1, signed by the crew members before they died when the shuttle caught fire during testing on January 27 1967.
He leads me into his ground-floor study and unlocks a “cabinet of space wonder”. There are folders and folders of the stuff. “I’ve got photos from [the Soviet astronaut] Yuri Gagarin’s flight surgeon that were autographed the day after he completed an orbit of the Earth in 1961.”
“And this guy is Sergei Korolev, the brains behind the Soviet space programme. He died from a bungled heart surgery – they may have got to the moon first had he lived.”
“In terms of nerdy dates, facts and figures, space missions would be my chosen topic on Mastermind. I’m a bit sad like that,” he says.
Shadbolt, who now divides his time between his professorship at the University of Southampton and the ODI in London, says open and big data “is one of the transformational agents in the 21st century”. “We released our data on survival rates for cardiac surgery in the UK. When it revealed that some surgeons and some hospitals were doing much better than others, people asked questions about why this was the case and how those who were not doing so well could improve.”
Earlier this year, OpenCorporates, one of the start-ups based at the ODI, produced a visualisation of the global corporate networks of the six biggest banks in the US. The graphic plots the number of subsidiaries by country, along with the ownership structure, and is released as open data, allowing anyone to edit and correct the information it contains.
“Being able to look at the extremely complex connections that exist among global corporations is of enormous benefit to society and the economy,” says Shadbolt.
Still, he says, there is “a shocking need” to draw up regulations and social conventions for this “new world”. “While the net was still growing up it was easy to believe that it could topple everything before it; today we’re paying more attention to the big corporates and internet giants that sit on huge deposits of our data and stare back at us from the other side of the screen. Google, for example, has become a monopoly more powerful than many states.”
As for the recent controversy surrounding the Conservative party’s apparent attempt to delete a 10-year backlog of speeches from the internet, including pledges for a new kind of transparent politics David Cameron and George Osborne made when they were campaigning for election, Shadbolt says: “Was this a conspiracy or a cock up? I don’t know, but there ought to be some kind of serious introspection into how this was allowed to happen. Today, it is imperative that we hold governments to account over transparency.”
Is it especially galling considering that, last year, the ODI received £10m worth of funding from the government? “At the ODI we are scrupulously independent but, yes, this situation is deeply ironic . . . But before this the coalition government had done some very good work on releasing data.”
For a long time, in the comfortable confines of our homes, we rarely thought about data. Keeping a diary was acceptable; a spreadsheet was weird. “But today exercise, food, sleep, location, productivity and even spiritual wellbeing are being shared, measured and displayed,” says Shadbolt.
Like all geeks, he is fond of gadgets. He puts aside his unfinished coffee, and pulls out an iPhone to take a photograph of his face. Using an app called Cardio, he is able to see that his heart is beating 63 times per minute. “I like to collect data about myself, it’s all part of my life logging.”
Then he rolls up his shirt sleeve to reveal a wristband. “It measures in shocking detail my lardyness at various points in time, showing me which days I’m active and how many calories I’m burning,” he says. “Personal data, big data, open data: it’s time has come. It’s as simple as that.”
“I started going fossil hunting in Derbyshire when I was six or seven. My parents thought I was mad,” says Shadbolt. “All the limestone there is shot through with crinoids.” Displayed in a glass cabinet in one of the living areas, his collection contains a “primitive looking” 600m-year-old rock scorpion and a fossilised lobster he bought “It’s exquisite, very beautiful to look at.”
Get alerts on Big Data when a new story is published