Lightning in a bottle

Good ideas thrive in closely-packed spaces - or that’s what Steven Johnson, science writer and entrepreneur, believes. Tim Harford says, ‘Eureka!’

In late 2005, I was working on the closing chapters of a book about the 1854 cholera outbreak in London and the brilliant map generated by the physician John Snow that ultimately solved the mystery of cholera’s waterborne nature. I was writing about London from my home on the south side of Brooklyn’s Park Slope, part of what is often called “Brownstone Brooklyn”, a hub of rapidly gentrifying New York City neighbourhoods where many writers, professional and aspiring, had settled. Some of those writers had begun blogging about their communities, writing about local real-estate gossip and school-board elections and crime data, in blogs such as Brownstoner and Only The Blog Knows Brooklyn.

I was fascinated by the way those local bloggers were covering their communities: they were writing on a block-by-block scale that closely matched the experience of actually living in a city neighbourhood, with a “granularity” – that fine level of detail you get when you zoom in on a map – that a big city newspaper (with its rather wider map) could never hope to achieve. I grew so fascinated, in fact, that they became my primary procrastination device as I was writing my book. Each morning, before crafting some new sentences about the neighbourhoods of Victorian London, I would linger on the latest blog posts about my own.

Somehow, in the collision between these two different spaces, a new idea began to take shape in my mind, prodded along by the news that Google had recently opened up its mapping software to outside developers. Perhaps, I thought, someone could create a site that would take all these different local voices and literally put them on a map, and show you all the conversations that were happening about places on your block, or in your postcode. A few months after the idea first occurred to me, I mentioned it to two friends – a film producer and a screenwriter – and they liked it enough to offer some seed capital to build a prototype. I found a few talented collaborators who turned my hunch into something much more substantial, and, by the autumn of 2006, we had a start-up with employees and a beta version of our site online at Four years later, more than 100 news organisations, including CNN, rely on our platform for neighbourhood-level news.

My experience in starting was hardly unusual. It turned out that all around me in New York City, new web start-ups were flourishing. Just a month ago, a study by market research firm CB Insights declared that New York has overtaken the original East Coast technology hub, Boston, in the amount of venture capital and early-stage investment flowing into internet companies, leaving it second only to Silicon Valley in the US rankings.

Some of the most emulated web companies, such as the gossip site Gawker; location-based social media site Foursquare; the personal health network Everyday Health; and Etsy, a marketplace for handcrafted goods, have originated in Manhattan and Brooklyn over the past five years. For the first time in nearly a half century, since the heyday of Bell Labs (the research offshoot of AT&T), New York has become a world leader in high-tech innovation.

The musician and artist Brian Eno coined the odd but apt word “scenius” to describe the unusual pockets of group creativity and invention that emerge in certain intellectual or artistic scenes: philosophers in 18th-century Scotland; Parisian artists and intellectuals in the 1920s. In Eno’s words, scenius is “the communal form of the concept of the genius.” New York hasn’t yet reached those heights in terms of internet innovation, but clearly something powerful has happened. There is genuine digital-age scenius on its streets. This is good news for my city, of course, but it’s also an important case study for any city that wishes to encourage innovative business. How did New York pull it off?

There are no easy answers. Kevin Kelly, co-founder of Wired magazine and author of What Technology Wants (2010), writes in the book: “The serendipitous ingredients for scenius are hard to control. They depend on the presence of the right early pioneers. A place that is open, but not too open. A buffer that is tolerant of outlaws. And some flash of excitement to kick off the virtuous circle. You just can’t order this.”

And yet, even if scenius is lightning in a bottle, there are surely some practices that make you more likely to capture the lightning when it does strike. In an age of public sector austerity on both sides of the Atlantic the good news is that most of them don’t involve massive top-down government spending. (Although, in New York’s case, it does help to have a tech-geek entrepreneur as mayor, in the form of billionaire Michael Bloomberg.)

A vibrant start-up culture requires a healthy community of venture capital firms willing to back risky ideas. New York has a number of gifted venture and angel investors, led by Fred Wilson and Brad Burnham at Union Square Ventures, which supports a number of the city’s web start-ups including But investors need ideas perhaps more than ideas need investors, particularly in an age when starting a web business is amazingly cheap. So the real question is: how did New York find itself generating so many interesting ideas?

Scenius may fuel commercial ventures, but its roots often lie outside the marketplace. One secret to New York’s technological success lies in the Interactive Telecommunications Program (ITP), a two-year graduate course at New York University. In spite of its focus on technology, the ITP is nonetheless based in the Tisch School of the Arts, and its official description emphasises an “imaginative use of communications technologies.”

Founded by the visionary scholar Red Burns, who created the programme in the late 1970s, the faculty includes high-profile new media guru Clay Shirky. The author of Here Comes Everybody (2008) – a book that outlines how technology allows us to organise without the need for formal organisations – is also mentor to countless aspiring web entrepreneurs. ITP graduates have gone on to found or work for hundreds of startups in the area. Dennis Crowley, Foursquare’s founder, co-created the first draft of the location-based social network – a service called Dodgeball, eventually sold to Google – while he was still a student at ITP in 2000. My two early collaborators on, John Geraci and Cory Forsyth, had both been at ITP.

The physical density of the city also encourages innovation. Many start-ups, both now and during the first, late-1990s internet boom, share offices. This creates informal networks of influence, where ideas can pass from one company to the other over casual conversation at the espresso machine or water cooler. When we started, we shared a Brooklyn office with a documentary film company for its first year of existence. Today, our much larger office in Manhattan also houses three other smaller start-ups working on unrelated projects. By crowding together, we increase the likelihood of interesting ideas or talents crossing the companies’ borders. The proximity also helps to counter the natural volatility of start-ups: in’s early days, we “borrowed” a few talented employees from the documentary film company, which was temporarily downsizing. When the projects picked up again, some of those employees moved back. Others had found a new calling in the web world and stayed with us.

Economists have a telling phrase for the kind of sharing that happens in these densely populated environments: “information spillover.” When you share a civic culture with millions of people, good ideas have a tendency to flow from mind to mind, even when their creators try to keep them secret.

All of these spaces – the graduate schools, the co-working offices, the media environments – exhibit the final trait that has been key to New York’s technological success: its diversity. A number of studies have established an essential connection between diversity and innovation. One such study, by the Stanford Business School professor Martin Ruef, interviewed 766 graduates of the school who had gone on to have entrepreneurial careers. Ruef was interested in the diversity of professions and disciplines, not of race or sexual orientation. He created an elaborate system for scoring innovation based on a combination of factors: the introduction of new products, say, or the filing of trademarks and patents. Then he tracked each graduate’s social network – not just the number of acquaintances but the kind of acquaintances they had. Some graduates had large social networks that were clustered within their organisation; others had small insular groups dominated by friends and family. Some had wide-ranging connections outside their inner circle of friends and colleagues.

Ruef discovered that the most creative individuals consistently had broad social networks that extended outside their organisation and involved people from various fields of expertise. In groups united by shared values and long-term familiarity, conformity and convention tended to dampen any potential creative sparks. The limited reach of the network meant that concepts from the outside rarely entered the entrepreneur’s consciousness. But the entrepreneurs who built bridges outside their “islands,” as Ruef called them, were able to borrow or co-opt new ideas from these external environments.

As a diverse city that supports countless industries and maverick interests, New York excels at creating those eclectic networks. Subcultures and small businesses generate ideas and skills that inevitably diffuse through society, influencing other groups. As the sociologist Claude Fischer put it in an influential essay on subcultures published in 1975, “The larger the town, the more likely it is to contain, in meaningful numbers and unity, drug addicts, radicals, intellectuals, ‘swingers’, health-food faddists, or whatever; and the more likely they are to influence (as well as offend) the conventional center of the society.”

Those unusual influences leak out into the business world, and shape the ideas – and the personnel – of startups. The same pattern can be found in the last great flowering of high-tech scenius in Silicon Valley, which was shaped as much by the counterculture that thrived in the San Francisco Bay Area as it was by the engineering prowess of Stanford University.

That diversity shows in the kinds of technology companies New York has produced. They are not just tech-for-tech’s-sake projects. Instead, they use software to enhance other passions: parenting, crafts, gossip, gameplay, health, and so on. They use technology in clever ways, but they are fundamentally about something else. I think that quality derives from their metropolitan roots, from the scenius of a big city. It is no accident that the slogan of Meetup, the éminence grise of New York start-ups, which allows groups of people with shared interests to organise face-to-face meetings, is “using the Internet to get off the Internet”. You can’t stay at home, staring at a screen all night, when there’s so much happening on your doorstep.

We hear a great deal about the social virtues of diversity and multiculturalism; we’re reminded constantly that we’ll be better and more tolerant human beings if we open ourselves up to different perspectives. But approaching the virtues of diversity from this perspective of its impact on business innovation suggests that exposing yourself to a wide range of perspectives and fields of expertise, and creating environments where those different perspectives can clash and share resources, creates as much economic value as it does social or aesthetic value.

This is related to the essential argument that sociologist Richard Florida makes for the economic importance of cultural creatives, most notably in his 2002 book, The Rise Of The Creative Class. Vibrant music scenes or poetry salons doesn’t just make your city more colourful and quirky; they create economic capital as well as cultural capital, precisely because the diversity of views and expressions shapes the minds of entrepreneurs and investors. Good ideas need marketplaces and investors to support their growth into mature businesses, but they also need environments that help trigger those original ideas in the first place.

Living in a society where we encounter different backgrounds and professions in our daily routines makes for a more tolerant society; it also makes us smarter, more original in the ideas we have – and in the companies we create.

‘Where Good Ideas Come From: The Natural History of Innovation’ (Allen Lane, £20), by Steven Johnson, was published earlier this month. Johnson will be giving a free public lecture at the LSE on Tuesday November 2 at 6.30pm. Details at


Tim Harford on Steven Johnson: How can you inspire that elusive ‘eureka’ moment?

In 1961, a young scientist named Mario Capecchi sought the advice of Harvard University’s James Watson, the co-discoverer of DNA. Capecchi was excited by the new field of molecular biology and wanted to seek Watson’s view about where he should study it.

“Here,” came the reply. “You would be f**king crazy to go anywhere else.” As Capecchi recalls: “The simplicity of the message was very persuasive.”

Steven Johnson’s latest work Where Good Ideas Come From offers the same message, and it is equally persuasive: if you want to develop new ideas, get yourself right at the heart of the action.

Think of Athens in the fifth century BC, Vienna in the 19th century, or Silicon Valley since the 1950s – ideas breed ideas. This particular view has been explored in great detail by such writers Jane Jacobs and Peter Hall, and more recently by Richard Florida and Matt Ridley.

Yet Johnson now offers something new: a discursive style that is the very opposite of James Watson’s bluntness. No point is too minor to be illustrated with a flourish.

My favourite discovery was that many enlightenment-era thinkers kept what they called a “commonplace” book in which to transcribe favourite quotations and ideas. Some of these books were laboriously indexed to allow different concepts to bleed into each other.

But there is another side to any good story. Mario Capecchi, for example, eventually rejected James Watson’s advice and moved to the University of Utah – as far from the buzz of molecular biology as one can imagine. Capecchi had decided that Harvard was too focused on short-term, shallow results.

His strategy was the same as Lockheed’s “skunk works”, which developed the U2 and the stealth aircraft; or Scaled Composites, the company that designed the first private spaceship in the desert town of Mojave. Capecchi deliberately sought a measure of isolation while he worked on breakthroughs in gene therapy. Three years ago, he received his reward: the Nobel prize in medicine.

Johnson is right to emphasise that ideas emerge from a tangle of other ideas – but once they have emerged and need somewhere to develop, the story becomes messier. I’ll be discussing this question, and others, with Steven Johnson in an podcast on Monday.

Tim Harford is the FT’s Undercover Economist

Copyright The Financial Times Limited 2016. All rights reserved. You may share using our article tools. Please don't cut articles from and redistribute by email or post to the web.

More on this topic

Suggestions below based on New York

Twelve days in west Africa

Exhilaration, frustration and promise: Lionel Barber’s personal journey from Nigeria to Ghana allows him a glimpse of the region’s future