Rapid results without a rugby scrum

Scrums are big news at WildCard Systems – it has more than 35 of them in progress – but the Florida-based specialist in stored value cards has not gone rugby mad. Nor has Yahoo, the internet services company, which is running 15. Nor even conchango, an IT consultancy based in the UK, the home of the sport.

None of these three companies is looking for staff with legs like tree trunks, willing to crunch shoulder against shoulder in a melee of collapsing limbs and mud. Instead, they have all adopted Scrum, a process for developing software that breaks a project down into small chunks, each of which produces a tangible result, to be carried out by self-managing teams (see panel below).

Traditional approaches to software development take too long to produce results, and by the time they are delivered, the customer’s priorities and business conditions may have changed, says Ken Schwaber, a US software developer and industry consultant who co-developed Scrum in the early 1990s with fellow software expert Jeff Sutherland.

Worse still, people involved in the process hate coming to work every day and sitting in cubicles to be given work and more work, he says.

Scrum is different, he says, as it “takes people out of cubicles and puts them in a room with whiteboards. They have to talk about a problem and address it as a team. If you come into a room where people are using Scrum, the key indicator is the noise level, because people are talking. If it’s dead quiet, they’re not using Scrum.

“The customers like it because they say ‘Holy bananas, this kind of software development finally delivers results, and in one month’.”

It is only recently that the debate over methodologies has had much relevance beyond the arcane, and often introverted world of software development. Rather like Linux and the open source software movement, Scrum became popular largely by word of mouth among software developers, gradually developing a following of admirers.

Now it is being targeted more systematically at chief executives of large companies that might benefit from using it. “We aim at CEOs because CIOs [chief information officers] are part of the problem, not the solution,” says Mr Schwaber. “We are trying to directly connect the software developers with the customer, and CIOs are not too thrilled about being cut out of the action.”

This initiative coincides with a growing feeling among some companies that traditional methods of software development are holding them back. “We felt the old methodologies had run their course and were not able to scale for us,” says Tim Dorsey, senior vice-president of performance improvement at WildCard.

The company used the classic “waterfall” method of software development, where groups of people work on separate stages of several projects simultaneously, handing off to the next group in sequence over a period that could typically last 12 months. The methodology assumes that nothing has changed over that time, says Mr Dorsey. And with a lot of people working on different projects, “if someone drops a ball, it can affect five or six other projects.”

With Scrum’s dedicated teams, that issue does not arise. Also, he says: “With the waterfall methodology you are developing software for a client. With Scrum you are developing it with a client. It’s 100 per cent transparent all the way.”

At Yahoo, Pete Deemer, vice-president of product development, is less critical of the traditional waterfall approaches – it can be cumbersome but also pretty quick, he says, crediting it with helping the company launch a range of innovative products. The company started using Scrum in February, but only as an alternative. “We’re always looking for new ways of doing things,” says Mr Deemer “and we’re trying to figure out where it works best.”

Mr Schwaber says 2,000 people are trained to run the Scrum process – so-called scrum-masters – but he has no idea how widespread its use is, and he is determined not to turn it into a product. “It’s free,” he says. “Once you commercialise it, all you think about is making money from it.”

Instead, he spends much of his time helping companies implement Scrum – an important task because the process is hard to use, as Mr Schwaber is the first to admit. “We expect only about one-third of companies that try to use it will succeed,” he says. “Those companies that do succeed with it are often those for whom technology is their lifeblood, and if they don’t succeed with it, they risk going out of business.”

One of the main challenges for companies adopting Scrum is how to deal with the huge cultural change involved in moving to an approach based on small teams, and the costs associated with that. “It’s not cheap, you’ve got to change and retool the culture,” says Mr Dorsey at WildCard. “The training costs are significant.”

On top of this, companies need “a lot of really good leadership skills,” according to Bob Schatz, chief development officer at Solstice Software, which makes software for enterprises to test their integration infrastructure. “Many people have a lot invested in the old way of doing things, and it becomes more about ego, but you have to drop that.”

How Scrum works

Scrum is a key element in the so-called Agile process revolution for software development, which has its own manifesto and non-profit supporting organisation, the AgileAlliance, of which Ken Schwaber is founder and director.

The main principles are daily collaboration between business people and developers throughout the project, frequent delivery of working software and openness to frequent change of customers’ requirements.

Scrum does this by creating a “product backlog” - a compilation of all the features the program needs, prioritised by the “product owner,” who represents the interests of everyone with a stake in the project and its results. The highest-priority items are taken for completion in a “sprint”. These normally last a month and end with a demonstration of results, after which another Sprint begins.

Every day of the Scrum begins with a 15-minute meeting where the scrum-master asks the same three questions of the team: what did you do since the last meeting; what are you doing until the next one; and what prevented you from doing more work.

The process requires plenty of discipline, says Bob Schatz at Solstice Software. “People have said it’s a wild approach but it’s one of the most disciplined things I’ve ever done. You ask the same questions at the daily meeting and some people will say ‘Enough already’ but you have to do it. It’s an intense 15 minutes that helps the team achieve its goals.”

Pete Deemer at Yahoo! worried initially that the pressure on staff to perform might be lifted, because a manager prioritises the work but the Scrum team commits to what it can deliver by a certain date. His worries were misplaced, however: “when the team make the commitment themselves [rather than having it imposed on them], they develop a deeper sense of focus to deliver well, and on time,” he says.

The self-managing aspect of Scrum also obliges team members to learn how to manage their own time, says Ian Shimmings at conchango, one of the few UK consultancies that has used Scrum and the Agile approach to developing software. “You can’t just think about writing the code,” he says.

Mr Schwaber departs from his sporting metaphors only to distinguish the Scrum participants or “pigs” – product owner, scrum-master and team – from interested observers or “chickens.” This comes from the old joke about the pig and chicken that discussed opening a restaurant called “Ham and Eggs.” The pig, which didn’t think much of the idea, would have been committed, while the chicken would merely have been involved.