Microsoft’s Kinect motion controller has sparked a robotics and 3D revolution, inspiring innovations that allow vacuum cleaners to be controlled with a hand gesture and surgeons to flick through medical scans without touching them in the operating theatre.
Features announced at Microsoft’s press conference at the E3 video game trade show – such as finger-tracking and whole-body scans – already exist, albeit as hidden capabilities.
Within weeks of its launch last November, gamers, developers and scientists reverse engineered the Kinect’s capabilities and created new applications.
The Kinect has gone on to sell more than 10m units – and games for it such as Dance Central are proving popular with consumers.
But hackers have shown how the Kinect can be adapted for much more than entertainment. “The amount of innovation has exploded over the past six months,” said Josh Hyman of kinecthacks.net, a website that records new uses for the device.
These include controlling the movements of a robotic vacuum cleaner with the wave of a hand and allowing a model helicopter to sense walls, floors and objects in its path as it flies indoors. Surgeons have even used it to go through medical scans with gestures in an operating theatre, rather than having to repeat sterilisation procedures after handling them in the normal way.
While some of the robotics applications require engineering knowhow, videos and web tutorials show how the Kinect can be easily commandeered to scan and re-create hologram-style 3D objects or whole rooms on screen.
“People share, this is all open source, so there are a lot of answers to be had. It’s like magic, you put it together and get results quick,” said Mr Hyman.
The bar-shaped Kinect accessory has an infrared depth sensor included with its camera, allowing motion and gesture recognition to control a game or the Xbox’s dashboard.
PrimeSense, an Israeli company, is responsible for the key 3D technology in the device. It has allowed developers access to its OpenNI software that reduces huge amounts of raw data from the sensors down to easily manipulated skeletons of what is being recorded.
“We are enabling a new kind of digital media that will allow people to bring original ideas to market in a similar way to how the iPhone enabled apps,” said Inon Beracha, PrimeSense chief executive.
Robotics engineers have seized on the Kinect as a game-changing device in every sense.
“It really has transformed robotics,” said Ken Conley, senior software engineer at Willow Garage, a Silicon Valley robot maker whose most sophisticated models can cost $400,000. The key has been the $150 cost of the Kinect, he said, when similar sensors had cost more than $1,000 before.
Microsoft announced on Monday that the Kinect would add capacity for detailed tracking of finger movements to allow drawing with them in 3D. People-scanning would create realistic avatars and scanning of real-world objects could put them into games.
The Kinect hackers have already achieved similar results and they may be tempted to submit future ideas to Kinect Fun Labs – a community Microsoft launched, which will allow innovations to be shared with others.
“When we launched Kinect, we knew we were only scratching the surface of what we could do and the possibilities of how it could grow over time were limitless,” said Kudo Tsunoda, creative director for Kinect. “But we’ve seen so much innovation and creativity by people around the world in building things with Kinect that even we couldn’t imagine.”
Microsoft has been given credit for not encrypting the raw data from Kinect in the first place and allowing hacking to take place. The encouragement of a developer community should allow innovation at a more rapid pace.
“In the coming years, when people have time to mature these [application] libraries and come up with new ideas, and as these sensors get better and better, we’re going to see amazing things,” said Mr Conley.