Johnny Chung Lee has been working quietly on Microsoft’s breakthrough Project Natal, which was recently demoed at E3. The vision video sells the concept:
As someone who has been working in interface and sensing technology for nearly 10 years, Lee says that this is an astonishing combination of hardware and software:
The 3D sensor itself is a pretty incredible piece of equipment providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost. Depth cameras provide you with a point cloud of the surface of objects that is fairly insensitive to various lighting conditions allowing you to do things that are simply impossible with a normal camera.But once you have the 3D information, you then have to interpret that cloud of points as “people”. This is where the researcher jaws stay dropped. The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision research. The sophistication and performance of the algorithms rival or exceed anything that I’ve seen in academic research, never mind a consumer product. At times, working on this project has felt like a miniature “Manhattan project” with developers and researchers from around the world to coming together to make this happen.
We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction.