Holographic Capture from Microsoft

There’s a whole slew of so-called “VR cameras” being announced every week now. For the most part these camera systems are really 360º camera systems which means that even though you can look all around you, your perspective is fixed to wherever the camera was placed in the scene.

For a fully compelling immersive experience we are going to need to be able to to walk around a scene and explore it from multiple perspectives.

Of course this is a MUCH harder problem to solve. How do you film video of a scene from every possible angle with both texture and depth data needed to fully simulate the objects, people and environments in a scene?

Microsoft has come up with a mind blowing system for full volumetric capture that uses a combination of over 100 RGB and IR cameras to collect image and point cloud data to capture live action in full 3D.  You can then view the scene from anywhere in the scene.

Not only are the results amazingly realistic, MS has also figured out a way to refine, compress and encode the data into and low data rate (12Mbps) MPEG format allowing it to to potentially be streamed on demand into your VR/AR expereince. (This is an impressive achievement in itself in that the initial raw data capture of these scenes probably amounts to terabytes per minute.)

This is one of the many amazing tools being created to develop live action VR content for their upcoming Hololens AR googles.

Check out the demo video from MS below or a write up on RoadtoVR for more details.