NVIDIA Untethers Virtual Camera Technology in SIGGRAPH Demo

by Wil Braithwaite

Imagine a filmmaker, tablet in hand, walking through a live-action set and recording actors as they perform, while simultaneously seeing fluid updates of computer-generated elements within the shot.

Imagine no more. At SIGGRAPH this week, we’re demonstrating virtual camera technology that does just that.

Virtual camera technology is powerful stuff, but until recently was primarily confined to stage environments equipped with tracking markers and motion-capture cameras.

Our technology uncouples the virtual camera from the controlled environment of a set and lets directors frame scenes from any angle. By putting the camera quite literally in the hands of filmmakers, they have the freedom to tell stories in new ways, combining computer-generated images and a live-action set.

Virtual Camera demo

To make virtual production more accessible, we’ve worked with Google to use the robot-vision capabilities of its NVIDIA Tegra K1-powered Project Tango tablet, as well as a host of other graphics computing power of the GPU.

The technology works by transferring the position of the “camera” (in the case of our SIGGRAPH demo, the Project Tango tablet) to a digital content creation application for rendering.

Computer-generated elements are rendered by NVIDIA Quadro GPUs, and then streamed back to the tablet using NVIDIA GRID technology. The Tegra K1 then mixes live action with the computer-generated content.

So whether on-set in a controlled stage environment, or off-set in real world, live-action shoot scenarios, filmmakers can get the shot right the first time.

Broadcast live streaming video on Ustream