Where Paul McCartney, virtual reality and a 3D camera collide

Sir Paul McCartney is sitting less than five feet from me at a grand piano, singing his 1973 hit “Live and Let Die.” When I turn my head to the right, his drummer is about six feet away. If I swivel 90 degrees, I can see the production crew hoisting cameras and running power cords behind the stage.

I’m on stage with McCartney and his band as he performs at San Francisco’s Candlestick Park. Except, I’m not actually on stage. I’ve never even been to Candlestick Park. I’m sitting in a conference room in downtown San Francisco with an Oculus Rift virtual reality headset snugly fit around my head and a pair of headphones wrapped around my ears.

The concert snippet I’m watching was filmed earlier this year using a prototype 3D camera developed by Palo Alto-based company Jaunt. The camera is a strange-looking prism about the size of a toaster that has lenses on each of its faces. With this technology, it can film in 360-degree stereoscopic 3D. The camera doesn’t just capture scenes; it captures entire surroundings.

Up until now, virtual reality has primarily been the domain of video games, with companies such as Oculus VR and Sony investing in headsets that let players feel completely immersed in video game worlds. Jaunt wants to bring the technology to cinematic experiences. Think being on set with actors as they act out a scene, standing on a cliff overlooking the ocean in a nature documentary, or being on stage with rock stars.

“We were initially inspired by nature,” says Jaunt’s co-founder and CEO, Jens Christensen. “One of our co-founders went on a trip to the forest and took photos, but when he came back he felt the pictures didn’t capture the experience of what he’d seen. So we thought, what if we could capture that? What if we could make you feel like you were there?”

Jaunt’s team of technologists started creating prototypes with filmmakers in mind, thinking of ways the camera could record footage that would make viewers feel as if they were in the video’s world. Their first prototypes were so big and clunky that the technologists couldn’t even get them out of a room. So they recorded themselves in that room playing catch with each other. The results of the initial experiment were promising. When the viewer put on the VR headset, it looked, felt and sounded as if they were in the room with Jaunt’s founders as they tossed around a ball.

According to Christensen, the demo of McCartney’s concert, which is available for download from Google Play, shows only a fraction of Jaunt’s ambitions. For example, Jaunt filmed the concert with the camera in a stationary position. It has plans to experiment with a moving camera, and giving viewers the option to change the angle from which they view a scene. The McCartney concert is also not interactive; the viewer can only watch what’s happening in the environment. Jaunt has plans to experiment with interactivity in a virtual reality movie.

While few people currently own virtual reality headsets, Alex Taussig, a member of Jaunt’s board of directors and a partner at venture capital firm Highland Capital Partners, believes virtual reality will in five to 10 years be a real platform similar to the way mobile is today.

“The idea that you can transport yourself to another place in time in the real world, experience something through someone else’s eyes -- that’s a very compelling thing,” he says. “And we don’t have to explain it to people. When you show them, they just get it instantly. If you’re a content creator, this is a new type of medium, similar to what film was when it first came out. And if you’re an innovative person, you want to be part of that. You don’t want to be on the wrong side of history.”

Jaunt is currently working with Los Angeles-based filmmaker New Deal Studios (whose list of credits include “Inception,” “Green Lantern,” “Hugo” and “Cowboys and Aliens”) on a cinematic project using its 3D cameras, and it’s also developing a horror and World War II film project. Both Taussig and Christensen acknowledge that there will initially be challenges in creating virtual reality movies because traditional filmmakers will have to rethink how they approach directing and set design. But some creators have already come up with ideas for ways to direct the viewer’s attention using sound and light, and others see a chance to create new experiences for their audience. According to Christensen, the creators he has spoken with don’t just see the challenges -- they see the opportunities.

“It’s a new medium,” Christensen says. “How often does a new medium come along, right?”

Twitter: @traceylien