Tactics That Keep Film Craft At The Heart Of New Technology
In the last few years, immersive 360° video technology has taken the consumer market by storm. Everyone seems to have tried it, from filmmakers and visual artists experimenting in new art forms to the everyday consumer looking to just have some fun.
As with all new technology though, there are plenty of non-artistic examples around. With 360° technology and VR hardware getting cheaper and easier to use, the amount of videos made just for easy clicks will continue to climb.
It’s easy to lose faith in the possibility for real artistry in the early stages of a new technology. Yet, in all of the hustle and bustle, we found filmmakers out there using it in smart and innovative ways to share deep and personal stories. Though the examples of projects we give here have big budgets and post-production companies behind them, the principles of good filmmaking can be applied to anyone wanting to tell a good 360° cinematic story.
Frame Your 360° Environment With Sound
Perhaps no VR/360° immersion filmmaker is melding artistry and technology better than the Emmy award winning artist and filmmaker Lynette Wallworth.
She’s had the rare opportunity to make two immersive short films now with the backing of major companies (Dolby Atmos/Technicolor/Jaunt VR). She’s also the first ever Sundance VR artist in residence, and is confirming our suspicion that VR will only become more part of the film festival culture. But from her very first film incorporating 360° technology, Collisions, there was a clear sense she was onto something both new and rooted in cinematic tradition.
Historically, one of the greatest challenges of 360° filming is in the framing. The whole history of cinema has relied upon the concept of the stable frame. The vision of a director and the lighting choice of a DP live and die by the positioning of a particular frame in a shot. Yet, with new immersion technology, it’s often hard or impossible to know where your audience is looking at any one time.
To ground Collisions in a traditional sense of the cinematic, Wallworth looked to the professionals at Skywalker Sound to mix sound exclusively for the headset/HMD (head-mounted display) experience.
As the above video shows, sound editors and mixers relied on unprecedented technology to follow traditional principles in filmmaking. In every new shot, certain sounds would need to be manipulated (typically magnified or delayed) to cue a viewer to what Wallworth called “trigger points” and “key moments”. These were the focal parts of a particular scene or shot, such as a new character introduction.
Since Wallworth cannot necessarily control where a viewer will feel compelled to look, she relied on sound artists to frame or bracket scenes in a way that took visual-dominant concepts like foreground or rack focus and incorporated them into the world of sound. Wallworth has actually spoken about this very process…
It’s completely delightful to watch people tracking sound, and to have the joy of seeing what would previously be framing being directed by the [Dolby] Atmos.
Because most of us don’t have Dolby Atmos staff in our basement, what are some good 360° audio options? For industry-standard quality that will guarantee great mixing capabilities later, the omni-binaural microphone is the way to go. This creates 360° sound capabilities while also allowing sound channels to respond to where the viewer will be looking. For more dynamic projects, the 8ball microphone is a great product, with a body already built to mount rigs and move through cables. Companies like Sennheiser are also in the VR game, and provide a more traditional microphone design.
No matter what microphone you must use in the end, follow Wallworth’s approach to sound and treat each scene as if it’s being framed by the aural experience. Become extremely attuned to the limitations of the sounds in the space you’re filming. If possible, plan your sound set-ups right alongside your lighting setups in the pre-production process.
Make Every New Image A Dynamic One
Complaints about motion-sickness during VR experiences have been frequent enough to make many filmmakers afraid of moving their camera or making their images more dynamic.
It’s the same thing that happened with 3D too, so it’s ultimately unavoidable to make good immersive films without potentially bothering more sensitive stomachs. Therefore, it’s best to take the lead of VR director Angel Soto, and his use of movement in the highly-acclaimed 360° short, Dinner Party. Funded by the Engadget Alternate Realities grant program, Dinner Party is based on the real accounts of a UFO abduction of an interracial couple in the early 1960s. Its plotline has been the focal point leading up to all of its festival stops this year, but what can’t be under-sold is the dynamic filmmaking behind the storytelling.
Relying on the likes of Technicolor and the Sundance New Frontiers Lab to fund motion-capture technology and VR-specific camera rigs, Soto and crew were able to create a very balletic cinematic experience in a format more know for stiff, stationary camerawork.
In the video above, Soto cites the unbroken takes in Goodfellas and Birdman as influences on Dinner Party’s camera choices. This required creating an elaborate soundstage and mounting the 360° camera onto a Technocrane for limitless movement potential. The short’s cinematographer, Sam Gezari, had this to say on blending new technology with tried-and-true cinematic filmmaking…
Often we used motion capture systems or robotic systems of some sort, but I really wanted to use traditional filmmaking tools, and Technocrane in particular, because that gives me a really wide range of movement — possibility of freedom I wouldn’t have otherwise.
How exactly did Gezari’s team capture the fluidity of a Scorcese film into a 360° perspective? They relied on the Z CAM V1 Pro to deliver industry-standard stereoscopic 360° recording. Several great alternatives (with consumer-friendly prices) exist out there: most popularly the GoPro Fusion or the Samsung Gear 360. Yet we can’t not mention another professional-level camera series, this time from Jaunt. Used on Lynette Wallworth’s Collisions, the versatility and quality of images are clearly on display with each and every shot. The Jaunt ONE, for example, has over 10 stops of dynamic range and supports up to 120 frames per second capture.
Wallworth’s most recent immersive short, Awavena, has been a serious hit at every film festival it’s premiered. While the story of the first woman shaman of the Yawanawa is a gripping one, the camera and lighting choices only enhance this once-in-a-lifetime experience.
In Awavena, the camera is placed on a boat that takes the viewer down river. There are also great night shots, lit well enough to create cinematic contrast in a 360° environment — something that’s very hard to do, even on a traditional screen.
Keep Every Decision Rooted In The Story
Whether you’re shooting on VR or Super 8 film, this principle is a guiding one for the cinematic medium.
The short documentary on the making of Awevena shows us that every groundbreaking piece of technology was only used to better serve the story told. Because much of the film deals with the experience of hallucinogenics in the Amazon, visual effects artists relied on both point cloud scans of the actual rain forest and photogrammetry data to create an abstracted, wholly subjective vision that still felt rooted in reality.
The post-production crew also integrated real footage of fluorescent insects in the Amazonian night with confocal imagery techniques. The amount of time and labor put into this project seems to rival a major blockbuster, but the major sentiment one gets from everyone involved is that what they were doing, no matter how cool or revolutionary, only worked because it served the main character’s perspective and journey.
Similarly, Dinner Party’s second half relies on heavy effects work and CGI animation to tell its fractured perspectives on the actual alien abduction. Yet, the experience itself remains rooted to the story’s very human elements.
For example, whenever the film is exhibited, viewers are brought into a model room recreated to look like the room in the movie. Though Dinner Party can’t be interacted with while it’s playing, director Angel Soto certainly understands the power of the real and tangible . This decision too, certainly an expensive and time-consuming one, is rooted in the film’s themes of the gray area between reality and fiction.
So even if you don’t have the large budget and resources of directors Soto or Wallworth, realize that the ultimate success of these films hinge just as much on the priceless elements of story and character as they do the ever-evolving technologies used to bring them to life.
