by Over here in VRenetic’s production department we’ve come to find that some of the most compelling 360 content comes packaged in a single-shot experience. While the long worshiped tenets of filmmaking insist on an ample supply of inserts and close-ups, the ultra wide angle simplicity of a 360 camera’s field of view puts the viewer in control. Sitting in the director’s chair viewers can watch and listen to the world at their own pace — you can drop in on the plot lines and events that are occurring in your active surroundings by turning your head to focus on them. For this reason the kind of editing standard based around hard cuts between shots that we’re used to seeing is a bit trickier to pull off in 360. The best cuts come across as a reaction to something in the world of the 360 experience — someone throws a towel over the camera and pulls it off to reveal a new location, or a bit of dialog alerts the viewer to a presumptive change of scenery. But in an ideal virtual reality, there are no “cuts” only flows from one moment to the next.
With this in mind I ended up looking more towards theater than film in choreographing for 360. Most 360 scenes rely on all the actors being in character even when they’re not the focus of the scene, which presents its own challenges as it requires that everyone around the camera perform perfectly at the same time. This rarely happens in scripted content and the complications only multiply with more variables, actors, and choreography. It’s much like a one-take shot in film, except without the easy solution of cutting to different moments from your favorite takes.
The problems with choreographed single-shot 360 became most evident during post-production of a music video that VRenetic shot and edited for a local band. In prior music-related productions we’d relied on dance improvisation to make a seamless one-take experience, but in this music video the band wanted something more. With four choreographed performers and three band members who needed to be synced directly to the music, the challenge of the 360 one-shot reared its ugly head. The reality in this shoot was that the seven people could not all be their best at the same time for the same take — not in a shoot that was scheduled to last only half of a day anyways. The combinations and complications with seven performers multiplied so that even the best cumulative take did not do justice to the total performance.
In preparing for this shoot I knew I wanted to have each performer and band member occupy their own space in front of each of the six lenses on the Insta360 pro camera that we were to use. Intending to avoid any conflict with camera overlaps creating artifacts on the performers the video outputs from each of lenses worked on their own as super wide angle long shots of each of the performers. Furthermore for each camera I had seven takes of the song to equate to 42 different videos. In this particular instance this method of interstitching kind of came about out of necessity because of the number of lenses featured on the pro-sumer Insta360 pro. In practice the more lenses you have on your 360 camera, the less of an issue the stitching lines become. The Jaunt ONE has like 24 cameras, and each captures with plenty of overlap with its neighbors, so the stitching artifacts are less evident. But limitations breed creativity — and that’s not to say that even with your hyper-fancy camera you shouldn’t try this same technique.
What I wanted to do was to take the best part of each of the songs and somehow layer them on top of each other in a quilted combination. The solution to the problem was to find the best take on each camera and stitch the resulting six cameras into a master stitch of the best takes from each performer. In a perfect world of 360 video production you as supreme ruler can direct all of the moving objects in frame around the camera to stay within the borders of the camera lens that they started in. This is not the case in the everyday universe — and wasn’t the case in the choreography of this music video. At one point in the song the dancers each migrate from their camera across the musician next to them and then into the next camera’s sight-line. But this maneuver was not consistently executed during the same moments in the song, which prevented me from doing an across the board stitch of the six cameras from different takes.
In Adobe Premiere I’ve synchronized all of the takes from each lens
The choreography and lighting of this piece involved plentiful strobing in a windowless warehouse. In the midst of these flashes there are a few frames of complete darkness where the dancers and band members were completely obscured. Since the programmed lighting lined up consistently with the song I was able to stack up and synchronize all seven takes of each camera in six different sequences.
Cutting down to the take that I want for each section I’ve made a continuous but intercut version of the individual lenses.
I then went through the footage and cut through all of the takes during the moments of complete darkness that made sense for jump cuts during the song. Once I had all of the footage sliced up I went through each intervening section of each take on each camera, rating each section of each take on a scale of 1-10. For the sections with the dancers’ crossover between the cameras (which occur twice during the choreography) I looked for my favorite complete take, and locked in those crossover moments as instances when all six cameras had to be from the same take. Other than these I took my own route through my favorite moments of each camera, making sure only to keep the video consistently synchronized (and reflective of the song being played).
A big part of this whole process is fooling your stitching software into thinking that these exported clips are actually original video origins directly imported from your camera. In order to most effectively accomplish this for VRenetic’s Insta360 Pro was to crank out the six intercut videos with the same bitrate, file format, and codec as the originals. Some playing around with the stitching software — including modifications to some stitching lines to avoid a few artifacts that stuck around when the Jester character spent too long between frames — led to the first rendering of the interstitched product.
I’ve stitched my finished camera origins together in Autopano Video Pro 3
Though rudimentary and rough this initial render gave me an idea of the more interesting side of intercutting. The most compelling parts of the process came through visually as quick jumps by the dancers and musicians that seemed to elapse instantly during strobing scenes (where I had found the most ease in cutting between takes). Using this as the beginning concept for the effects in the video I began to do more rapid and dramatic cuts between the different takes, opting to correlate the more intense parts of the song to fast teleportation like strobing of the dancers from close to the camera to deep in the background, coupled at times with unexpected leap between different character (where there was a blonde appears a brunette, appears a man, et cetera). So while advantageously selecting and displaying the best performances I was also able to promote the transitions as their own standalone practical effects.
Some background: I was an avid photographer in high school, and in my second year of Mrs. Bowman-Smith’s photography class we were tasked with a project that involved literally knitting multiple photographs together. The process started with four photographs of the same 8in x 10in size but not always the same image. Two of these photographs you’d slice along one centimeter lines horizontally. We’d take these fine strips of photographic paper and lay them A,B,A,B… next to each other to make a double tall version of the now-distorted photograph. Using clear tape we’d tape this combination together completely. Now with the other two photographs we’d do the same except with the same thin slices laid out vertically, C,D,C,D… The effect was similar but we’d end up with a double wide version of the photograph instead. Now we’d take the ABAB and carefully slice that along 1 centimeter spaced out vertical lines so we’d get these pixel-like lines of squares of photograph, and we’d take the CDCD and slice that horizontally and get these same strips. Keeping these strips in order we’d then weave together the AB and CD photographs in an over-under weave. The result was an intricate mat of ABCD patterning — a larger than life photograph, repetitive but incorporating different moments and scenes into one canvas. I was compelled to do the same thing with other media — newspapers, homework, chip bags.
Intercutting takes this idea and subtly applies it in 360 degrees. In theory it is a knitted quilt of different moments. If these moments are close enough together the result is a somewhat consistent VR experience. But if the footage sources are varied enough the results can be shocking. Popping new worlds, artifacts, glitching, give you a reason to look around and see something impossible.
Watch the finished experience here.
By: Ethan Stockwell – Digital Video Editor at VRenetic