On an auspicious Friday night in Asheville, North Carolina on June 22, a group of artists, writers, filmmakers, playwrights, actors, and musicians gathered to kick off the start of an immersive museum titled ZED.
What does the name ZED mean? Nothing and everything. It was the word that was in black graffiti on the 10,000 square foot warehouse that the ZED team is looking to use as its first home and it stuck. Because any other name would seem techno-babble.
Although the first permanent immersive museum in North Carolina, this is not the first of its kind.
There are soon to be three in the United States under the name Meow Wolf, with the first one being in Santa Fe, New Mexico, and soon to be opened in Denver, Colorado and Las Vegas, Nevada. Team Lab is opening one in Tokyo, Japan. In New York City, the American Museum of Natural History is opening an immersive experience called Our Senses.
Note that immersive doesn't strictly mean virtual reality or augmented reality or even technology of any sort. It can simply be a place where you submerge into a physical piece of artwork.
However, immersive is becoming more than a trend in the art world as it’s a total body experience – body, mind, and mostly importantly, soul – in contrast to a passive viewing and a loose philosophical interpretation.
Maybe that's why digital immersive, such as virtual reality and augmented reality, hasn't taken off for films, series, and video games as it was expected to do.
The beholder hasn't found the beauty in immersive yet.
Further, there are several issues such as motion sickness associated with virtual reality viewing for extended periods of time. Secondly, there is the cost for the viewing technology. And finally, there has not been a killer piece of content that has created a groundswell of adoption.
Also, it requires people to wean themselves from their mobile phone addiction of checking their Twitter or Facebook every 30 seconds.
The opposite of Twitter or Facebook is virtual reality as it's a solo sport.
Virtual reality storytelling has been more voyeurism than active involvement. The disconnect in one's brain to have all senses activated but merely watch or stand on the sidelines prevents the affinity between immersive content and the viewer.
Previously, I have talked about the other issues that immersive storytelling poses.
But why virtual reality is floundering has a more fundamental issue – how can you sell something in it?
Adoption comes from the ability to sell products via advertising. If you can be advertised to, the technology proliferates and the storytelling matures.
So the race has begun to use immersive as a product-selling mechanism. New York City ad agencies such as AdVRtze and Colorado-based Hypercube are looking to find ways to demarcate points of gazing to gauge interest.
Companies such as Admix are using the immersive lingering as a way to do pop ups in the story to allow you to purchase or acquire coupons for future use. More importantly, it allows both in video games and organic filmmaking to use green-screened products to be used as real-time bidding objects.
For example, in the content, perhaps the person in front of you is drinking a cup of coffee. However, during filming the coffee cup is green to be replaced digitally later by whoever bids highest for the object during the time of viewing – much how banner ads act now. Or if a filmmaker is creating a virtual reality film, it will make as many objects as possible blue- or green-colored to be replaced by the company who pays the most for the branding or the full production.
Also note that the credits of immersive films will be the highest point of purchase following what Marvel did with its Avengers series – that people wait until after the credits to see a secret scene that tells more story or preludes to the next episode. Instead with immersive, the end credits, with the help of companies such as TheTake, all the products worn, used, and discussed will be shown as purchasable. However, a filter will drill down based on tracking your eyes such as the FOVE VR headset.
So the key is to stay until the end of the content to get discounts on the things you want to buy.
Eyes are not the only ways to engage your immersive senses.
Sound will also be used to introduce product placements with the help of companies such as Sonic Union who specializes in Spatial Audio Mix.
Speaker company Dutch & Dutch just patented a new machine learning algorithm to help its speakers to create an immersive experience, and its new Wavvy wand records sound experiences that mimic the real world with very little studio post production.
The other issue is the massive amounts of infrastructure required to stream 1200+ media files that make up a common 8K immersive experience to represent every direction – up, down, left, right, and behind you.
This is why Colorado company Hypercube teamed up with Nokia following the release of the OZO camera, to build a standard for all immersive infrastructure that allows 8K and even 16K experiences to not be forced to render within an app but can be brought by a single URL. It can even detect location to do the best load balancing and offers the algorithm in a pre-packaged virtual environment that be deployed in a public or private cloud. Finally, it allows the viewer to be returned to a website or another experience when the content completes. The content can be controlled by access controls, number of allowed views, and geographic location.
The key to this infrastructure is repeatability with the same quality of service. In fact, the more you view, the clearer the experience gets. Very similar to when you begin viewing Netflix and in the beginning of watching content, the picture looks grainy but quickly becomes very crisp and distinct.
But the most important repeat is the storytelling itself.
If this scene could be recreated using immersive filmmaking – this would be a perfect scene. It allows for cross talking – multiple angles, multiple conversations, and multi-engagement for the viewer who's point of view could easily be a) the waitress b) or one of the guns for hire.
The best aspect is the ability to draw viewers back into the scene over and over to get a new experience or learn something new. That's why the key to immersive storytelling will be theater productions or theater in the round actors – to allow multiple scenes layer with a focus being put on the main action but allowing the viewer to tip their ear into a direction to get other sounds and conversations.
From a selling point, you can watch the scene over and over and a new product could be injected from the coffee, the cigarettes, to the suit and ties which are available to be bid on as purchasable.
The production team could then make a choice on how to allow advertising – either in story purchases where pop ups would show such as Admix but pause the narrative, or end credit purchases such as TheTake which allows the entire story to be told first.
So as Asheville begins the process to launch its first immersive museum, ZED, it will begin to let artists of all types experiment with immersive worlds of touch, sound, and visual.
In essence, immersive requires all of your imagination.
Limiting ZED to only one sensory experience will easily end Asheville's ZED before it begins. Or to quote Bruce Willis in the film "Pulp Fiction" by Quentin Tarantino, "ZED's dead."