Post a Comment On: Mayerson on Animation

I am not an expert in virtual reality and frankly haven't been following it that closely.  However, it's beginning to pop up more and more, so I've started to think about how it might affect animated storytelling.

You've probably seen the video of Glen Keane drawing in three dimensions.  I've just seen a video from Occulus Medium about sculpting in a virtual reality.  I don't doubt that video games will take advantage of VR before any other form of animated storytelling, but how will stories that are not interactive accommodate themselves to VR?

The simplest approach will probably be to let the viewer wander through a scene, looking wherever he or she wants to at any given moment.  This is the closest to what now occurs on the stage.  When watching a play, you are free to look where you wish.  There are many directorial tricks to influence where the audience is looking, but there's nothing to stop an audience member from staring at a performer's shoe for the entire play.

(Will the hardware will track head or eye positions?  We move our eyes much faster than we turn our heads.  Besides speeding up the audience experience, tracking eyes will prevent the feeling of vertigo when the entire world spins while you turn your head.  Will the viewer glide through the scene like a film camera or will the viewer have to walk or run to keep up with the action?  Both have emotional validity, but will the action scale itself to the viewer's ability to keep up with it?)

There are two issues with giving the viewer the freedom to wander through a scene that I see.  The job of director exists specifically to shape a story visually and direct the audience to what is important at any given moment.  If you give the audience the freedom to look where they please without strategies to influence where they look, you could end up with stories that are less dramatic or comic than intended, simply because the audience is not catching the important action.

The other issue, and one that will be common to all forms of VR, has to do with increased production demands.  Right now, if you cut from character A to character B, character A does not have to be animated so long as he is off-screen.  With VR giving the viewer the ability to look where he pleases, character A must always be alive in case the viewer is looking in his direction.  This will require more animation for each character in a scene and may also require more background animation (wind, water, etc.)  If the characters will truly be present in the story, and not just on some kind of breathing cycle or other nondescript movement, this will require more work and larger budgets.  It will also force animators to learn how to have characters listen, something live actors learn early on, but that animated films typically avoid except for reaction shots.

Another alternative to this would be something similar to directing live action TV dramas from the 1950s.  Imagine that every character has a virtual camera focused on her as well as a camera that covers the master shot.  The viewer could switch between cameras, choosing who to look at.  The danger here, as above, is that on a first viewing the viewer doesn't know where the story is headed. She might miss important details or emotional moments.  Will a story have to be viewed multiple times in order to fully understand its dynamics?  And once understood, will the viewer have evolved a preferred way of experiencing a particular story that will be different from other viewers' approach to the same material?  Imagine a shared VR experience where you are looking through the eyes of the person next to you, experiencing how they experience a story.  Then you switch places.

Another possibility is the subjective camera where the viewer is in the story.  This edges more into an interactive approach, where characters would address the viewer, even if the viewer's responses were prerecorded audio.  In this way, it would be possible to be in the film as whatever character you desired and see the story through their eyes.  On additional viewings, you could take the part of another character.  Would Star Wars be more or less interesting if you could take the part of any of the characters and see the film through that character's eyes?

While videogames have been around for decades, there is still a separation between games and narrative media.  With VR, will they bleed into each other more?  Besides playing a character, will the viewer have hands to interact with the environment?  And if so, will the viewer be able to influence the story based on actions taken?  A hit or a miss with a weapon could have a major impact for how a story unfolds.  With each action taken, a parallel story possibility exists, perhaps making this approach far too expensive to produce.  If it turns out "these ARE the droids we're looking for," or the trash compactor kills one or more characters or the death star doesn't blow up, you have three other movies.  Multiply that by every fork in the story and how long would it take to produce the film?

Eventually, conventions will develop.  There's no reason that red means stop and green means go except that we all agree on it.  There's no reason that a right mouse click does certain things compared to a left mouse click except that we've been trained across various software packages to accept this.  Just as there are storytelling conventions in film, such as the cut and the dissolve, VR will inevitably develop its own conventions so that audiences feel confident as to the best way to experience a story.  What those conventions will be and how long they will take to develop is unknown at this point, but that's the next frontier.  People look back in wonder at the development of animation in the 1930s and the development of cgi in the '80s and '90s.  The next explosion of creativity in the animation field is going to be the development of VR as a storytelling medium.