Categories

VR Storytelling: A Tale of Two Strategies

VR Storytelling: A Tale of Two Strategies

Hello! Dooley Murphy, here; PhD candidate applying cognitive theory to VR storytelling.

Current-gen systems have been available for ~3 years now. In that time, what general strategies of VR storytelling have emerged, and how can we understand them? More importantly, what can we learn from their successes and inherent limitations?

In the above video essay (dur. 18:43), I identify and discuss two main strategies of VR storytelling. But before I reiterate said strategies and build upon what’s covered in the video, I think it’s worth outlining a case for why we might even entertain such a simple, bipartite analysis. After all, one influential article from 2015 identifies exactly four serviceable, seemingly mutually exclusive modes of VR storytelling.

The article, “Redefining the Axiom of Story,” takes a diegesis-centred view of the nexus of interaction and narration, asking whether you (or your character) are part of the virtual world, and whether or not you have influence. It’s a brilliantly forward-thinking article. However, a minor problem with its compartmental “existence × influence” approach is that not all VR stories are explicit about whether you’re really meant to “exist” in the story-world; some are ambiguous, some hybrid, and some seemingly self-contradictory. Baobab Studios' Crow: The Legend, Oculus Story Studios' Henry, and Google Spotlight Stories' Piggy are all examples of well-known experiences that present difficulties for this way of carving things up.

Other articles point to as many as 27 different techniques seen in various VR storytelling experiences. This taxonomic approach provides a fascinating overview of what’s being attempted, but seems to gloss over issues of comfort, quality, and medium suitability. Also, the problem with attempting to exhaustively classify such a nascent medium is that outliers and anomalies continually emerge, forcing the invention of new categories, or the endless modification of existing ones.

To keep it simple, I work with a binary, input-centred perspective: VR storytelling can employ interaction in a manner that is incidental, or in a manner that is instrumental. Per this view, there’s no in-between, because a given VR experience will either demand input at some point, or it won’t: Either it'll idle until you do something, or it'll carry on regardless (how rude!).

John Legend as  Crow , photographed at the exact moment he realises the limitations of cinematic narration.

John Legend as Crow, photographed at the exact moment he realises the limitations of cinematic narration.

Incidental interaction covers experiences in which no input is required to progress the narrative’s events. At all. (This obviously excludes Start/Pause menus.) Incidentally interactive experiences are therefore typically time-bound (that is, uninterruptible) and are often dubbed “passive”. But incidental interaction doesn’t refer solely to experiences that resemble “passive,” non-participatory 360° video: Sometimes you’re given hands, but don’t need to use them. The aforementioned Crow, for instance, gives you hands and a role to play, and appears to demand your active participation. But ultimately, it’ll crack on with or without you. You can toss your motion controllers aside and the experience will proceed with only cosmetic differences.

Since head tracking is a form of input, it can also be considered a potential conduit of interactivity, albeit one which is not frequently used to affect events within the story-world (at least beyond mobile VR, in 2019). As I state in the video, in experiences like The Bond, Allumette, Henry, Pearl, and Dear Angelica, “the only interactivity comes from the incidental head movements that you use to survey the scene and track the action.” Thus, most (but not all) experiences that do not support hand presence end up being incidentally interactive. In turn, most incidentally interactive experiences—wittingly or otherwise, to a greater or lesser extent—rely on the language of cinema to narrate their stories. This, I argue in the video, is sub-optimal.

Though I am broadly critical of incidental interaction, I am not suggesting that these experiences are always automatically sub-par. Crow is fantastic, as are others mentioned, and incidental interaction is an economically sensible approach for studios such as Baobab, who may wish to release 360° and/or flat-screen versions of their VR productions in order to boost profits and maximise backers’ ROIs. Moreover, consumers know and enjoy Pixar and DreamWorks-style animations, so why not stick with their tried-and-tested storytelling paradigm? Interaction can get away with only being superficial if participants are rapt with characters, right?

Let’s overlook my thinly-veiled dissatisfaction with capitalism’s apparent disinclination to foster risky, high-budget, radical innovation and instead talk about the second category: Instrumental interaction.

Experiences are instrumentally interactive when your participation is absolutely required in order for events to progress. Story-driven games like Wilson's Heart and Lone Echo fit neatly into this category. However, I think discussion can be better focused if we look to experiences that abstain from puzzle-like situations, intentional challenge, and/or fail-states.

Accounting  (pictured) and  Virtual Virtual Reality  both make ample use of an object with which the participant will already have IRL experience. It doesn’t take a genius to figure out what to do with this thing!

Accounting (pictured) and Virtual Virtual Reality both make ample use of an object with which the participant will already have IRL experience. It doesn’t take a genius to figure out what to do with this thing!

Accounting/Accounting+, Virtual Virtual Reality, Museum of Symmetry, A Short History of the Gaze, and The Lab's Robot Repair are prime examples of narrative experiences employing instrumental interaction. They reject mainstream expectations of games and gaming mainly because they usher or draw you through rather than putting up resistance. Neither dexterity nor problem-solving prowess are required here. Much of the joy of these experiences resides in simple, instantly-intelligible interactions that nevertheless manage to surprise and delight. They thrust you into unfamiliar, often bizarre situations, and scaffold your performance with a thoughtful blend of simulated social presence (i.e., characters telling you what to do, either subtly or hilariously unsubtly) and UX-driven object and environment design.

In these experiences, each aspect of the environment you interact with is garbed in contextually-relevant signifiers and affordances, and interactions deliver timely, satisfying feedback that doubles as narration; the communication of narratively-relevant information. If the terms “signifiers,” “affordances,” and “feedback” are familiar to you, then congratulations—perhaps you already look at interactive VR storytelling through the eyes of an interaction designer! If these terms don’t ring any bells, then fear not—in Part II of the video, I recap and apply key points from Don Norman's famous UX design bible, The Design of Everyday Things.

In short, the argument I’m developing is that where incidentally interactive experiences trade in something resembling camerawork and “editing,” instrumentally interactive experiences blaze a trail in pursuit of medium-specificity by turning instead to interaction and UX design for structural and narrational inspiration. When principles of interaction design are applied correctly to VR storytelling, an experience with five; ten; twenty interactions per minute can feel every bit as seamless and fluid as a time-bound (“cinematic”) experience. How they achieve this, I believe, is in part by doing away with classical conceptions of story and plot. Accounting and Virtual Virtual Reality concern themselves not a jot with how their stories read on paper. (FYI, it’d be second-person, like: “It is your first day at the office and you blah blah blah… Will you be able to survive/uncover the truth?” etc.)

To heck with  synopsisable  plots!  Museum of Symmetry  (pictured) and other similar experiences may seem pacy, surreal, and hard to pitch to backers, but there’re still underpinned by a cohesive causal logic (i.e.,  your  actions). And in a sense, isn’t that what plot is all about?

To heck with synopsisable plots! Museum of Symmetry (pictured) and other similar experiences may seem pacy, surreal, and hard to pitch to backers, but there’re still underpinned by a cohesive causal logic (i.e., your actions). And in a sense, isn’t that what plot is all about?

To coin an adjective: Forget about making your VR story synopsisable. It shouldn’t matter whether the prospective participant can discern narrative structure from reading a blurb: VR storytelling is not about structure, it’s about process; it’s about your moment-to-moment experience, which isn’t always fully comprehensible by people other than yourself… Or even yourself. The most interesting VR experiences I’ve had are ones in which minor but nevertheless unexpected twists are thrown at you constantly. You’re always on your toes; always processing and responding to something unanticipated, even when it’s as seemingly menial as accountants lying to you about where they put your office supplies. In these borderline-frenetic situations, it’s hard to find time to reflect until after you remove the HMD. Does the oddly compelling nature of scenarios such as these translate well onto flat screens? No. Should their creators care? No—it’s not like film, where you can detach and coolly contemplate characters’ relations and situations, gaining logical and affective distance while considering what would be their best course of action.

If you’ve not tried the kind of instrumentally interactive experiences I’m referring to, then my descriptions might sound a little alien; even a little overwhelming. As I said; they certainly don’t feel the same on-screen as they do in VR. But nevertheless, in my opinion, the sensation and psychological frame produced by constant goal-directed thoughts and actions is an incredibly powerful one that can easily be yolked by narration in VR storytelling, and shrewdly leveraged in bursts of up to 25 minutes or so. We might even call it immersion.

@DooleyMurphy

Launch Post: VR Rhythm Games: A Design Case Study

Launch Post: VR Rhythm Games: A Design Case Study