The Wild West of Virtual Reality Design

My fascination with VR design began sometime last year. I started researching virtual reality and even assembled my own VR setup — not only to experience VR, but try designing and developing for it.

That's when I was introduced to Timoni West's work. While looking into VR design it's impossible to miss the futuristic things Timoni is working on over at Unity Labs. For those who are new to all this, Unity is a game development platform you can use to develop your own 2D or 3D games. And of course, VR experiences.

The more I got lost in the wonderful world of virtual reality, the more I started wondering how one can have a professional career in this field. I'll admit I still don't have much practice designing or developing for VR, but even my little experiments made me curious about the future of VR and how we are going to design for it in the future.

So I thought I'd ask the master herself.

Hey Timoni, I know you lead the Immersive Authoring Tools group at Unity Labs. Simply put, you’re working on VR tools that help people design for VR inside of VR. Did I get this right?

Yes, that’s exactly what I do. But I’d broaden up the category a bit. Right now, we work primarily in VR because VR headsets are widely available and it’s easy to have a solid, immersive experience.

In contrast, the most well-known MR [mixed reality] headsets, the Hololens and Meta, are still developer kits and their tech is in some ways quite limited. Great positional tracking but a small field of view, and instead of controllers, they’ve gone the hand gesture route — which is, at the current tech level, tedious and frustrating.

But as soon as we have a solid MR headset with a wide field of view and top-notch gesture recognition, we’ll be supporting it.

“At their very best, computers allow you to manipulate reality to your liking.”

Virtual reality has experienced its up and downs over the last couple years — at least that’s what it feels like to me. One month everyone is talking about it, and the next month news around VR is slowing down again. What are your thoughts on the current state of VR? Should the design community take it more seriously?

I see the current generation of immersive design as laying the foundation for design patterns that will continue on in devices yet to be dreamed of. In Labs, our goal is to take the tech seriously, and seriously consider how to make best use of it — before it hits critical mass and especially before it reaches its final form.

I very much admire the work Xerox PARC did under Rob Taylor in the ‘70s and ‘80s. They didn’t know how influential the Xerox Star would be. And they surely never guessed that in 2017, the most common type of computer would be pocket-sized devices we still insist on calling “phones.” But it didn’t matter. Regardless of size, most computers today still use the GUI patterns PARC came up with: icons, folders, tapping to select.

Regardless of how the current round of consumer VR ends up, I truly believe the future of computer-human interaction is immersive. Almost any movie with a futuristic interface can tell you that. In our dreams, computers are friendly and ubiquitous. A companion you talk to, a set of flexible interfaces that allow you to type, draw or sculpt. At their very best, computers allow you to manipulate reality to your liking.

Does immersive computing need to have a headset? Absolutely not. But in 2017, this is what we have to work with.

I’ve noticed very few of my peers in the design community are working in VR. One of the reasons may be that the barrier to entry is high and the opportunities seem rare. You either have to work at one of the big corporations that invest in VR or do it in your spare time. Something like “VR freelance designer” still seems a bit unrealistic. Do you agree? What would need to happen to change this?

This is absolutely the case and at least in the short term, I don’t see it changing. VR is niche hardware in an era where funding, success metrics and attention are all based on scale. VR has not scaled; there are not a lot of numbers out there now, but it’s likely there are less than a million high-end headsets.

That being said, there’s a bit of upside: I would absolutely hire a great freelance VR designer if they had a solid self-directed portfolio, because I know the opportunities aren’t out there. But I would likely not hire a mobile or web designer who didn’t have some company experience in the field.

The technical requirements to get started certainly are daunting. Getting a full hardware setup running can cost you a couple thousand dollars. Do you have any advice for people who want to do VR development but can’t afford a premium headset and computer to run the HTC Vive or Oculus Rift?

Here’s a industry-wide open secret: We’re desperate for good content. If you’ve downloaded Unity and done some 2D prototypes, if you’ve got mocks and a solid roadmap, everything that points to a solid vision, likely someone wants to help you out. Email HTC. Email Oculus. Email the Google Daydream team. Bug people on Twitter. Tell them about what you want to build.

If that doesn’t work, go to meet ups and join Slack channels. Ask around and see if someone will lend you hardware. Beg, borrow or steal.

“You simply can’t be militant, and that’s in an era when design militancy is very much prized at large tech companies.”

What makes a good VR designer and how do they differ from a more “traditional” interactive designer who has worked with web and mobile before? What are the key differences in thinking and technical abilities?

The largest difference is in toolset. You’ve simply got to work in 3D and there’s no way around it, unless you have a dedicated 3D modeler and animator on your team who works closely with you. This can be daunting if you’ve only worked in 2D, and will involve extra effort and learning. Most great designers I know love to learn, but if you’re a designer with 10 years or more of experience, it will require you to diverge your skill set at a point where you’d most likely be doing a deep-dive into your craft.

In terms of thinking, a lot of design, particularly in web and mobile, is all about best practices. At this stage the best VR design does not draw on existing best practices. The underlying theory is consistent — things should be simple, intuitive and easy to use — but the design patterns themselves are different. For example, scrollbars are very annoying in VR. Scrollbars! Possibly one of the most common design patterns in computer platforming history.

So you simply can’t be militant, and that’s in an era when design militancy is very much prized at large tech companies.

Right now VR seems focused on gaming. But browsing the Steam store, some of my favorite apps have been educational ones — zipping through the solar system or studying the anatomy of the human body. VR has made me interested in things I found boring and dry back in school.

What fields of interest do you think could get a little extra push with VR?

I completely agree with you. My favorite apps happen to be creation apps. Lucky for me, I get to lead a team that works on creation apps! I think education apps can fall in this category too. It’s great to see a visualization of the body or a cell, for example, but how much cooler when you can interact with it and really get your hands dirty.

In general, you can think of VR as being best for emulating experiences that are otherwise tedious, difficult or expensive. This is why apps like SoundStage are so great. Buying that much physical musical equipment is simply impossible. It takes up too much room, it’s hard to find and it’s cost-prohibitive. But in VR, you can have a giant warehouse of synths, tape decks and drum kits.

What are your top three favorite VR experiences?

I hesitate to say my top experiences now because, like movies, I feel the need to start categorizing or pointing out specific favorite elements. But I will say that stepping outside the rocket in Apollo II when it’s on the way to the moon is one of my favorite aha! moments, and one of my favorite memories in VR.

“In VR, you can be anywhere! So why do I keep ending up in some boring, ordinary looking kitchen?”

When I think of VR I imagine the endless possibilities of dream worlds, things that don’t exist. Yet I’ve seen many VR experiences that are trying to replicate what we already have in real life. It almost seems like a waste of VR’s potential. Put me on a spaceship, not a virtual recreation of the city I live in.

Why do you think we tend to recreate real world experiences with VR instead of going crazy with our imagination? Is it just part of the process?

This is a pet peeve of mine. In VR, you can be anywhere! So why do I keep ending up in some boring, ordinary looking kitchen?

I think there are a few reasons for this and lack of imagination is not one of them — or at least, not the main culprit. Here’s what I think is happening.

First: Often people who are trying out VR for the first time don’t have big budgets, so they go to the asset store and download a cheap package just to test things out. This is sensible. Not knocking anybody here. I’ve done it myself.

Second, when you’re looking for a package, you might have also noticed that VR freaks people out a little. Some folks hate that they can’t see their hands. Some folks don’t like the dark or hate not having a solid floor. So starting somewhere comfortable and normal then moving on to something more interesting seems like a reasonable idea.

Combine this problem with the tenets of modern product design: “Keep it simple” and “Don’t make me think,” and “Well-designed products are intuitive.” None of these tenants, solid as they are for product design, allow for fantastic exploration. And that’s how you end up in a bunch of boring digital kitchens instead of on a beautiful green gas planet swimming in an electric storm with robotic sea dragons.

As someone who’s read a lot of sci-fi, I have already become used to the idea of fantastic digital realities. I’d encourage people to read more — get inspired! Read all 3,000 pages of Tad Williams’ Otherland. Watch every movie Gmunk ever worked on. Borrow my copy of Prosthetic Realities. Then start working on your VR experience.

Say I’m totally hooked on the idea of developing for VR but I have no idea where to begin. What should I do? Is there a simple three-step guide to VR development by Timoni West?

Why, yes.

1. Try out a lot of VR.

a. Buy a rig yourself and go to town. ($$$!)

b. Find somebody, anybody, you know that has a rig. Beg and borrow so you can try out as much as you possibly can.

2. Download Unity and Blender (free!), and grab some packages from the Asset Store. I’m a fan of Synty Studio’s stuff and it’s really fun to play with in VR.

a. If you have access to a Rift or Vive, download the experimental build of EditorVR so you can edit and build out your world right in the headset.

b. If not, you can always develop VR for your phone!

3. Learn! Check out my colleague Tony Parisi’s O’Reilly book or just download Unity’s tutorials.

When I first tried designing for VR, I noticed a completely new set of challenges. One of them is movement, especially if your space is smaller than the physical room-scale VR space. With some research I realized that each game tackles this challenge differently.

Do you think we eventually need new standards for VR, similar to swipe gestures on mobile devices that everyone understands and follows? Is VR in the wild west phase right now?

Yes, things will eventually start to standardize, but we are in a tricky, fun phase now. The controllers need to standardize more first, of course, before we can standardize UX patterns. Game console controllers and hotkeys on PCs vs Mac never quite standardized completely, but they’re close enough that it’s not a huge context shift from one platform to another.

Once controller designs come closer together, one common pattern will likely emerge from the variations that will be designed in the meantime. But this is the fun experimental part. What feels best on a Vive controller? What do the Rift controllers do best? And what works best for Daydream?

Another challenge I noticed is focus and attention. If a person in VR has the chance to freely move around and look in any direction they want, how do you design to get their attention? How do you solve this and what role do audio cues play in this scenario?

The best way to get a user’s attention is to put them in a grey, boring room where there’s absolutely nothing interesting to look at, figure out where their ear is by determining the position of the HMD, and have the audio shout “LOOK OVER HERE!” in a frantic voice that sounds like it’s coming from just over their shoulder.

Ha ha. I kid. I mean, it’ll work. They’ll look behind them. But it’s not a great experience. And in VR you can have any experience you want, so let’s talk about how to design for flexibility.

In today’s world, designers are trained to optimize for a specific task. You're competing for attention and time, so you have to get users where they want to go FAST. But the cool thing about VR is that it’s extremely opt-in — in fact, aggressively so. You have to go out of your way to be in an immersive experience. So you have some time to let the user settle in, about thirty seconds or so, where they probably won’t rip off the headset, and simply can’t get bored and check their phone or hit the back button. You can do some useful things during this time.

First, you can set the baseline. I could tell you to have your action affordances be standard things like hover outlines, movement or sound cues. But what if your world is a shimmery paradise where everything moves and everything hums? That’s fine; the point is that affordances are by nature unusual and different. So give users time to figure out the underlying structure of the world — things hum, they move and glow— and then provide the action affordance as a big, non-moving, non-humming, non-glowing thing. Or make it extra moving, glowing and humming. Up to you, but you must make it feel different.

I don’t often like to say that “design can learn from video games” because video game players generally have a wildly different motivation than product users. But in our brave new reality, where experiences are completely immersive and can take on any aesthetic they choose, we can learn from those who have made new worlds before. Like this.

How would someone working as a freelance designer get VR projects? Do you have any tips on how they could sell it to potential customers?

If you want to find VR gigs, I’d do a few things. I’d start tweeting about looking for gigs, put my information up on Unity Connect, join some AR/VR slack channels, go to meetups and do my research on what studios are getting the gigs — The Mill, MPC, Framestore, Tool, FakeLove, etc. Reach out to your favorite indie VR devs and ask them for advice. Don't be afraid to cold call; the community is still small and people are really nice.

If you want to convince your clients to do a VR experience, you should focus on those clients who want to be known as forward-thinking, future tech innovators. Also focus on clients with bigger budgets; realistically speaking, any immersive project you do will simply take quite a lot of time and effort to be any good.

In conversations, focus on what virtual reality brings to the table. Say, SpaceX: It’s expensive to build a rocket ship and impossible to have ordinary people visit it, but you can do it in VR — and heck, viewers can even ride with the rocket ship.

Say, Nike: Nike can never build a real store shaped like a giant Air Jordan 8 which, when you walk inside, has an endless galaxy of showing space and every shoe Nike ever made, but in virtual reality, they can. Again, sell your clients on what VR brings to the table: things that are impossible, expensive and fantastic.

_________

Timoni, thank you so much for this fantastic interview.

VR does feels a bit like the wild west right now, but at the same time that's the good part about it. Everything is open and everything has yet to be defined. Getting into it early means working hard but also having fun. And as there is a big need for VR designers right now, you might want to consider looking into it yourself.

If this article inspired you, make sure to follow Timoni on Twitter and tell her you enjoyed reading this. If you're new to VR in general, check out this article where I talk about assembling my own VR machine, and this article about some of my favorite VR starter experiences.

Thank you for reading,
Tobias