Eye Tracking in Rec Room

Recently we spent some time experimenting with the latest and greatest eye-tracking technologies. We’ve been really excited about the impact eye-tracking would have on social VR, so we were extra pumped to start experimenting with the tech.  Check out the sneak peek below:

 

There are many uses of eye tracking in VR, and you’ve probably seen a few of them in the past.  We’ve seen eye tracking used for selection, aiming, foveated rendering, auto IPD measurement, and even authentication.

 

Those features are all incredibly exciting, but we wanted to spend time experimenting with ways we could use eye tracking to enhance the fun and silly social interactions that are uniquely Rec Room.

 

In the course of our experimentation we found two fun ways to enhance the Rec Room experience with eye tracking .  The first is pretty straightforward.  It’s gaze tracking.  This lets users see where other people are looking.  We experimented with a few different art styles. 

One which is subtle:

 

ezgif.com-crop.gif

 

And one that’s more caffeinated. 

 

We like sharing both our successes and our failures with all of you out there in the Rec Room community, and those caffeinated eyes are more in the latter category than the former.

 

We ended up going with sticking with our more subtle art style.  I guess the lesson here is, don't overdo it.  In real life, humans are laser attuned to the tiniest cues that come from the eyes.  The same is also true in VR and in Rec Room.  Simple and subtle can yield tremendous impact.  Just being able to see eye direction and blinks added a surprising amount of tone and context to 1:1 conversations.  

 

We noticed an even bigger benefit in group settings.  The tiny movements of the eyes are a powerful social cue indicating whose turn it is to talk.  With eye tracking enabled, we actually observed a noticeable decrease in people talking over one another.   

 

The second feature we added was something called VR Emojis ;)  The facial expressions of Rec Room avatars are driven by a wide range of variables, but we let a few new expressions be driven entirely by the eyes. 

 

When we see that someone closes just one eye, we show a wink, complete with hammy smile.

 

When we detect that both eyes are pointed up we show a perplexed face.

 

And when we see that someone is crossing their eyes, we show a silly face.

 

These facial expressions give people another opportunity to act a little goofy.  So naturally, they mix nicely with the rest of the Rec Room.  

 

You might be wondering when eye tracking is going to make it into a playable version of Rec Room.  The answer is, we're not sure just yet.  A lot of smart people are working hard to get eye tracking into a consumer headset, and when that happens, it’s a technology that we’re excited to support.

 

It’s hard to fully describe the impact of eye tracking with just words and photos.  Like VR itself, this is something you really need to experience to fully appreciate.  We here at Against Gravity are really excited for what eye tracking can bring to the future of social VR.