We use our eyes for more than just seeing the world around us, we use them to orient ourselves and focus on important objects we need to interact with. Our eyes only focus on a set amount of space, you can test this right now by looking just to the left of your screen and attempting to read anything in this article. What if VR could focus on those specific spots, following your eyes in perfect 1:1 tracking?
This technique is called foveated rendering. It’s a concept we’ll explore more in depth shortly, but it’s one of many game changing applications that come with eye tracking improvements to VR. Get ready for a richer fitness experience in 2018, especially as more of this technology hits the mainstream market.
Understand the User’s Intent
Eye tracking removes some of the burden from mocap systems, which are often the only systems at play when trying to determine what a user is doing or viewing. Essentially, VR has to estimate where you’re looking based on where you’re pointing, which is effective but not perfect. With eye tracking, the system can rely less on what your hands are doing and look at your eyes for a combination of data and more realistic feedback.
No more almost-but-not-quite-on-target shots in Holopoint.
This better accuracy is important for immersion and nausea as well. Your eyes and hands want to work together, in fact your brain expects them to. With eye tracking, there’s no more discrepancy between what you’re doing and what you’re seeing. This is crucial in helping the brain accept VR immersion and improve your experience.
Locomotion
Current VR systems utilize teleportation pretty effectively. You can’t move everywhere, but you can move within a set space by teleporting to that location and physically walking within it. This works exceptionally well with walking simulators, or exploratory games like Fallout 4.
Eye tracking would allow you to focus on a location, press a button and move immediately. This would help reduce the number of steps necessary to get somewhere, and remove some of the game UI stuff that makes immersion difficult. Think about it this way, every time you aim your wand and teleport, your mind is remembering you’re in a game. Those subtle immersion breaks undercut what VR is trying to do, so eye tracking would really open up a whole new world of locomotion and immersion.
Foveated Rendering
The fovea is a small indentation in the retina where visual acuity is highest. In the test we asked you to perform earlier, your fovea was too focused on whatever was off screen to allow for readability of text, even though you can recognize what the text looked like and its colors. This is important for rendering purposes, and will ultimately require headsets to draw on less processing power to present richer visual experiences.
There are two benefits that we see. The first is that visuals can improve because the game only needs to render a small portion of what you’re seeing based on what your eyes are looking at. The second is that the cost of entry for VR will lower, even if headset technology costs increase, because the PC cost will decrease. Also, those with lower spec systems will see a boost in performance. Neat!
Available and Upcoming Technology
Fove, is an eye tracking headset that came from a Japanese startup focused on refining eye tracking. It retails for around $600, but the applications aren’t there yet. Maybe they can partner with a larger platform for a richer experience, but the experience offers a ton of promise.
There is rumored to be a $220 upgrade kit on the horizon that will offer eye tracking, and that appears to be in partnership with HTC. This accessory appears to be available only in China for the time being. Might the Vive Pro include some of this technology? Might companies working with Tobii, an eye tracking technology provider, create a better solution faster?
Google also acquired Eyefluence, and we suspect this may be part of a secret weapon its using to improve locomotion for its own VR offerings.
[youtube https://www.youtube.com/watch?v=L_I5Dxdbjqg&w=560&h=315]
This kind of technology will also give developers greater insight into what we look at in VR, where our eyes focus and why. Although it may take time for us to see this kind of tech emerge in consumer VR, we can be sure that eye tracking will add some incredible improvements to the already robust VR experience.
