Many companies and experts within the AR/VR industry talk about the benefits of foveated rendering—the ability to only render a portion of the scene that a user is looking at. Some companies have implemented the technology to improve performance on standalone VR headsets, such as the Oculus Quest, HTC Vive Pro Eye and Pico Neo 2 Eye. After all, there is no point in rendering the VR or AR display at full resolution if our eyes can only see a small area at full resolution at a time. Still, many methods of foveated rendering can affect performance and power consumption, and very few people know the benefits of using eye-tracking to dynamically render content based on where the iris of the eye is looking. Moor Insights & Strategy recently published an in-depth paper on the topic, which I encourage you to read if interested. Here’s a very brief overview of the topics covered.
What are the benefits of foveated rendering?
There are different types of foveated rendering, and various vendors implement the technology in different ways. It is essential to understand what kind of impact the technology, in its various forms, can have in terms of performance. Many numbers get thrown around regarding the performance improvements of foveated rendering. Tobii Technologies, one of the world’s leading providers of complete eye-tracking hardware and software solutions, has shown improvements of anywhere between 57% and 72% across multiple headsets and game engines. The significant differences come from whether the technology is dynamic or static/fixed. One of the benefits of fixed foveated rendering is that it does not require any additional hardware, but still delivers some performance speedups. With fixed foveated rendering, however, if the user ever moves their gaze away from the center of the display, there is a noticeable degradation in image quality.
Dynamic foveated rendering, on the other hand, requires additional hardware to track the gaze of the human eye, but renders an area based on where the eye is looking at that exact moment. This allows for a much more granular rendering of what is on the screen, which allows for even more aggressive variable-rate shading (an industry term for different render resolutions for different areas of the image). This technology is beneficial because, when combined with eye-tracking, it maximizes the performance of a VR or AR headset. However, it is difficult to measure these performance benefits; many companies who implement eye-tracking solutions have not publicly communicated the performance benefits of dynamic foveated rendering versus fixed, or even the benefits of having foveated rendering at all. Measuring benefits is especially difficult because dynamic foveated rendering involves tracking the human eye’s location. When you include a human in the equation of providing a benchmark, things can get a little dicey.
That’s why it’s notable that Tobii Technologies recently established a series of benchmarking best practices and metrics to help the industry understand these performance improvements. In the paper published today, we cover the importance of dynamic foveated rendering and how different parts of the XR community—developers, OEMs and enterprise users—can utilize the added performance. Tobii’s approach to measuring and quantifying dynamic foveated rendering performance should serve as a guide to the rest of the industry on how to talk about the benefits of this technology. If you’d like to learn more about the benefits and the testing methodology on both PC-based and standalone VR systems, click here to read the in-depth paper.