Categories
Blog Posts

What is Foveated Rendering?

What is Foveated Rendering?

Foveated Image

January 23rd, 2016

One of the main issues with processing images for Virtual Reality is the intense workload it places on the computer’s GPU. With current steroscopic VR setups, there are two slightly different copies of each video image rendering each frame. One image is sent to one eye while the other image is sent to the other eye to get the 3D effect. Additionally, these images have to be refreshed a very high rate to keep the experience smooth and without judder. For VR applications, the refresh rate has to be even higher than normal because the image takes up our whole field of view and immerses us in the environment. This causes us to notice much more subtle differences that can lead to VR sickness. The goal for most VR companies has been to render both images at 90 Hz, which would mean 90 frames a second for each image. This is what the consumer version of the Oculus Rift and the HTC Vive will be running at. Needless to say, running two images, one for each eye at 90 frames per second is very processing intensive for a GPU. This is also the reason, the spec requirements are so high for the computers needed to run good VR headsets. The high specs and extra costs keeps VR out of the hands of many consumers.

Foveated Rendering looks to solve this issue by greatly reducing the processing power needed by the GPU to render the two images at 90Hz. The idea behind foveated rendering is that only the area the eye is focused on is rendered in full resolution. A bigger area surrounding the focused area is rendered in just a medium resolution, and the remaining areas of the screen are rendered in a low resolution. This enables the GPU to very rapidly render the low and medium resolution areas while only spending time to render the high resolution areas on a small portion of each image. The image above provides a example of the different layers of resolution that would be rendered for the image. This technique simulates the way we use our eyes where the area we are focused on is clear while our peripheral vision is blurred.

In addition to the rendering technique, foveated rendering also requires eye-tracking technology. As the user’s focus moves around a screen, it is neccesary to see where the user’s eye is focused so the system knows where to render the medium and high resolution areas. Obviously, you can imagine the eye-tracking and the rendering need to both happen very quickly to pull this off. Earlier this month, SensoMotoric Instruments (SMI), a German company specializing in eye-tracking technology has introduced their consumer-ready eye-tracking platform with the ability to do this as 250Hz, or 250 frames per second. Oculus has already reported they are working on eye-tracking technology for their 2nd generation headset, and I would expect to see foveated rendering incorporated. If so, this will go a long way in allowing more detailed graphics to be displayed but also in reducing the required specs a computer needs to display VR. This would make VR accesible to even more people. The video below gives a more detailed look at SMI’s foveated rendering in action.

More Articles


20160108_142028


CES 2016 Concept Cars

CES 2016 CONCEPT CARS More Articles Hulu Investing Heavily Into VR Hulu Investing Heavily Into VR Add Hulu to the list of companies getting bullish on Virtual Reality. Ben Smith, Hulu’s Head of Experience recently told Variety they are “spending a lot of time on virtual reality.” The company has been working on an app


Gear VR Still1


Virtual Reality Takes Over CES 2016

Virtual Reality Takes Over CES 2016 Attack of the VR…it’s everywhere! Virtual Reality has been at CES before. It has also been at several other conventions over the past couple of years as an emerging technology. But 2016 is VR’s coming out party to consumers, and it had a dominating presence this year at CES.


CES


CES – VR, Drones, & 8K TVs

CES – VR, Drones, & 8K TVs CES 2016 was full of VR but there was plenty of other things to see there too. In fact, there was also a heavy presence of drones as well as self-driving car technology. In the TV department, companies have already moved past 4K TVs and are already selling

2 replies on “What is Foveated Rendering?”

Leave a Reply