Categories
Blog Posts

What is Foveated Rendering?

What is Foveated Rendering?

Foveated Image

January 23rd, 2016

One of the main issues with processing images for Virtual Reality is the intense workload it places on the computer’s GPU. With current steroscopic VR setups, there are two slightly different copies of each video image rendering each frame. One image is sent to one eye while the other image is sent to the other eye to get the 3D effect. Additionally, these images have to be refreshed a very high rate to keep the experience smooth and without judder. For VR applications, the refresh rate has to be even higher than normal because the image takes up our whole field of view and immerses us in the environment. This causes us to notice much more subtle differences that can lead to VR sickness. The goal for most VR companies has been to render both images at 90 Hz, which would mean 90 frames a second for each image. This is what the consumer version of the Oculus Rift and the HTC Vive will be running at. Needless to say, running two images, one for each eye at 90 frames per second is very processing intensive for a GPU. This is also the reason, the spec requirements are so high for the computers needed to run good VR headsets. The high specs and extra costs keeps VR out of the hands of many consumers.

Foveated Rendering looks to solve this issue by greatly reducing the processing power needed by the GPU to render the two images at 90Hz. The idea behind foveated rendering is that only the area the eye is focused on is rendered in full resolution. A bigger area surrounding the focused area is rendered in just a medium resolution, and the remaining areas of the screen are rendered in a low resolution. This enables the GPU to very rapidly render the low and medium resolution areas while only spending time to render the high resolution areas on a small portion of each image. The image above provides a example of the different layers of resolution that would be rendered for the image. This technique simulates the way we use our eyes where the area we are focused on is clear while our peripheral vision is blurred.

In addition to the rendering technique, foveated rendering also requires eye-tracking technology. As the user’s focus moves around a screen, it is neccesary to see where the user’s eye is focused so the system knows where to render the medium and high resolution areas. Obviously, you can imagine the eye-tracking and the rendering need to both happen very quickly to pull this off. Earlier this month, SensoMotoric Instruments (SMI), a German company specializing in eye-tracking technology has introduced their consumer-ready eye-tracking platform with the ability to do this as 250Hz, or 250 frames per second. Oculus has already reported they are working on eye-tracking technology for their 2nd generation headset, and I would expect to see foveated rendering incorporated. If so, this will go a long way in allowing more detailed graphics to be displayed but also in reducing the required specs a computer needs to display VR. This would make VR accesible to even more people. The video below gives a more detailed look at SMI’s foveated rendering in action.

More Articles


20160108_142028


CES 2016 Concept Cars

CES 2016 CONCEPT CARS More Articles Hulu Investing Heavily Into VR Hulu Investing Heavily Into VR Add Hulu to the list of companies getting bullish on Virtual Reality. Ben Smith, Hulu’s Head of Experience recently told Variety they are “spending a lot of time on virtual reality.” The company has been working on an app


Gear VR Still1


Virtual Reality Takes Over CES 2016

Virtual Reality Takes Over CES 2016 Attack of the VR…it’s everywhere! Virtual Reality has been at CES before. It has also been at several other conventions over the past couple of years as an emerging technology. But 2016 is VR’s coming out party to consumers, and it had a dominating presence this year at CES.


CES


CES – VR, Drones, & 8K TVs

CES – VR, Drones, & 8K TVs CES 2016 was full of VR but there was plenty of other things to see there too. In fact, there was also a heavy presence of drones as well as self-driving car technology. In the TV department, companies have already moved past 4K TVs and are already selling

Categories
Blog Posts

Virtual Reality Takes Over CES 2016

Virtual Reality Takes Over CES 2016

Gear VR Still1

January 16th, 2016

Attack of the VR…it’s everywhere!

Virtual Reality has been at CES before. It has also been at several other conventions over the past couple of years as an emerging technology. But 2016 is VR’s coming out party to consumers, and it had a dominating presence this year at CES. Not only was there a large area devoted specifically to VR but there were so many other non-VR companies using it for their demos. CES had a record breaking 2.47 million square feet of exhibit space this year and you could not walk 30 feet without seeing a demo using a VR head-mounted display.

HTC This is RealAfter years of hype, VR is now real! The HTC Vive was set up outside of the convention center with lines that had several hour long waits. By 2pm each day the wait was already long enough for them to shut down the line for the day. HTC was initially primed to beat Oculus to the market with shipping dates scheduled for this past December but last month they pushed it out to April because of a “very big technological breakthrough” that reportedly renders the initial headset obsolete. The breakthrough was revealed to be the integration of front-facing cameras that can allow the userHTC 3 Hour Wait to see objects in the real world. This creates more possibilities for augmented reality in addition to virtual reality. It also allows for developers to integrate the cameras in ways to maintain a mostly virtual world while alerting also users when they are getting close to real world objects.

HTC Line

Inside the convention center, Oculus had the substantial presence that was expected. It was easily the largest area with demos taking place inside a huge structure and long lines surrounding it from all sides. The pictures below provide an example of the size of the structure and the lines along the different walls that persisted throughout each day.

Oculus Line1

Oculus Line2

Oculus Line3

Also on display at Oculus was the consumer version they just began taking preorders for. An entire Oculus system is expected to cost about $1,500 with its initial launch, which includes the cost for the PC needed to run it as well as the Oculus kit. The Oculus kit alone is $599 and will start shipping out to consumers this March. The kit will contain the head-mounted display, a sensor, a wireless Xbox One controller, and a remote.

Oculus Box

Oculus also recently announced their Touch controllers, which will not be ready until the second half of 2016, and so they are not part of the preorders. The reviews from the Oculus Touch controllers have been amazing. The pair of controllers are a mirror image of each other, one for each hand. There are sensors in them that allow the Oculus system to track your hands in the virtual world. User have reported them as being the most natural and intuitive VR controllers to date. The Oculus Touch controllers are shown below on display in the Oculus booth.

Oculus Touch

The very nature of VR and its immersive qualities inspire non-traditional controllers beyond the typical hand controllers. One of the more prominent ones is the Omni by Virtuix. The Omni is an omnidirectional treadmill for use within virtual environments. It utilizes a concave surface with shoes designed for low friction. You are strapped to a harness and can rotate 360-degrees so you are able to turn, walk, and run in your virtual worlds. The Virtuix Omni accessory is already for available preorders as well starting at $699. A picture of the Omni is shown below along with a video of it in action.

Omni1    

While Oculus and the HTC Vive start prepping for their upcoming preorders, there is one VR headset that has already hit the market. The consumer version of the Samsung Gear VR was released last November. This product comes from an Oculus collaboration with Samung. The Gear VR is much less powerful but still very impressive. The advantage of the Gear VR is that instead of requiring a PC it runs off of a Samsung Galaxy S5, Galaxy S6, or a Note 5 phone. It can only be used if you have one of these phone devices But if you do, it is only $99 and has the added benefit of being an untethered  mobile VR option.

Gear VR Still3

The setup for the Gear VR at CES was probably the rowdiest area with the most energy. There was music blasting over three rows of about 40 seats where users were taken on a roller coaster ride in the Gear VR. While it looked liked everyone using the Gear VR was having a blast, it may have been just as fun watching everyone smile and throw there hands up during the ride, blissfully unaware of everyone around them watching their reactions. Some were having more fun than others…

Gear VR Still2

Check out the video below for the full effect…

 As mentioned earlier, VR was everywhere…even in the areas where it was not the product, I was amazed at how prevalent it was at the convention for demoing other products. Denso, one of the world’s largest auto-parts manufacturer from Japan was utilizing the Oculus Rift in an exhibit to show how their technologies help with collision avoidance, road hazards, and utilize eye-tracking so the vehicle can respond to where you are looking inside the car.

VR01

There was a host of other companies on display using VR including Kia’s Drive Wise brand for self-driving cars, Intel for various products it powers, the open source OSVR head-mounted display, the Allie VR headset for streaming from its Allie portable action cameras, and Avegant’s Glyph, which won a “Best of CES” award for their portable theater that uses optics and microscopic mirrors to project images directly into the eyes.

VR02  VR05  VR07

VR06  VR08  VR09

VR03  VR10

 The 90’s called, and they’re totally jeally…

More Articles


Foveated Image


What is Foveated Rendering?

What is Foveated Rendering? January 23rd, 2016 One of the main issues with processing images for Virtual Reality is the intense workload it places on the computer’s GPU. With current steroscopic VR setups, there are two slightly different copies of each video image rendering each frame. One image is sent to one eye while the


Hulu vr


Hulu Investing Heavily Into VR

Hulu Investing Heavily Into VR February 19th, 2016 Add Hulu to the list of companies getting bullish on Virtual Reality. Ben Smith, Hulu’s Head of Experience recently told Variety they are “spending a lot of time on virtual reality.” The company has been working on an app for the Gear VR that was originally slated


20160108_142028


CES 2016 Concept Cars

CES 2016 CONCEPT CARS More Articles Hulu Investing Heavily Into VR Hulu Investing Heavily Into VR Add Hulu to the list of companies getting bullish on Virtual Reality. Ben Smith, Hulu’s Head of Experience recently told Variety they are “spending a lot of time on virtual reality.” The company has been working on an app

Categories
Blog Posts

CES – VR, Drones, & 8K TVs

CES – VR, Drones, & 8K TVs

CES

January 16th, 2016

CES 2016 was full of VR but there was plenty of other things to see there too. In fact, there was also a heavy presence of drones as well as self-driving car technology. In the TV department, companies have already moved past 4K TVs and are already selling us on 8K TVs.

 

Drone01

 

Shown above is the Ehang 184, an autonomous smart drone designed to carry a live human being. It can carry up to 220 pounds and no training or flying is required by the passenger. Upon entering the drone you input your destination and the drone autonomously flies you to where you’re going. It currently flies at 60 mph and can travel about 10 miles. Personally, I feel this is the future of travel. The powers that be are working to take our cars from us but I’ll submit to them if they trade me for a drone. Self-driving vehicles and human-sized drones are the perfect mashup for my future reality. Here is a video showing the Ehang 184 in action: 

 

 

Of course there were several other companies there with more familiar sized drones. DJI had a sizable area showing off their array of products. And with drone popularity greatly expanding, we are seeing a lot more interest taken in form design to separate companies from the pack and appeal to a broader set of consumers. Some of these designs include partially and fully covering the visibility of the propellers, which gives off a futuristic hovering effect. 

 

Drone4 Drone03 Drone02 Drone5 Drone6 Drone8 Drone7 DroneCircle

 

CES is always full of new cool concept cars as well. This year, in addition to the designs of the cars there was a lot of self-driving technology on display. It was obvious that almost all car manufactures are on board with self-driving technology and they are either investing heavily with putting more on the road in the near future or they are starting to significantly increase their research and testing in this area. Pictured below is a monitor showing off a car’s sensor ability to detect everything around it. One company that seems to be everywhere these days is NVIDIA. In addition to being all over the powerful new GPUs for VR and trying to stuff desktop GPUs in laptops, they are also hard at work in the self-driving car field. NVIDIA’s CEO and CES Keynote speaker, Jen-Hsun Huang, introduced their Drive PX2 platform with impressive deep learning technology and significant improvement of processing power over the Drive PX. Haung’s excitability and leather jacket at the announcement definitely suggest he’s ready to change the world.

 

CarRadar         NVIDIA

 

There is always a ton of new TVs at CES and this year is no different. Samsung convinced me I need to own about 80 TVs with an impressive display of their modular TV. With these TVs you can keep buying more TVs over the years to connect with the ones you already own and continually grow the size of your TV. What’s your 4K got to do wit me…I’m not tryin’ to hear that see. Moving beyond 4K TVs was a slew of 8K TVs. This is something that may seem unnecessary with TVs but I love to see this technology push the boundaries so we can get more pixels in phones for VR technology.

 

8KTV

 

In the category of random things, I was definitely far too impressed with this car driving around a record and playing the song through its ‘car speakers’. It also wouldn’t be CES if there wasn’t something from Star Wars so, of course, we had an appearance of BB-8. 

 

               StarWars

 

And I don’t know what this is but it was really cool to look at so I took a picture, and now you get to see it too.

 

Cubes

More Articles


FutureGearVR


How Will the Virtual Reality Industry Evolve?

How Will the Virtual Reality Industry Evolve? February 24th, 2016 “The VR Opportunity” was a forum at the Mobile World Congress this week in Barcelona centered on how Virtual Reality will evolve in the marketplace. This is a topic I like to discuss, and I am in the camp that feels gaming will lead the


20160108_142028


CES 2016 Concept Cars

CES 2016 CONCEPT CARS More Articles Hulu Investing Heavily Into VR Hulu Investing Heavily Into VR Add Hulu to the list of companies getting bullish on Virtual Reality. Ben Smith, Hulu’s Head of Experience recently told Variety they are “spending a lot of time on virtual reality.” The company has been working on an app


Gear VR Still1


Virtual Reality Takes Over CES 2016

Virtual Reality Takes Over CES 2016 January 16th, 2016 Attack of the VR…it’s everywhere!   Virtual Reality has been at CES before. It has also been at several other conventions over the past couple of years as an emerging technology. But 2016 is VR’s coming out party to consumers, and it had a dominating presence

Categories
Blog Posts

CES 2016 Concept Cars

CES 2016 CONCEPT CARS

More Articles


Hulu vr


Hulu Investing Heavily Into VR

Hulu Investing Heavily Into VR Add Hulu to the list of companies getting bullish on Virtual Reality. Ben Smith, Hulu’s Head of Experience recently told Variety they are “spending a lot of time on virtual reality.” The company has been working on an app for the Gear VR that was originally slated for last fall.


FutureGearVR


How Will the Virtual Reality Industry Evolve?

How Will the Virtual Reality Industry Evolve? “The VR Opportunity” was a forum at the Mobile World Congress this week in Barcelona centered on how Virtual Reality will evolve in the marketplace. This is a topic I like to discuss, and I am in the camp that feels gaming will lead the early adoption phase


Foveated Image


What is Foveated Rendering?

What is Foveated Rendering? One of the main issues with processing images for Virtual Reality is the intense workload it places on the computer’s GPU. With current steroscopic VR setups, there are two slightly different copies of each video image rendering each frame. One image is sent to one eye while the other image is