英伟达：走向Gaze跟踪的虚拟现实的移动渲染.docxVIP

Towards Foveated Rendering for Gaze-Tracked Virtual Reality Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, Aaron Lefohn GIVEN A VIRTUAL REALITY HEADSET WITH AN EYETRACKER… Slide 2 Conventional Foveated Rendering This Paper: Perceptual Target ~2x less perceivable foveation This Paper: Final Renderer ~2x less perceivable foveation 2-3x less shaded pixels MOTIVATION: HIGH-PERFORMANCE, HIGH-QUALITY VIRTUAL REALITY Application Resolution Framerate MPixels / sec Desktop game 1920 x 1080 x 1 60 124 Slide 6 MOTIVATION: HIGH-PERFORMANCE, HIGH-QUALITY VIRTUAL REALITY Application Resolution Framerate MPixels / sec Desktop game 1920 x 1080 x 1 60 124 2x 2016 VR (HTC Vive) 1080 x 1200 x 2 90 233 Slide 7 MOTIVATION: HIGH-PERFORMANCE, HIGH-QUALITY VIRTUAL REALITY Application Resolution Framerate MPixels / sec Desktop game 1920 x 1080 x 1 60 124 20x 2016 VR (HTC Vive) 1080 x 1200 x 2 90 233 2020 VR* 4000 x 4000 x 2 90 2880 * Prediction by Michael Abrash, October 2016 Slide 8 MOTIVATION: VR IS ABOUT RENDERING PERIPHERAL PIXELS Pixels in the central 20? field-of-view 96.20% 26.72% 3.64% 2.04% iPhone 7 Plus  27 Desktop Monitor  2016 VR HMD  2020 VR HMD* Slide 9 FOVEATED RENDERING: RENDER PERIPHERAL PIXELS CHEAPLY High-fidelity foveal pixels Low-fidelity peripheral pixels Slide 10 MODERN FOVEATED RENDERING: ADAPTIVE IMAGE RESOLUTION Multi-pass multi-resolution rendering [Guenter et al. 2012] Slide 11 MODERN FOVEATED RENDERING: ADAPTIVE IMAGE RESOLUTION Multi-pass multi-resolution rendering [Guenter et al. 2012] Pass 1 Lowest resolution Scene Slide 12 MODERN FOVEATED RENDERING: ADAPTIVE IMAGE RESOLUTION Multi-pass multi-resolution rendering [Guenter et al. 2012] Pass 1 upsample Lowest resolution Scene Slide 13 MODERN FOVEATED RENDERING: ADAPTIVE IMAGE RESOLUTION Multi-pass multi-resolution rendering [Guenter et al. 2012] Pass 1 upsample Lowest resolution Scene Pass 2 upsample Medium resolution Slide 14 MODERN FOVEATED RENDERING: ADA