Previous slide
Next slide
Toggle fullscreen
Open presenter view
NeRF - Neural Radiance Fields
Implicit surfaces rendering in PyTorch3D
3D Graphics Systems - IMPA 2023 - Lab Class #14
Goal
Given a set of sparse views of a scene, we would like to learn how to render a new viewpoint of this scene.
Strategy
Scene is encoded in a neural network
Input: 3D spatial coordinates + 2D viewing direction coordinates
Output: 3D color + 1D density
"Volumetric version of light field"
View Dependent Effects
Your browser does not support the video tag.
Strategy
Two Pass Rendering: Coarse
Two Pass Rendering: Fine
Volume Rendering is Differentiable
Positional Encoding
Your browser does not support the video tag.
Positional Encoding
Learn on a higher dimensional space (from
to
)
is applied separately to each coordinate
Results
Results
Your browser does not support the video tag.
Limitations
Speed - hundreds of thousands of iterations for high-res
Sampling resolution along rays
Generalization - specific for one scene
No more?
PixelNeRF
(CVPR, 2023 - happening NOW!)
Implicit Surfaces in PyTorch3D
Let's check the code on Colab!