Unbiased Inverse Volume Rendering with Differential Trackers
Abstract
Volumetric representations are popular in inverse rendering because they have a simple parameterization, are smoothly varying, and transparently handle topology changes. However, incorporating the full volumetric transport of light is costly and challenging, often leading practitioners to implement simplified models, such as purely emissive and absorbing volumes with “baked” lighting.
One such challenge is the efficient estimation of the gradients of the volume's appearance with respect to its scattering and absorption parameters. We show that the straightforward approach—differentiating a volumetric free-flight sampler—can lead to biased and high-variance gradients, hindering optimization. Instead, we propose using a new sampling strategy: differential ratio tracking, which is unbiased, yields low-variance gradients, and runs in linear time.
Differential ratio tracking combines ratio tracking and reservoir sampling to estimate gradients by sampling distances proportional to the unweighted transmittance rather than the usual extinction-weighted transmittance. In addition, we observe local minima when optimizing scattering parameters to reproduce dense volumes or surfaces. We show that these local minima can be overcome by bootstrapping the optimization from nonphysical emissive volumes that are easily optimized.
Results presented in the paper are best viewed through our interactive results viewer.
Code: our implementation of Differential Ratio Tracking, based on Mitsuba 3, is available under the BSD 3-clause license on GitHub.
Video
Figures
Text citation
Merlin Nimier-David, Thomas Müller, Alexander Keller, and Wenzel Jakob. 2022. Unbiased Inverse Volume Rendering with Differential Trackers. In Transactions on Graphics (Proceedings of SIGGRAPH) 41(4).
BibTeX
@article{nimierdavid2022unbiased, author = {Merlin Nimier-David and Thomas M\"uller and Alexander Keller and Wenzel Jakob}, title = {Unbiased Inverse Volume Rendering with Differential Trackers}, journal = {ACM Trans. Graph.}, issue_date = {July 2022}, volume = {41}, number = {4}, month = jul, year = {2022}, pages = {44:1--44:20}, articleno = {44}, numpages = {20}, url = {https://doi.org/10.1145/3528223.3530073}, doi = {10.1145/3528223.3530073}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {differentiable rendering, inverse rendering, volumetric rendering, radiative backpropagation, importance sampling} }