Monte Carlo Estimators for Differential Light Transport

Tizian Zeltner Sébastien Speierer Iliyan Georgiev Wenzel Jakob

In Transactions on Graphics (Proceedings of SIGGRAPH 2021)

Differentiable rendering of a scene featuring specular interreflection between metallic surfaces of varying roughness. We differentiate the image with respect to the combined roughness of all objects, which produces the gradients shown in the first column with insets. A disconcertingly large number of differential estimators can solve this problem, albeit with drastically different statistical efficiency: the following four columns highlight the standard deviation of emitter sampling and three material-based strategies. An overview of the exhaustive set of combinations (21 methods) and results for an additional four estimators are provided in the supplemental material, which also contains uncropped images. The objective of our work is to provide intuition on how to navigate the large design space of differential Monte Carlo estimators.

Resources

Paper (6.3 MB)

Supplemental material (11.7 MB)

Video (12.6 MB)

Talk video (411.9 MB)

Publisher version

Cite

Abstract

Physically based differentiable rendering algorithms propagate derivatives through realistic light transport simulations and have applications in diverse areas including inverse reconstruction and machine learning. Recent progress has led to unbiased methods that can simultaneously compute derivatives with respect to millions of parameters. At the same time, elementary properties of these methods remain poorly understood.

Current algorithms for differentiable rendering are constructed by mechanically differentiating a given primal algorithm. While convenient, such an approach is simplistic because it leaves no room for improvement. Differentiation produces major changes in the integrals that occur throughout the rendering process, which indicates that the primal and differential algorithms should be decoupled so that the latter can suitably adapt.

This leads to a large space of possibilities: consider that even the most basic Monte Carlo path tracer already involves several design choices concerning the techniques for sampling materials and emitters, and their combination, e.g. via multiple importance sampling (MIS). Differentiation causes a veritable explosion of this decision tree: should we differentiate only the estimator, or also the sampling technique? Should MIS be applied before or after differentiation? Are specialized derivative sampling strategies of any use? How should visibility-related discontinuities be handled when millions of parameters are differentiated simultaneously? In this paper, we provide a taxonomy and analysis of different estimators for differential light transport to provide intuition about these and related questions.

Video

Figures

Text citation

Tizian Zeltner, Sébastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo Estimators for Differential Light Transport. In Transactions on Graphics (Proceedings of SIGGRAPH) 40(4).

BibTeX

@article{Zeltner2021MonteCarlo,
    author = {Tizian Zeltner and Sébastien Speierer and Iliyan Georgiev and Wenzel Jakob},
    title = {Monte Carlo Estimators for Differential Light Transport},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH)},
    volume = {40},
    number = {4},
    year = {2021},
    month = aug,
    doi = {10.1145/3450626.3459807}
}