RGL
Realistic Graphics Lab
EPFL Logo

Monte Carlo Estimators for Differential Light Transport

In Transactions on Graphics (Proceedings of SIGGRAPH 2021)

Dif­fer­en­ti­able ren­der­ing of a scene fea­tur­ing spec­u­lar in­ter­re­flec­tion between metal­lic sur­faces of vary­ing rough­ness. We dif­fer­en­ti­ate the im­age with re­spect to the com­bined rough­ness of all ob­jects, which pro­duces the gradi­ents shown in the first column with in­sets. A dis­con­cert­ingly large num­ber of dif­fer­en­tial es­tim­at­ors can solve this prob­lem, al­beit with drastic­ally dif­fer­ent stat­ist­ic­al ef­fi­ciency: the fol­low­ing four columns high­light the stand­ard de­vi­ation of emit­ter sampling and three ma­ter­i­al-based strategies. An over­view of the ex­haust­ive set of com­bin­a­tions (21 meth­ods) and res­ults for an ad­di­tion­al four es­tim­at­ors are provided in the sup­ple­ment­al ma­ter­i­al, which also con­tains un­cropped im­ages. The ob­ject­ive of our work is to provide in­tu­ition on how to nav­ig­ate the large design space of dif­fer­en­tial Monte Carlo es­tim­at­ors.

Abstract

Phys­ic­ally based dif­fer­en­ti­able ren­der­ing al­gorithms propag­ate de­riv­at­ives through real­ist­ic light trans­port sim­u­la­tions and have ap­plic­a­tions in di­verse areas in­clud­ing in­verse re­con­struc­tion and ma­chine learn­ing. Re­cent pro­gress has led to un­biased meth­ods that can sim­ul­tan­eously com­pute de­riv­at­ives with re­spect to mil­lions of para­met­ers. At the same time, ele­ment­ary prop­er­ties of these meth­ods re­main poorly un­der­stood.

Cur­rent al­gorithms for dif­fer­en­ti­able ren­der­ing are con­struc­ted by mech­an­ic­ally dif­fer­en­ti­at­ing a giv­en prim­al al­gorithm. While con­veni­ent, such an ap­proach is simplist­ic be­cause it leaves no room for im­prove­ment. Dif­fer­en­ti­ation pro­duces ma­jor changes in the in­teg­rals that oc­cur throughout the ren­der­ing pro­cess, which in­dic­ates that the prim­al and dif­fer­en­tial al­gorithms should be de­coupled so that the lat­ter can suit­ably ad­apt.

This leads to a large space of pos­sib­il­it­ies: con­sider that even the most ba­sic Monte Carlo path tracer already in­volves sev­er­al design choices con­cern­ing the tech­niques for sampling ma­ter­i­als and emit­ters, and their com­bin­a­tion, e.g. via mul­tiple im­port­ance sampling (MIS). Dif­fer­en­ti­ation causes a ver­it­able ex­plo­sion of this de­cision tree: should we dif­fer­en­ti­ate only the es­tim­at­or, or also the sampling tech­nique?  Should MIS be ap­plied be­fore or after dif­fer­en­ti­ation? Are spe­cial­ized de­riv­at­ive sampling strategies of any use? How should vis­ib­il­ity-re­lated dis­con­tinu­it­ies be handled when mil­lions of para­met­ers are dif­fer­en­ti­ated sim­ul­tan­eously? In this pa­per, we provide a tax­onomy and ana­lys­is of dif­fer­ent es­tim­at­ors for dif­fer­en­tial light trans­port to provide in­tu­ition about these and re­lated ques­tions.

Video

Video

Figures

Text citation

Tizian Zeltner, Sébastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo Estimators for Differential Light Transport. In Transactions on Graphics (Proceedings of SIGGRAPH) 40(4).

BibTeX
@article{Zeltner2021MonteCarlo,
    author = {Tizian Zeltner and Sébastien Speierer and Iliyan Georgiev and Wenzel Jakob},
    title = {Monte Carlo Estimators for Differential Light Transport},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH)},
    volume = {40},
    number = {4},
    year = {2021},
    month = aug,
    doi = {10.1145/3450626.3459807}
}