Realistic Graphics Lab

Monte Carlo Estimators for Differential Light Transport

To appear in Transactions on Graphics (Proceedings of SIGGRAPH 2021)

Dif­fer­en­ti­able ren­der­ing of a scene fea­tur­ing spec­u­lar in­ter­re­flec­tion between metal­lic sur­faces of vary­ing rough­ness. We dif­fer­en­ti­ate the im­age with re­spect to the com­bined rough­ness of all ob­jects, which pro­duces the gradi­ents shown in the first column with in­sets. A dis­con­cert­ingly large num­ber of dif­fer­en­tial es­tim­at­ors can solve this prob­lem, al­beit with drastic­ally dif­fer­ent stat­ist­ic­al ef­fi­ciency: the fol­low­ing four columns high­light the stand­ard de­vi­ation of emit­ter sampling and three ma­ter­i­al-based strategies. An over­view of the ex­haust­ive set of com­bin­a­tions (21 meth­ods) and res­ults for an ad­di­tion­al four es­tim­at­ors are provided in the sup­ple­ment­al ma­ter­i­al, which also con­tains un­cropped im­ages. The ob­ject­ive of our work is to provide in­tu­ition on how to nav­ig­ate the large design space of dif­fer­en­tial Monte Carlo es­tim­at­ors.


Phys­ic­ally based dif­fer­en­ti­able ren­der­ing al­gorithms propag­ate de­riv­at­ives through real­ist­ic light trans­port sim­u­la­tions and have ap­plic­a­tions in di­verse areas in­clud­ing in­verse re­con­struc­tion and ma­chine learn­ing. Re­cent pro­gress has led to un­biased meth­ods that can sim­ul­tan­eously com­pute de­riv­at­ives with re­spect to mil­lions of para­met­ers. At the same time, ele­ment­ary prop­er­ties of these meth­ods re­main poorly un­der­stood.

Cur­rent al­gorithms for dif­fer­en­ti­able ren­der­ing are con­struc­ted by mech­an­ic­ally dif­fer­en­ti­at­ing a giv­en prim­al al­gorithm. While con­veni­ent, such an ap­proach is simplist­ic be­cause it leaves no room for im­prove­ment. Dif­fer­en­ti­ation pro­duces ma­jor changes in the in­teg­rals that oc­cur throughout the ren­der­ing pro­cess, which in­dic­ates that the prim­al and dif­fer­en­tial al­gorithms should be de­coupled so that the lat­ter can suit­ably ad­apt.

This leads to a large space of pos­sib­il­it­ies: con­sider that even the most ba­sic Monte Carlo path tracer already in­volves sev­er­al design choices con­cern­ing the tech­niques for sampling ma­ter­i­als and emit­ters, and their com­bin­a­tion, e.g. via mul­tiple im­port­ance sampling (MIS). Dif­fer­en­ti­ation causes a ver­it­able ex­plo­sion of this de­cision tree: should we dif­fer­en­ti­ate only the es­tim­at­or, or also the sampling tech­nique?  Should MIS be ap­plied be­fore or after dif­fer­en­ti­ation? Are spe­cial­ized de­riv­at­ive sampling strategies of any use? How should vis­ib­il­ity-re­lated dis­con­tinu­it­ies be handled when mil­lions of para­met­ers are dif­fer­en­ti­ated sim­ul­tan­eously? In this pa­per, we provide a tax­onomy and ana­lys­is of dif­fer­ent es­tim­at­ors for dif­fer­en­tial light trans­port to provide in­tu­ition about these and re­lated ques­tions.



Text citation

Tizian Zeltner, Sébastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo Estimators for Differential Light Transport. In Transactions on Graphics (Proceedings of SIGGRAPH) 40(4).

    author = {Tizian Zeltner and Sébastien Speierer and Iliyan Georgiev and Wenzel Jakob},
    title = {Monte Carlo Estimators for Differential Light Transport},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH)},
    volume = {40},
    number = {4},
    year = {2021},
    month = aug,
    doi = {10.1145/3450626.3459807}