Realistic Graphics Lab

Path Replay Backpropagation: Differentiating Light Paths using Constant Memory and Linear Time

To appear in Transactions on Graphics (Proceedings of SIGGRAPH 2021)

In­verse re­con­struc­tion of a scene with com­plex light­ing and het­ero­gen­eous struc­ture. Giv­en the ini­tial­iz­a­tion (a), we seek to re­con­struct the tar­get (b) in­volving nor­mal-mapped sur­face vari­ation and rough­ness changes on the fish sculp­ture, and the ad­di­tion of a plant based on tri­an­gu­lar geo­metry. Us­ing three rendered views of the tar­get, we ap­ply our pro­posed path re­play back­propaga­tion (PRB) (c) and a lin­ear-time ver­sion of ra­di­at­ive back­propaga­tion (RB) [Ni­mi­er-Dav­id et al. 2020] (d) to re­con­struct the mod­i­fied sculp­ture and a het­ero­gen­eous me­di­um ap­prox­im­at­ing the plant. Our meth­od com­putes un­biased gradi­ents and is able to con­verge to a high­er-qual­ity solu­tion at equal time. The second and third rows show in­sets and PRB's con­ver­gence over time.


Dif­fer­en­ti­able phys­ic­ally-based ren­der­ing has be­come an in­dis­pens­able tool for solv­ing in­verse prob­lems in­volving light. Most ap­plic­a­tions in this area jointly op­tim­ize a large set of scene para­met­ers to min­im­ize an ob­ject­ive func­tion, in which case re­verse-mode dif­fer­en­ti­ation is the meth­od of choice for ob­tain­ing para­met­er gradi­ents. However, ex­ist­ing tech­niques that per­form the ne­ces­sary dif­fer­en­ti­ation step suf­fer from either stat­ist­ic­al bi­as or a pro­hib­it­ive cost in terms of memory and com­pu­ta­tion time. For ex­ample, stand­ard tech­niques for auto­mat­ic dif­fer­en­ti­ation based on pro­gram trans­form­a­tion or Wengert tapes lead to im­prac­tic­ably large memory us­age when ap­plied to phys­ic­ally-based ren­der­ing al­gorithms. A re­cently pro­posed ad­joint meth­od by Ni­mi­er-Dav­id et al. 2020 re­duces this to a con­stant memory foot­print, but the com­pu­ta­tion time for un­biased gradi­ent es­tim­ates then be­comes quad­rat­ic in the num­ber of scat­ter­ing events along a light path. This is prob­lem­at­ic when the scene con­tains highly scat­ter­ing ma­ter­i­als like par­ti­cip­at­ing me­dia. In this pa­per, we pro­pose a new un­biased back­propaga­tion al­gorithm for ren­der­ing that only re­quires con­stant memory, and whose com­pu­ta­tion time is lin­ear in the num­ber of scat­ter­ing events (i.e., just like path tra­cing). Our ap­proach builds on the in­vert­ib­il­ity of the loc­al Jac­obi­an at scat­ter­ing in­ter­ac­tions to re­cov­er the vari­ous quant­it­ies needed for re­verse-mode dif­fer­en­ti­ation. Our meth­od also ex­tends to spec­u­lar ma­ter­i­als such as smooth dielec­trics and con­duct­ors that can­not be handled by pri­or work.



Text citation

Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2021. Path Replay Backpropagation: Differentiating Light Paths using Constant Memory and Linear Time. In Transactions on Graphics (Proceedings of SIGGRAPH) 40(4). 108:1–108:14.

    author = {Delio Vicini and Sébastien Speierer and Wenzel Jakob},
    title = {Path Replay Backpropagation: Differentiating Light Paths using Constant Memory and Linear Time},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH)},
    volume = {40},
    number = {4},
    pages = {108:1--108:14},
    year = {2021},
    month = aug,
    doi = {10.1145/3450626.3459804}