Phys­ic­ally based dif­fer­en­ti­able ren­der­ing has re­cently evolved in­to a power­ful tool for solv­ing in­verse prob­lems in­volving light. Meth­ods in this area per­form a dif­fer­en­ti­able sim­u­la­tion of the phys­ic­al pro­cess of light trans­port and scat­ter­ing to es­tim­ate par­tial de­riv­at­ives re­lat­ing scene para­met­ers to pixels in the rendered im­age. To­geth­er with gradi­ent-based op­tim­iz­a­tion, such al­gorithms have in­ter­est­ing ap­plic­a­tions in di­verse dis­cip­lines, e.g., to im­prove the re­con­struc­tion of 3D scenes, while ac­count­ing for in­ter­re­flec­tion and trans­par­ency, or to design meta-ma­ter­i­als with spe­cified op­tic­al prop­er­ties.

The most ver­sat­ile dif­fer­en­ti­able ren­der­ing al­gorithms rely on re­verse-mode dif­fer­en­ti­ation to com­pute all re­ques­ted de­riv­at­ives at once, en­abling op­tim­iz­a­tion of scene de­scrip­tions with mil­lions of free para­met­ers. However, a severe lim­it­a­tion of the re­verse-mode ap­proach is that it re­quires a de­tailed tran­script of the com­pu­ta­tion that is sub­sequently re­played to back-propag­ate de­riv­at­ives to the scene para­met­ers. The tran­script of typ­ic­al ren­der­ings is ex­tremely large, ex­ceed­ing the avail­able sys­tem memory by many or­ders of mag­nitude, hence cur­rent meth­ods are lim­ited to simple scenes rendered at low res­ol­u­tions and sample counts.

We in­tro­duce ra­di­at­ive back­propaga­tion, a fun­da­ment­ally dif­fer­ent ap­proach to dif­fer­en­ti­able ren­der­ing that does not re­quire a tran­script, greatly im­prov­ing its scalab­il­ity and ef­fi­ciency. Our main in­sight is that re­verse-mode propaga­tion through a ren­der­ing al­gorithm can be in­ter­preted as the solu­tion of a con­tinu­ous trans­port prob­lem in­volving the par­tial de­riv­at­ive of ra­di­ance with re­spect to the op­tim­iz­a­tion ob­ject­ive. This quant­ity is “emit­ted” by sensors, “scattered” by the scene, and even­tu­ally “re­ceived” by ob­jects with dif­fer­en­ti­able para­met­ers. Dif­fer­en­ti­able ren­der­ing then de­com­poses in­to two sep­ar­ate prim­al and ad­joint sim­u­la­tion steps that scale to com­plex scenes rendered at high res­ol­u­tions. We also in­vest­ig­ated biased vari­ants of this al­gorithm and find that they con­sid­er­ably im­prove both runtime and con­ver­gence speed. We show­case an ef­fi­cient GPU im­ple­ment­a­tion of ra­di­at­ive back­propaga­tion and com­pare its per­form­ance and the qual­ity of its gradi­ents to pri­or work.

Code: as of Mit­suba 3.0.0, Path Re­play Back­propaga­tion, an al­gorithm im­prov­ing on Ra­di­at­ive Back­propaga­tion, is the de­fault meth­od used to com­pute scene gradi­ents. It is im­ple­men­ted in the prb (sur­faces) and prbvolpath (volumes) in­teg­rat­or plu­gins.

The ori­gin­al re­search code and scene data for Ra­di­at­ive Back­propaga­tion are avail­able for down­load be­low. Please note that this im­ple­ment­a­tion is ex­per­i­ment­al and un­sup­por­ted. It is based on an older ver­sion of Mit­suba 2, which uses Op­tiX 6.

• Code (55.3 MB, last up­dated: 2021-08-31)
• Scene data (187MB, last up­dated: 2020-08-21)