If we could back­propag­ate de­riv­at­ives through a ren­der­ing al­gorithm, then it should be pos­sible to em­ploy some vari­ant of gradi­ent des­cent to run a ren­der­ing al­gorithm “in re­verse” and re­con­struct the world from im­ages. This turns out to be sur­pris­ingly hard: ren­der­ing al­gorithms are very big pro­grams, which makes naïve back­propaga­tion slow and costly. We de­vel­op al­gorithms that ex­ploit phys­ic­al laws to be faster.

We build math­em­at­ic­al mod­els and al­gorithms that cap­ture the visu­al rich­ness of the world. This in­volves ana­lyz­ing samples in RGL's state-of-the-art meas­ure­ment labor­at­ory and sim­u­lat­ing sur­face mi­cro­struc­ture along with the spec­trum and po­lar­iz­a­tion of light.

We de­vel­op com­pilers that can trans­form de­scrip­tions of ren­der­ing and dif­fer­en­ti­able ren­der­ing tasks in­to ef­fi­cient com­pu­ta­tion­al ker­nels for CPUs or GPUs with ray tra­cing hard­ware ac­cel­er­a­tion. Ob­tain­ing high per­form­ance re­quires ker­nel fu­sion, dif­fer­en­ti­ation, and spe­cial­ized op­tim­iz­a­tion passes.