Projective Sampling for Differentiable Rendering of Geometry

In Transactions on Graphics (Proceedings of SIGGRAPH Asia 2023)

The vis­ib­il­ity func­tion plays a cru­cial role in dif­fer­en­ti­able ren­der­ing of geo­metry. Para­met­er changes that in­flu­ence vis­ib­il­ity (e.g., a ro­ta­tion of the light source in (a) can gen­er­ate a sig­ni­fic­ant de­riv­at­ive con­tri­bu­tion shown in (b) that is dif­fi­cult to sample us­ing cur­rently ex­ist­ing meth­ods. We pro­ject path seg­ments gen­er­ated by prim­al ren­der­ing al­gorithms (e.g., dir­ect il­lu­min­a­tion sampling) onto nearby sil­hou­ettes and or­gan­ize them in a uni­form or ad­apt­ive guid­ing data struc­ture to en­hance nu­mer­ic­al in­teg­ra­tion of the chal­len­ging bound­ary term. Com­pared to pri­or work [Zhang et al. 2020], uni­form guid­ing cuts er­rors (RMSE) by an av­er­age of 8.1x, and ad­apt­ive guid­ing yields an ad­di­tion­al 2.7x im­prove­ment.


Dis­con­tinu­ous vis­ib­il­ity changes at ob­ject bound­ar­ies re­main a per­sist­ent source of dif­fi­culty in the area of dif­fer­en­ti­able ren­der­ing. Left un­treated, they bi­as com­puted gradi­ents so severely that even ba­sic op­tim­iz­a­tion tasks fail.

Pri­or path-space meth­ods ad­dressed this bi­as by de­coup­ling bound­ar­ies from the in­teri­or, al­low­ing each part to be handled us­ing spe­cial­ized Monte Carlo sampling strategies. While con­cep­tu­ally power­ful, the full po­ten­tial of this idea re­mains un­real­ized since ex­ist­ing meth­ods of­ten fail to ad­equately sample the bound­ary pro­por­tion­al to its con­tri­bu­tion.

This pa­per presents the­or­et­ic­al and al­gorithmic con­tri­bu­tions. On the the­or­et­ic­al side, we trans­form the bound­ary de­riv­at­ive in­to a re­mark­ably simple loc­al in­teg­ral that in­vites present and fu­ture de­vel­op­ments.

Build­ing on this res­ult, we pro­pose a new strategy that pro­jects or­din­ary samples pro­duced dur­ing for­ward ren­der­ing onto nearby bound­ar­ies. The res­ult­ing pro­jec­tions es­tab­lish a vari­ance-re­du­cing guid­ing dis­tri­bu­tion that ac­cel­er­ates con­ver­gence of the sub­sequent dif­fer­en­tial phase.

We demon­strate the su­per­i­or ef­fi­ciency and ver­sat­il­ity of our meth­od across a vari­ety of shape rep­res­ent­a­tions, in­clud­ing tri­angle meshes, im­pli­citly defined sur­faces, and cyl­indric­al fibers based on Bézi­er curves.



Text citation

Ziyi Zhang, Nicolas Roussel, and Wenzel Jakob. 2023. Projective Sampling for Differentiable Rendering of Geometry. In Transactions on Graphics (Proceedings of SIGGRAPH Asia) 42(6).

    author = {Ziyi Zhang and Nicolas Roussel and Wenzel Jakob},
    title = {Projective Sampling for Differentiable Rendering of Geometry},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH Asia)},
    volume = {42},
    number = {6},
    year = {2023},
    month = dec,
    doi = {10.1145/3618385}