RGL
EPFL Logo

Projective Sampling for Differentiable Rendering of Geometry

In Transactions on Graphics (Proceedings of SIGGRAPH Asia 2023)

The vis­ib­il­ity func­tion plays a cru­cial role in dif­fer­en­ti­able ren­der­ing of geo­metry. Para­met­er changes that in­flu­ence vis­ib­il­ity (e.g., a ro­ta­tion of the light source in (a) can gen­er­ate a sig­ni­fic­ant de­riv­at­ive con­tri­bu­tion shown in (b) that is dif­fi­cult to sample us­ing cur­rently ex­ist­ing meth­ods. We pro­ject path seg­ments gen­er­ated by prim­al ren­der­ing al­gorithms (e.g., dir­ect il­lu­min­a­tion sampling) onto nearby sil­hou­ettes and or­gan­ize them in a uni­form or ad­apt­ive guid­ing data struc­ture to en­hance nu­mer­ic­al in­teg­ra­tion of the chal­len­ging bound­ary term. Com­pared to pri­or work [Zhang et al. 2020], uni­form guid­ing cuts er­rors (RMSE) by an av­er­age of 8.1x, and ad­apt­ive guid­ing yields an ad­di­tion­al 2.7x im­prove­ment.

Abstract

Dis­con­tinu­ous vis­ib­il­ity changes at ob­ject bound­ar­ies re­main a per­sist­ent source of dif­fi­culty in the area of dif­fer­en­ti­able ren­der­ing. Left un­treated, they bi­as com­puted gradi­ents so severely that even ba­sic op­tim­iz­a­tion tasks fail.

Pri­or path-space meth­ods ad­dressed this bi­as by de­coup­ling bound­ar­ies from the in­teri­or, al­low­ing each part to be handled us­ing spe­cial­ized Monte Carlo sampling strategies. While con­cep­tu­ally power­ful, the full po­ten­tial of this idea re­mains un­real­ized since ex­ist­ing meth­ods of­ten fail to ad­equately sample the bound­ary pro­por­tion­al to its con­tri­bu­tion.

This pa­per presents the­or­et­ic­al and al­gorithmic con­tri­bu­tions. On the the­or­et­ic­al side, we trans­form the bound­ary de­riv­at­ive in­to a re­mark­ably simple loc­al in­teg­ral that in­vites present and fu­ture de­vel­op­ments.

Build­ing on this res­ult, we pro­pose a new strategy that pro­jects or­din­ary samples pro­duced dur­ing for­ward ren­der­ing onto nearby bound­ar­ies. The res­ult­ing pro­jec­tions es­tab­lish a vari­ance-re­du­cing guid­ing dis­tri­bu­tion that ac­cel­er­ates con­ver­gence of the sub­sequent dif­fer­en­tial phase.

We demon­strate the su­per­i­or ef­fi­ciency and ver­sat­il­ity of our meth­od across a vari­ety of shape rep­res­ent­a­tions, in­clud­ing tri­angle meshes, im­pli­citly defined sur­faces, and cyl­indric­al fibers based on Bézi­er curves.

Video

Figures

Text citation

Ziyi Zhang, Nicolas Roussel, and Wenzel Jakob. 2023. Projective Sampling for Differentiable Rendering of Geometry. In Transactions on Graphics (Proceedings of SIGGRAPH Asia) 42(6).

BibTeX
@article{Zhang2023Projective,
  author = {Zhang, Ziyi and Roussel, Nicolas and Jakob, Wenzel},
  title = {Projective Sampling for Differentiable Rendering of Geometry},
  year = {2023},
  issue_date = {December 2023},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  volume = {42},
  number = {6},
  issn = {0730-0301},
  url = {https://doi.org/10.1145/3618385},
  doi = {10.1145/3618385},
  journal = {ACM Trans. Graph.},
  month = dec,
  articleno = {212},
  numpages = {14},
  keywords = {differentiable rendering, geometry reconstruction}
}