RGL
EPFL Logo

Projective Sampling for Differentiable Rendering of Geometry

In Transactions on Graphics (Proceedings of SIGGRAPH Asia 2023)

The visibility function plays a crucial role in differentiable rendering of geometry. Parameter changes that influence visibility (e.g., a rotation of the light source in (a) can generate a significant derivative contribution shown in (b) that is difficult to sample using currently existing methods. We project path segments generated by primal rendering algorithms (e.g., direct illumination sampling) onto nearby silhouettes and organize them in a uniform or adaptive guiding data structure to enhance numerical integration of the challenging boundary term. Compared to prior work [Zhang et al. 2020], uniform guiding cuts errors (RMSE) by an average of 8.1x, and adaptive guiding yields an additional 2.7x improvement.

Abstract

Discontinuous visibility changes at object boundaries remain a persistent source of difficulty in the area of differentiable rendering. Left untreated, they bias computed gradients so severely that even basic optimization tasks fail.

Prior path-space methods addressed this bias by decoupling boundaries from the interior, allowing each part to be handled using specialized Monte Carlo sampling strategies. While conceptually powerful, the full potential of this idea remains unrealized since existing methods often fail to adequately sample the boundary proportional to its contribution.

This paper presents theoretical and algorithmic contributions. On the theoretical side, we transform the boundary derivative into a remarkably simple local integral that invites present and future developments.

Building on this result, we propose a new strategy that projects ordinary samples produced during forward rendering onto nearby boundaries. The resulting projections establish a variance-reducing guiding distribution that accelerates convergence of the subsequent differential phase.

We demonstrate the superior efficiency and versatility of our method across a variety of shape representations, including triangle meshes, implicitly defined surfaces, and cylindrical fibers based on Bézier curves.

Video

Figures

Text citation

Ziyi Zhang, Nicolas Roussel, and Wenzel Jakob. 2023. Projective Sampling for Differentiable Rendering of Geometry. In Transactions on Graphics (Proceedings of SIGGRAPH Asia) 42(6).

BibTeX
@article{Zhang2023Projective,
  author = {Zhang, Ziyi and Roussel, Nicolas and Jakob, Wenzel},
  title = {Projective Sampling for Differentiable Rendering of Geometry},
  year = {2023},
  issue_date = {December 2023},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  volume = {42},
  number = {6},
  issn = {0730-0301},
  url = {https://doi.org/10.1145/3618385},
  doi = {10.1145/3618385},
  journal = {ACM Trans. Graph.},
  month = dec,
  articleno = {212},
  numpages = {14},
  keywords = {differentiable rendering, geometry reconstruction}
}