A Simple Approach to Differentiable Rendering of SDFs

Zichen Wang Ziyi Zhang Wenzel Jakob Steve Marschner

In Transactions on Graphics (Proceedings of SIGGRAPH Asia 2024)

The jump discontinuities at the visibility boundary B make physically based rendering non-differentiable. To explicitly sample and address such discontinuities, we propose to relax the visibility boundary to a thin band of relaxed boundary A. This corresponds to relaxing the surfaces to thin volumes for sampling discontinuities but retaining surface representations for forward rendering. Consequently, we are able to compute accurate gradients of the rendered images with respect to the scene parameters and, in turn, achieve high-quality inverse rendering results, with less complexity than previous methods.

Resources

Paper (15.4 MB)

Video (YouTube)

Publisher version

Cite

Abstract

We present a simple algorithm for differentiable rendering of surfaces represented by Signed Distance Fields (SDF), which makes it easy to integrate rendering into gradient-based optimization pipelines. To tackle visibility-related derivatives that make rendering non-differentiable, existing physically based differentiable rendering methods often rely on elaborate guiding data structures or reparameterization with a global impact on variance. In this article, we investigate an alternative that embraces nonzero bias in exchange for low variance and architectural simplicity. Our method expands the lower-dimensional boundary integral into a thin band that is easy to sample when the underlying surface is represented by an SDF. We demonstrate the performance and robustness of our formulation in end-to-end inverse rendering tasks, where it obtains results that are competitive with or superior to existing work.

Video

Text citation

Zichen Wang, Ziyi Zhang, Wenzel Jakob, and Steve Marschner. 2024. A Simple Approach to Differentiable Rendering of SDFs. In Transactions on Graphics (Proceedings of SIGGRAPH Asia) 43(6).

BibTeX

@article{Wang24Simple,
    author = {Zichen Wang and Ziyi Zhang and Wenzel Jakob and Steve Marschner},
    title = {A Simple Approach to Differentiable Rendering of SDFs},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH Asia)},
    volume = {43},
    number = {6},
    year = {2024},
    month = nov,
    doi = {10.1145/3680528.3687573}
}