RGL
Realistic Graphics Lab
EPFL Logo

Differentiable Signed Distance Function Rendering

In Transactions on Graphics (Proceedings of SIGGRAPH 2022)

Image-based shape and texture reconstruction of a statue given 32 (synthetic) reference images (a) and known environment illumination. We use differentiable rendering to jointly optimize a signed distance representation of the geometry and albedo texture by minimizing the L1 loss between the rendered and the reference images. Our method correctly accounts for discontinuities and we therefore do not require ad-hoc object mask or silhouette supervision. We visualize the reconstructed surface (b) and the re-rendered textured object (c). The view and illumination condition in (b) and (c) are different from the ones used during optimization. In (d) we render the ground truth triangle mesh.

Abstract

Phys­ic­ally-based dif­fer­en­ti­able ren­der­ing has re­cently emerged as an at­tract­ive new tech­nique for solv­ing in­verse prob­lems that re­cov­er com­plete 3D scene rep­res­ent­a­tions from im­ages. The in­ver­sion of shape para­met­ers is of par­tic­u­lar in­terest but also poses severe chal­lenges: shapes are in­ter­twined with vis­ib­il­ity, whose dis­con­tinu­ous nature in­tro­duces severe bi­as in com­puted de­riv­at­ives un­less costly pre­cau­tions are taken. Shape rep­res­ent­a­tions like tri­angle meshes suf­fer from ad­di­tion­al dif­fi­culties, since the con­tinu­ous op­tim­iz­a­tion of mesh para­met­ers can­not in­tro­duce to­po­lo­gic­al changes. One com­mon solu­tion to these dif­fi­culties en­tails rep­res­ent­ing shapes us­ing signed dis­tance func­tions (SD­Fs) and gradu­ally ad­apt­ing their zero level set dur­ing op­tim­iz­a­tion.

Pre­vi­ous dif­fer­en­ti­able ren­der­ing of SD­Fs did not fully ac­count for vis­ib­il­ity gradi­ents and re­quired the use of mask or sil­hou­ette su­per­vi­sion, or dis­cret­iz­a­tion in­to a tri­angle mesh. In this art­icle, we show how to ex­tend the com­monly used sphere tra­cing al­gorithm so that it ad­di­tion­ally out­puts a re­para­met­er­iz­a­tion that provides the means to com­pute ac­cur­ate shape para­met­er de­riv­at­ives. At a high level, this re­sembles tech­niques for dif­fer­en­ti­able mesh ren­der­ing, though we show that the SDF rep­res­ent­a­tion ad­mits a par­tic­u­larly ef­fi­cient re­para­met­er­iz­a­tion that out­per­forms pri­or work. Our ex­per­i­ments demon­strate the re­con­struc­tion of (syn­thet­ic) ob­jects without com­plex reg­u­lar­iz­a­tion or pri­ors, us­ing only a per-pixel RGB loss.

Video

Figures

Text citation

Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2022. Differentiable Signed Distance Function Rendering. In Transactions on Graphics (Proceedings of SIGGRAPH) 41(4). 125:1–125:18.

BibTeX
@article{Vicini2022sdf,
    author = {Delio Vicini and Sébastien Speierer and Wenzel Jakob},
    title = {Differentiable Signed Distance Function Rendering},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH)},
    volume = {41},
    number = {4},
    pages = {125:1--125:18},
    year = {2022},
    month = jul,
    doi = {10.1145/3528223.3530139}
}