Mitsuba 2: A Retargetable Forward and Inverse Renderer

In Transactions on Graphics (Proceedings of SIGGRAPH Asia 2019)

Four ap­plic­a­tions en­abled us­ing auto­mated trans­form­a­tions of a gen­er­ic ren­der­er. (a) Po­lar­ized spec­tral ren­der­ing of an op­tic­al ex­per­i­ment that ana­lyzes light with el­lipt­ic­al po­lar­iz­a­tion. (b) A co­her­ent MCMC ren­der­ing al­gorithm that ex­plores bundles of nearby light paths to im­prove con­ver­gence at equal render time. (c) A re­fract­ive slab op­tim­ized by in­verse ren­der­ing to fo­cus light with three primary col­ors in­to a rendi­tion of the paint­ing A Sunday Af­ter­noon on the Is­land of La Grande Jatte by Georges Seur­at. (d) Re­con­struc­tion of a smoke volume from ref­er­ence im­ages; mul­tiple it­er­a­tions of the op­tim­iz­a­tion are shown. Please refer to the sup­ple­ment­al video for an­im­ated visu­al­iz­a­tions of many res­ults shown in this pa­per.


Mod­ern ren­der­ing sys­tems are con­fron­ted with a daunt­ingly large and grow­ing set of re­quire­ments: in their pur­suit of real­ism, phys­ic­ally based tech­niques must in­creas­ingly ac­count for in­tric­ate prop­er­ties of light, such as its spec­tral com­pos­i­tion or po­lar­iz­a­tion. To re­duce pro­hib­it­ive ren­der­ing times, vec­tor­ized ren­der­ers ex­ploit co­her­ence via in­struc­tion-level par­al­lel­ism on CPUs and GPUs. Dif­fer­en­ti­able ren­der­ing al­gorithms propag­ate de­riv­at­ives through a sim­u­la­tion to op­tim­ize an ob­ject­ive func­tion, e.g., to re­con­struct a scene from ref­er­ence im­ages. Ca­ter­ing to such di­verse use cases is chal­len­ging and has led to nu­mer­ous pur­pose-built sys­tems—partly, be­cause ret­ro­fit­ting fea­tures of this com­plex­ity onto an ex­ist­ing ren­der­er in­volves an er­ror-prone and in­feas­ibly in­trus­ive trans­form­a­tion of ele­ment­ary data struc­tures, in­ter­faces between com­pon­ents, and their im­ple­ment­a­tions (in oth­er words, everything).

We pro­pose Mit­suba 2, a ver­sat­ile ren­der­er that is in­trins­ic­ally re­tar­get­able to vari­ous ap­plic­a­tions in­clud­ing the ones lis­ted above. Mit­suba 2 is im­ple­men­ted in mod­ern C++ and lever­ages tem­plate meta­pro­gram­ming to re­place types and in­stru­ment the con­trol flow of com­pon­ents such as BSD­Fs, volumes, emit­ters, and ren­der­ing al­gorithms. At com­pile time, it auto­mat­ic­ally trans­forms arith­met­ic, data struc­tures, and func­tion dis­patch, turn­ing gen­er­ic al­gorithms in­to a vari­ety of ef­fi­cient im­ple­ment­a­tions without the te­di­um of manu­al re­design. Pos­sible trans­form­a­tions in­clude chan­ging the rep­res­ent­a­tion of col­or, gen­er­at­ing a “wide” ren­der­er that op­er­ates on bundles of light paths, just-in-time com­pil­a­tion to cre­ate com­pu­ta­tion­al ker­nels that run on the GPU, and for­ward/re­verse-mode auto­mat­ic dif­fer­en­ti­ation. Trans­form­a­tions can be chained, which fur­ther en­riches the space of al­gorithms de­rived from a single gen­er­ic im­ple­ment­a­tion.

We demon­strate the ef­fect­ive­ness and sim­pli­city of our ap­proach on sev­er­al ap­plic­a­tions that would be very chal­len­ging to cre­ate without as­sist­ance: a ren­der­ing al­gorithm based on co­her­ent MCMC ex­plor­a­tion, a caustic design meth­od for gradi­ent-in­dex op­tics, and a tech­nique for re­con­struct­ing het­ero­gen­eous me­dia in the pres­ence of mul­tiple scat­ter­ing.


Text citation

Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: A Retargetable Forward and Inverse Renderer. In Transactions on Graphics (Proceedings of SIGGRAPH Asia) 38(6).

    author = {Merlin Nimier-David and Delio Vicini and Tizian Zeltner and Wenzel Jakob},
    title = {Mitsuba 2: A Retargetable Forward and Inverse Renderer},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH Asia)},
    volume = {38},
    number = {6},
    year = {2019},
    month = dec,
    doi = {10.1145/3355089.3356498}