Ren­der­ings of an­im­a­tion se­quences with phys­ics-based Monte Carlo light trans­port sim­u­la­tions are ex­ceed­ingly costly to gen­er­ate frame-by-frame, yet much of this com­pu­ta­tion is highly re­dund­ant due to the strong co­her­ence in space, time and among samples. A prom­ising ap­proach pur­sued in pri­or work en­tails sub­sampling the se­quence in space, time, and num­ber of samples, fol­lowed by im­age-based spa­tio-tem­por­al up­sampling and de­nois­ing. 

These meth­ods can provide sig­ni­fic­ant per­form­ance gains, though ma­jor is­sues re­main: first, in a mul­tiple scat­ter­ing sim­u­la­tion, the fi­nal pixel col­or is the com­pos­ite of many dif­fer­ent light trans­port phe­nom­ena, and this con­flict­ing in­form­a­tion causes ar­ti­facts in im­age-based meth­ods. Secondly, mo­tion vec­tors are needed to es­tab­lish cor­res­pond­ence between the pixels in dif­fer­ent frames, but it is un­clear how to ob­tain them for most kinds of light paths (e.g. an ob­ject seen through a curved glass pan­el). 

To re­duce these am­bi­gu­ities, we pro­pose a gen­er­al de­com­pos­i­tion frame­work, where the fi­nal pixel col­or is sep­ar­ated in­to com­pon­ents cor­res­pond­ing to dis­joint sub­sets of the space of light paths. Each com­pon­ent is ac­com­pan­ied by mo­tion vec­tors and oth­er aux­il­i­ary fea­tures such as re­flect­ance and sur­face nor­mals. The mo­tion vec­tors of spec­u­lar paths are com­puted us­ing a tem­por­al ex­ten­sion of man­i­fold ex­plor­a­tion and the re­main­ing com­pon­ents use a spe­cial­ized vari­ant of op­tic­al flow. Our ex­per­i­ments show that this de­com­pos­i­tion leads to sig­ni­fic­ant im­prove­ments in three im­age-based ap­plic­a­tions: de­nois­ing, spa­tial up­sampling, and tem­por­al in­ter­pol­a­tion.