We present a meth­od for re­du­cing er­rors—vari­ance and bi­as—in phys­ic­ally based dif­fer­en­ti­able ren­der­ing (PBDR). Typ­ic­al ap­plic­a­tions of PBDR re­peatedly render a scene as part of an op­tim­iz­a­tion loop in­volving gradi­ent des­cent. The ac­tu­al change in­tro­duced by each gradi­ent des­cent step is of­ten re­l­at­ively small, caus­ing a sig­ni­fic­ant de­gree of re­dund­ancy in this com­pu­ta­tion. We ex­ploit this re­dund­ancy by for­mu­lat­ing a gradi­ent es­tim­at­or that em­ploys a re­curs­ive con­trol vari­ate, which lever­ages in­form­a­tion from pre­vi­ous op­tim­iz­a­tion steps. The con­trol vari­ate re­duces vari­ance in gradi­ents, and, per­haps more im­port­antly, al­le­vi­ates is­sues that arise from dif­fer­en­ti­at­ing loss func­tions with re­spect to noisy in­puts, a com­mon cause of drift to bad loc­al min­ima or di­ver­gent op­tim­iz­a­tions. We ex­per­i­ment­ally eval­u­ate our ap­proach on a vari­ety of path-traced scenes con­tain­ing sur­face and volumes and ob­serve that prim­al ren­der­ing ef­fi­ciency im­proves by a factor of up to 10.