Title: Ap­plic­a­tions of Phys­ic­ally-based Visu­al Sim­u­la­tion

*Note*: This talk is sched­uled on Wed­nes­day, but most of the oth­er talks dur­ing the semester will take place on the new time (Fri­day).

Ab­stract: Phys­ic­ally-based sim­u­la­tion plays an im­port­ant role for com­puter graph­ics. We can cre­ate highly real­ist­ic im­ages by sim­u­lat­ing op­tic­al in­ter­ac­tion between light and vir­tu­al ob­jects. Real­ist­ic an­im­a­tion of com­plex phe­nom­ena can be gen­er­ated by solv­ing their gov­ern­ing equa­tions such as Navi­er-Stokes equa­tions for flu­ids. Many of the ex­ist­ing re­searches fo­cus on the ac­cur­ate and ef­fi­cient sim­u­la­tion of ex­ist­ing phys­ic­al phe­nom­ena. Our re­search group fo­cuses on a dif­fer­ent dir­ec­tion; we 'use' phys­ic­ally-based sim­u­la­tion for dif­fer­ent pur­poses. In this talk, I will in­tro­duce some of our re­search res­ults, in­clud­ing in­verse visu­al sim­u­la­tion of clouds, phys­ic­ally-based aero­dy­nam­ic sound syn­thes­is, and in­verse op­tic­al sim­u­la­tion for di­git­al fab­ric­a­tion.

Title: Com­pu­ta­tion­al Design of Ro­bot­ic Char­ac­ters and Ar­chi­tec­tur­al-Scale Struc­tures

Ab­stract: With the emer­gence of mod­ern man­u­fac­tur­ing and con­struc­tion tech­no­lo­gies, we are wit­ness­ing a shift of the design com­plex­ity of ro­bots and struc­tures to­ward com­pu­ta­tion. With the aid of com­pu­ta­tion­al design tools, we can man­age this rap­idly grow­ing design com­plex­ity when cre­at­ing cus­tom ro­bot­ic char­ac­ters and ar­chi­tec­tur­al-scale struc­tures.  

In this talk, I will first high­light how com­pu­ta­tion paves the way to­ward ro­bots that are light­weight and in­ex­pens­ive, yet func­tion­al. I will dis­cuss how we can lever­age dif­fer­en­ti­able sim­u­la­tion and op­tim­iz­a­tion to design con­tinu­ously de­form­ing ro­bots made of bent wire, and an­im­ate very soft ro­bot­ic sys­tems while sup­press­ing vis­ible mech­an­ic­al os­cil­la­tions. Fi­nally, I will dis­cuss how an op­tim­iz­a­tion un­der worst-case loads en­ables the use of large-scale bind­er jet­ting at the ar­chi­tec­tur­al scale.

Bio: Mor­itz Bäch­er is a Re­search Sci­ent­ist at Dis­ney Re­search where he leads the Com­pu­ta­tion­al Design and Man­u­fac­tur­ing group. His re­search in­terests lie at the in­ter­sec­tion of com­puter graph­ics, and com­pu­ta­tion­al fab­ric­a­tion and ro­bot­ics. Be­fore join­ing Dis­ney, he re­ceived a Ph.D. from the Har­vard School of En­gin­eer­ing and Ap­plied Sci­ences and gradu­ated with a Mas­ters from ETH Zurich.

Title: A Study of Col­or Ren­der­ing in the In-Cam­era Ima­ging Pipeline

Ab­stract: Con­sumer cam­er­as such as di­git­al single-lens re­flex cam­era (DSLR) and smart­phone cam­er­as have on­board hard­ware that ap­plies a series of pro­cessing steps to trans­form the ini­tial cap­tured raw sensor im­age to the fi­nal out­put im­age that is provided to the user. These pro­cessing steps col­lect­ively make up the in-cam­era im­age pro­cessing pipeline. This dis­ser­ta­tion aims to study the pro­cessing steps re­lated to col­or ren­der­ing which can be cat­egor­ized in­to two stages. The first stage is to con­vert an im­age's sensor-spe­cif­ic raw col­or space to a device-in­de­pend­ent per­cep­tu­al col­or space. The second stage is to fur­ther pro­cess the im­age in­to a dis­play-re­ferred col­or space and in­cludes photo-fin­ish­ing routines to make the im­age ap­pear visu­ally pleas­ing to a hu­man. 

In this talk I will sum­mar­ize four con­tri­bu­tions to­wards the study of cam­era col­or ren­der­ing. The first con­tri­bu­tion is the de­vel­op­ment of a soft­ware-based re­search plat­form that closely emu­lates the in-cam­era im­age pro­cessing pipeline hard­ware. This plat­form al­lows the ex­am­in­a­tion of the vari­ous im­age states of the cap­tured im­age as it is pro­cessed from the sensor re­sponse to the fi­nal dis­play out­put. Our second con­tri­bu­tion is to demon­strate the ad­vant­age of hav­ing ac­cess to in­ter­me­di­ate im­age states with­in the in-cam­era pipeline that provide more ac­cur­ate col­or­i­met­ric con­sist­ency among mul­tiple cam­er­as. Our third con­tri­bu­tion is to ana­lyze the cur­rent col­or­i­met­ric meth­od used by con­sumer cam­er­as and to pro­pose a modi­fic­a­tion that is able to im­prove its col­or ac­cur­acy. Our fourth con­tri­bu­tion is to de­scribe how to cus­tom­ize a cam­era ima­ging pipeline us­ing ma­chine vis­ion cam­er­as to pro­duce high-qual­ity per­cep­tu­al im­ages for der­ma­to­lo­gic­al ap­plic­a­tions. The talk con­cludes with an over­all sum­mary. 

Title: Bey­ond Cartesian Rep­res­ent­a­tions for Loc­al Descriptors

Ab­stract: In the talk I'll give a brief in­tro­duc­tion to the loc­al descriptors, where and how they are used, de­scribe cur­rent ap­proaches and state-of-the art solu­tions. And present a nov­el and bet­ter way of patch sampling - ex­trac­tion of the “sup­port re­gion” dir­ectly with a log-po­lar sampling by sim­ul­tan­eously over­sampling the im­me­di­ate neigh­bour­hood of the point and un­der­sampling re­gions far away from it. This al­lows train­able mod­el to match a much wider range of scales than was pos­sible be­fore, and also lever­age much lar­ger sup­port re­gions without suf­fer­ing from oc­clu­sions. 

Title: Back­propaga­tion-Friendly Ei­gen­decom­pos­i­tion

Ab­stract: Ei­gen­decom­pos­i­tion (ED) is widely used in deep net­works. However, the back­propaga­tion of its res­ults tends to be nu­mer­ic­ally un­stable, wheth­er us­ing ED dir­ectly or ap­prox­im­at­ing it with the Power It­er­a­tion meth­od, par­tic­u­larly when deal­ing with large matrices. While this can be mit­ig­ated by par­ti­tion­ing the data in small and ar­bit­rary groups, do­ing so has no the­or­et­ic­al basis and makes its im­possible to ex­ploit the power of ED to the full. In this pa­per, we in­tro­duce a nu­mer­ic­ally stable and dif­fer­en­ti­able ap­proach to lever­aging ei­gen­vectors in deep net­works. It can handle large matrices without re­quir­ing to split them. We demon­strate the bet­ter ro­bust­ness of our ap­proach over stand­ard ED and PI for ZCA whiten­ing, an al­tern­at­ive to batch nor­mal­iz­a­tion, and for PCA de­nois­ing, which we in­tro­duce as a new nor­mal­iz­a­tion strategy for deep net­works, aim­ing to fur­ther de­noise the net­work's fea­tures.

Title: Mit­suba 2: A Re­tar­get­able For­ward and In­verse Ren­der­er

Ab­stract: Mod­ern ren­der­ing sys­tems are con­fron­ted with a daunt­ingly large and grow­ing set of re­quire­ments: in their pur­suit of real­ism, phys­ic­ally based tech­niques must in­creas­ingly ac­count for in­tric­ate prop­er­ties of light, such as its spec­tral com­pos­i­tion or po­lar­iz­a­tion. To re­duce pro­hib­it­ive ren­der­ing times, vec­tor­ized ren­der­ers ex­ploit co­her­ence via in­struc­tion-level par­al­lel­ism on CPUs and GPUs. Dif­fer­en­ti­able ren­der­ing al­gorithms propag­ate de­riv­at­ives through a sim­u­la­tion to op­tim­ize an ob­ject­ive func­tion, e.g., to re­con­struct a scene from ref­er­ence im­ages. Ca­ter­ing to such di­verse use cases is chal­len­ging and has led to nu­mer­ous pur­pose-built sys­tems—partly, be­cause ret­ro­fit­ting fea­tures of this com­plex­ity onto an ex­ist­ing ren­der­er in­volves an er­ror-prone and in­feas­ibly in­trus­ive trans­form­a­tion of ele­ment­ary data struc­tures, in­ter­faces between com­pon­ents, and their im­ple­ment­a­tions (in oth­er words, everything).

We pro­pose Mit­suba 2, a ver­sat­ile ren­der­er that is in­trins­ic­ally re­tar­get­able to vari­ous ap­plic­a­tions in­clud­ing the ones lis­ted above. Mit­suba 2 is im­ple­men­ted in mod­ern C++ and lever­ages tem­plate meta­pro­gram­ming to re­place types and in­stru­ment the con­trol flow of com­pon­ents such as BSD­Fs, volumes, emit­ters, and ren­der­ing al­gorithms. At com­pile time, it auto­mat­ic­ally trans­forms arith­met­ic, data struc­tures, and func­tion dis­patch, turn­ing gen­er­ic al­gorithms in­to a vari­ety of ef­fi­cient im­ple­ment­a­tions without the te­di­um of manu­al re­design. Pos­sible trans­form­a­tions in­clude chan­ging the rep­res­ent­a­tion of col­or, gen­er­at­ing a “wide” ren­der­er that op­er­ates on bundles of light paths, just-in-time com­pil­a­tion to cre­ate com­pu­ta­tion­al ker­nels that run on the GPU, and for­ward/re­verse-mode auto­mat­ic dif­fer­en­ti­ation. Trans­form­a­tions can be chained, which fur­ther en­riches the space of al­gorithms de­rived from a single gen­er­ic im­ple­ment­a­tion.

In this talk, I will present some of the key ar­chi­tec­tur­al fea­tures of Mit­suba 2's and demon­strate it in an in­ter­act­ive Py­thon ses­sion. A second talk will be giv­en on Mit­suba 2 on Decem­ber 6th by De­lio Vi­cini, fo­cus­ing on some of the ap­plic­a­tions.

Title: Re­para­met­er­iz­ing dis­con­tinu­ous in­teg­rands for dif­fer­en­ti­able ren­der­ing

Ab­stract: Dif­fer­en­ti­able ren­der­ing has re­cently opened the door to a num­ber of chal­len­ging in­verse prob­lems in­volving photoreal­ist­ic im­ages, such as com­pu­ta­tion­al ma­ter­i­al design and scat­ter­ing-aware re­con­struc­tion of geo­metry and ma­ter­i­als from pho­to­graphs. Dif­fer­en­ti­able ren­der­ing al­gorithms strive to es­tim­ate par­tial de­riv­at­ives of pixels in a rendered im­age with re­spect to scene para­met­ers, which is dif­fi­cult be­cause vis­ib­il­ity changes are in­her­ently non-dif­fer­en­ti­able.

We pro­pose a new tech­nique for dif­fer­en­ti­at­ing path-traced im­ages with re­spect to scene para­met­ers that af­fect vis­ib­il­ity, in­clud­ing the po­s­i­tion of cam­er­as, light sources, and ver­tices in tri­angle meshes. Our al­gorithm com­putes the gradi­ents of il­lu­min­a­tion in­teg­rals by ap­ply­ing changes of vari­ables that re­move or strongly re­duce the de­pend­ence of the po­s­i­tion of dis­con­tinu­it­ies on dif­fer­en­ti­able scene para­met­ers. The un­der­ly­ing para­met­er­iz­a­tion is cre­ated on the fly for each in­teg­ral and en­ables ac­cur­ate gradi­ent es­tim­ates us­ing stand­ard Monte Carlo sampling in con­junc­tion with auto­mat­ic dif­fer­en­ti­ation. Im­port­antly, our ap­proach does not rely on sampling sil­hou­ette edges, which has been a bot­tle­neck in pre­vi­ous work and tends to pro­duce high-vari­ance gradi­ents when im­port­ant edges are found with in­suf­fi­cient prob­ab­il­ity in scenes with com­plex vis­ib­il­ity and high-res­ol­u­tion geo­metry. We show that our meth­od only re­quires a few samples to pro­duce gradi­ents with low bi­as and vari­ance for chal­len­ging cases such as glossy re­flec­tions and shad­ows. Fi­nally, we use our dif­fer­en­ti­able path tracer to re­con­struct the 3D geo­metry and ma­ter­i­als of sev­er­al real-world ob­jects from a set of ref­er­ence pho­to­graphs.

Note: This week's sem­in­ar talk will start slightly later than usu­al, around 12:30.

Title: Design and Struc­tur­al Op­tim­iz­a­tion of To­po­lo­gic­al In­ter­lock­ing As­sem­blies

Ab­stract: We study as­sem­blies of con­vex ri­gid blocks reg­u­larly ar­ranged to ap­prox­im­ate a giv­en free­form sur­face. Our designs rely solely on the geo­met­ric ar­range­ment of blocks to form a stable as­sembly, neither re­quir­ing ex­pli­cit con­nect­ors or com­plex joints, nor re­ly­ing on fric­tion between blocks. The con­vex­ity of the blocks sim­pli­fies fab­ric­a­tion, as they can be eas­ily cut from dif­fer­ent ma­ter­i­als such as stone, wood, or foam. However, design­ing stable as­sem­blies is chal­len­ging, since ad­ja­cent pairs of blocks are re­stric­ted in their re­l­at­ive mo­tion only in the dir­ec­tion or­tho­gon­al to a single com­mon planar in­ter­face sur­face. We show that des­pite this weak in­ter­ac­tion, struc­tur­ally stable, and in some cases, glob­ally in­ter­lock­ing as­sem­blies can be found for a vari­ety of free­form designs. Our op­tim­iz­a­tion al­gorithm is based on a the­or­et­ic­al link between stat­ic equi­lib­ri­um con­di­tions and a geo­met­ric, glob­al in­ter­lock­ing prop­erty of the as­sembly-that an as­sembly is glob­ally in­ter­lock­ing if and only if the equi­lib­ri­um con­di­tions are sat­is­fied for ar­bit­rary ex­tern­al forces and torques. In­spired by this con­nec­tion, we define a meas­ure of sta­bil­ity that spans from single-load equi­lib­ri­um to glob­al in­ter­lock­ing, mo­tiv­ated by tilt ana­lys­is ex­per­i­ments used in struc­tur­al en­gin­eer­ing. We use this meas­ure to op­tim­ize the geo­metry of blocks to achieve a stat­ic equi­lib­ri­um for a max­im­al cone of dir­ec­tions, as op­posed to con­sid­er­ing only self-load scen­ari­os with a single grav­ity dir­ec­tion. In the lim­it, this op­tim­iz­a­tion can achieve glob­ally in­ter­lock­ing struc­tures. We show how dif­fer­ent geo­met­ric pat­terns give rise to a vari­ety of design op­tions and val­id­ate our res­ults with phys­ic­al pro­to­types.

Pro­ject page: ht­tps://lgg.ep­fl.ch/pub­lic­a­tions/2019/To­po­lo­gic­al_In­ter­lock­ing/in­dex.php

Title: Deep­Sphere: a graph-based spher­ic­al CNN

Ab­stract: Equivari­ance has emerged as a design prin­ciple for (Con­vo­lu­tion­al) Neur­al Net­works (CNNs) that al­lows to re­duce sample com­plex­ity and guar­an­tees gen­er­al­iz­a­tion by ex­ploit­ing sym­met­ries. For a spher­ic­al CNN, prop­er equivari­ance to ro­ta­tion (us­ing the spher­ic­al har­mon­ic trans­form) is com­pu­ta­tion­ally ex­pens­ive. A com­monly used scal­able al­tern­at­ive is to loc­ally ap­ply 2D CNNs, ex­ploit­ing the loc­al re­semb­lance of the man­i­fold to Eu­c­lidean space. This ap­proach however dis­cards the spher­ic­al geo­metry and is not equivari­ant. To al­low for a con­trol­lable bal­ance between equivari­ance and scalab­il­ity, we pro­pose in­stead to mod­el the sampled sphere as a graph of con­nec­ted pixels. Moreover, this rep­res­ent­a­tion nat­ur­ally ac­com­mod­ates non-uni­formly dis­trib­uted, par­tial, and chan­ging samplings, without in­ter­pol­a­tion. As the main chal­lenge is to design dis­crete op­er­a­tions that re­spect the un­der­ly­ing con­tinu­ous geo­metry, we show both the­or­et­ic­ally and em­pir­ic­ally how equivari­ance is af­fected by the con­struc­tion of the graph. The ap­plic­a­tion of Deep­Sphere to the re­cog­ni­tion of 3D ob­jects, the dis­crim­in­a­tion of cos­mo­lo­gic­al mod­els, and the seg­ment­a­tion of ex­treme events in cli­mate sim­u­la­tions yields state-of-the-art per­form­ance and demon­strates the ef­fi­ciency and flex­ib­il­ity of the meth­od.

Title: Deep­Wave, a re­cur­rent neur­al net­work for real-time acous­tic ima­ging

Ab­stract: Com­pu­ta­tion­al ima­ging refers to the prob­lem of re­con­struct­ing a source ob­ject from in­dir­ect evid­ence of the lat­ter ob­tained us­ing an ac­quis­i­tion device. The tra­di­tion­al ap­proach to solv­ing these ima­ging tasks is to for­mu­late a reg­u­lar­ized in­verse prob­lem that takes in­to ac­count the pre­cise form of the for­ward op­er­at­or and pri­or know­ledge of the ob­ject of in­terest to ob­tain good es­tim­ates. Re­cently however, Deep Learn­ing meth­ods have been placed at the fore­front of the field due to their ex­cel­lent per­form­ance and fixed ex­e­cu­tion times.

In this talk, I will present how one can lever­age do­main-spe­cif­ic know­ledge and dec­ades of know-how in con­vex op­tim­iz­a­tion to de­rive im­pli­citly-reg­u­lar­ized net­work ar­chi­tec­tures tailored to a par­tic­u­lar ima­ging task.  Spe­cific­ally, we con­sider the prob­lem of es­tim­at­ing the spher­ic­al sound field from an ar­ray of mi­cro­phone meas­ure­ments. We show how the struc­ture of the op­tim­iz­a­tion prob­lem nat­ur­ally gives rise to a re­cur­rent ar­chi­tec­ture, Deep­Wave, with a parsi­mo­ni­ous para­met­er­iz­a­tion that fully ex­ploits the nat­ur­al struc­ture present phased-ar­ray ima­ging. Un­like net­work ar­chi­tec­tures com­monly used in im­age re­con­struc­tion, Deep­Wave is moreover cap­able of dir­ectly pro­cessing the com­plex-val­ued raw mi­cro­phone cor­rel­a­tions, learn­ing how to op­tim­ally back-pro­ject these in­to a spher­ic­al map. Fi­nally, we pro­pose a smart phys­ic­ally-in­spired ini­tial­iz­a­tion scheme that al­lows much faster train­ing than ran­dom ini­tial­iz­a­tion. Our real-data ex­per­i­ments show Deep­Wave has sim­il­ar com­pu­ta­tion­al speed to the state-of-the-art delay-and-sum im­ager with vastly su­per­i­or res­ol­u­tion.

Pa­per pre-print: ht­tps://in­fos­cience.ep­fl.ch/re­cord/265765/files/pa­per.pdf

Title: Mesh Mod­el­ing to As­sist Seg­ment­a­tion in Volu­met­ric Data

Ab­stract: CNN-based volu­met­ric meth­ods that la­bel in­di­vidu­al voxels now dom­in­ate the field of bio­med­ic­al seg­ment­a­tion. In this pa­per, we show that sim­ul­tan­eously per­form­ing the seg­ment­a­tion and re­cov­er­ing a 3D mesh that mod­els the sur­face can boost per­form­ance. 
To this end, we pro­pose an end-to-end train­able two-stream en­coder/de­coder ar­chi­tec­ture. It com­prises a single en­coder and two de­coders, one that la­bels voxels and the oth­er out­puts the mesh. The key to suc­cess is that the two de­coders com­mu­nic­ate with each oth­er and help each oth­er learn. This goes bey­ond the well-known fact that train­ing a deep net­work to per­form two dif­fer­ent tasks im­proves its per­form­ance.

Title: Mit­suba 2: A Re­tar­get­able For­ward and In­verse Ren­der­er: Ex­ample Ap­plic­a­tions

Ab­stract: Mod­ern ren­der­ing sys­tems are con­fron­ted with a grow­ing set of com­plex re­quire­ments: in their pur­suit of real­ism, phys­ic­ally based tech­niques must in­creas­ingly ac­count for in­tric­ate prop­er­ties of light, such as its spec­tral com­pos­i­tion or po­lar­iz­a­tion. To re­duce pro­hib­it­ive ren­der­ing times, vec­tor­ized ren­der­ers ex­ploit co­her­ence via in­struc­tion-level par­al­lel­ism on CPUs and GPUs. Dif­fer­en­ti­able ren­der­ing al­gorithms propag­ate de­riv­at­ives through a sim­u­la­tion to op­tim­ize an ob­ject­ive func­tion, e.g., to re­con­struct a scene from ref­er­ence im­ages. Ca­ter­ing to such di­verse use cases is chal­len­ging and has led to nu­mer­ous pur­pose-built sys­tems—partly, be­cause ret­ro­fit­ting fea­tures of this com­plex­ity onto an ex­ist­ing ren­der­er in­volves an er­ror-prone and in­feas­ibly in­trus­ive trans­form­a­tion of ele­ment­ary data struc­tures, in­ter­faces between com­pon­ents, and their im­ple­ment­a­tions (in oth­er words, everything).We re­cently pro­posed Mit­suba 2 (presen­ted at Sig­graph Asia 2019), a ver­sat­ile ren­der­er that is in­trins­ic­ally re­tar­get­able to vari­ous ap­plic­a­tions in­clud­ing the ones lis­ted above.
Mit­suba 2 sup­ports po­lar­ized, spec­tral, vec­tor­ized and dif­fer­en­ti­able ren­der­ing in a single C++ code base with ex­tens­ive Py­thon bind­ings.
In a pre­vi­ous sem­in­ar present­a­tion, Mer­lin Ni­mi­er-Dav­id presen­ted and dis­cussed some of the key ar­chi­tec­tur­al fea­tures of Mit­suba 2.
In this present­a­tion, I will show­case sev­er­al ex­ample ap­plic­a­tions of Mit­suba 2.
I will show how Mit­suba 2 can be used solve com­plex in­verse prob­lems mo­tiv­ated by ap­plic­a­tions in com­puter vis­ion and di­git­al fab­ric­a­tion. We will dis­cuss ex­amples such as caustic design, volume re­con­struc­tion and tex­ture re­pro­duc­tion for 3D print­ing.

Title: In­flat­ables: A New De­ploy­able Sur­face Struc­ture

Ab­stract: I will present our work on a new type of de­ploy­able sur­face struc­ture that trans­forms from a flat, eas­ily fab­ric­ated sheet to a curved 3D sur­face simply by in­flat­ing. Our “in­flat­ables” con­sist of two sheets of ma­ter­i­al that are fused to­geth­er along a pat­tern of curves to form pock­ets. When in­flated with air, these pock­ets ex­pand in­to tubes, in­du­cing trans­verse con­trac­tions of the sheet that force it to pop in­to a 3D shape. We use tools from dif­fer­en­tial geo­metry to un­der­stand this shape trans­form­a­tion and to ap­prox­im­ately solve the in­verse design prob­lem: giv­en a tar­get sur­face, we gen­er­ate a pat­tern of fus­ing curves so that the res­ult­ing in­flated struc­ture closely re­sembles it. We then use an ac­cur­ate phys­ic­al sim­u­la­tion of the in­fla­tion pro­cess to as­sess how well the tar­get sur­face is ap­prox­im­ated and run a shape op­tim­iz­a­tion on the fus­ing curves to bet­ter fit the tar­get. I will show sim­u­lated in­fla­tions for a vari­ety of sur­faces we are able to pro­duce with this meth­od, along with some fab­ric­ated ex­amples.

Title: Es­tim­at­ing People Flows to Bet­ter Count them in Crowded Scenes

Ab­stract: State-of-the-art meth­ods for count­ing people in crowded scenes rely on deep net­works to es­tim­ate people dens­it­ies in in­di­vidu­al im­ages. As such, only very few take ad­vant­age of tem­por­al con­sist­ency in video se­quences,  and those that do only im­pose weak smooth­ness con­straints across con­sec­ut­ive frames. 
 
In this pa­per, we show that es­tim­at­ing people flows across im­age loc­a­tions between con­sec­ut­ive im­ages and in­fer­ring the people dens­it­ies from these flows in­stead of dir­ectly re­gress­ing them makes it pos­sible to im­pose much stronger con­straints en­cod­ing the con­ser­va­tion of the num­ber of people, which sig­ni­fic­antly boost per­form­ance without re­quir­ing a more com­plex ar­chi­tec­ture. Fur­ther­more, it also en­ables us to ex­ploit the cor­rel­a­tion between people flow and op­tic­al flow to fur­ther im­prove the res­ults. 
 
We will demon­strate that we con­sist­ently out­per­form state-of-the-art meth­ods on five bench­mark data­sets.