RGL
Realistic Graphics Lab
EPFL Logo

Visu­al Com­put­ing Sem­in­ar (Spring 2020)

Friday
Food @ 11:50am,
Talk @ 12:15pm
Wenzel Jakob
Organizer

General information

The Visu­al com­put­ing sem­in­ar is a weekly sem­in­ar series on a vari­ety of top­ics in the broad­er area of Visu­al Com­put­ing.

Why: The mo­tiv­a­tion for cre­at­ing this sem­in­ar is that EPFL has a crit­ic­al mass of people who are work­ing on subtly re­lated top­ics in com­pu­ta­tion­al pho­to­graphy, com­puter graph­ics, geo­metry pro­cessing, hu­man–com­puter in­ter­ac­tion, com­puter vis­ion and sig­nal pro­cessing. Hav­ing a weekly point of in­ter­ac­tion will provide ex­pos­ure to in­ter­est­ing work in this area and in­crease aware­ness of our shared in­terests and oth­er com­mon­al­it­ies like the use of sim­il­ar com­pu­ta­tion­al tools — think of this as the visu­al com­put­ing edi­tion of the “Know thy neigh­bor” sem­in­ar series.

Who: The tar­get audi­ence are fac­ulty, stu­dents and postdocs in the visu­al com­put­ing dis­cip­lines, but the sem­in­ar is open to any­one and guests are wel­comed. There is no need to form­ally en­roll in a course. The format is very flex­ible and will in­clude 45 minute talks with Q&A, talks by ex­tern­al vis­it­ors, as well as short­er present­a­tions. In par­tic­u­lar, the sem­in­ar is also in­ten­ded as a way for stu­dents to ob­tain feed­back on short­er ~20min talks pre­ced­ing a present­a­tion at a con­fer­ence. If you are a stu­dent or postdoc in one of the visu­al com­put­ing dis­cip­lines, you’ll prob­ably re­ceive email from me soon on schedul­ing a present­a­tion.

BC 4<span class='ord'>th</span> floor map

Where and when: every Fri­day in BC410.  Food is served at 11:50, and the ac­tu­al talk starts at 12:15.

How to be no­ti­fied: If you want to be kept up to date with an­nounce­ments, please send me an email and I’ll put you on the list. If you are work­ing in LCAV, CVLAB, IVRL, LGG, LSP, IIG, CHILI, LDM or RGL, you are auto­mat­ic­ally sub­scribed to fu­ture an­nounce­ments, so there is noth­ing you need to do.
You may add the sem­in­ar events to Google Cal­en­dar (click the '+' but­ton in the bot­tom-right corner), or down­load the iC­al file.

 

Schedule

Date Lecturer Contents
28.02.2020 Julian Panetta

Title: Com­pu­ta­tion­al Design of Metama­ter­i­als and De­ploy­able Struc­tures [prac­tice job talk]

Ab­stract: Mod­ern di­git­al fab­ric­a­tion tech­no­lo­gies like 3D print­ers and CNC ma­chines of­fer great power to pro­duce cus­tom­ized, finely-tuned phys­ic­al ob­jects with ex­cit­ing ap­plic­a­tions in ar­chi­tec­ture, medi­cine, ro­bot­ics, and more. However, search­ing the vast space of geo­met­ries man­u­fac­tur­able by these tech­no­lo­gies for the best solu­tion to a giv­en design prob­lem re­quires soph­ist­ic­ated com­pu­ta­tion­al design tools, and this is the main bot­tle­neck pre­vent­ing us from lever­aging these tech­no­lo­gies' full po­ten­tial.

My talk will present my work on in­verse design prob­lems for di­git­al fab­ric­a­tion: de­vel­op­ing al­gorithms that take users' high-level spe­cific­a­tions of de­sired func­tion­al­it­ies of a de­form­able ob­ject and solve for the op­tim­al man­u­fac­tur­able geo­metry. These prob­lems re­quire new al­gorithms for fast, ac­cur­ate, and dif­fer­en­ti­able phys­ic­al sim­u­la­tion and ro­bust nu­mer­ic­al op­tim­iz­a­tion to pre­dict the func­tion­al­it­ies of can­did­ate designs and de­term­ine how to im­prove them. The highly non­con­vex op­tim­iz­a­tion prob­lems in­volved also ne­ces­sit­ate new com­pu­ta­tion­al strategies to pro­duce good ini­tial guesses (which we de­vel­op us­ing in­sights from dif­fer­en­tial geo­metry) and to es­cape from loc­al op­tima. I will dis­cuss the soft­ware and al­gorithms I have cre­ated for op­tim­iz­a­tion-based in­verse design in the con­text of two spe­cif­ic ap­plic­a­tions: elast­ic metama­ter­i­als and de­ploy­able struc­tures.

Elast­ic metama­ter­i­als are fine-scale spa­tial ar­range­ments of fab­ric­a­tion ma­ter­i­als that can ex­hib­it prop­er­ties not found in any nat­ur­al ma­ter­i­al and achieve great­er strength-to-weight ra­tios than pure solids. Metama­ter­i­als have prom­ising ap­plic­a­tions in fields like pros­thet­ics, soft ro­bot­ics, and aerospace, and are a per­fect set­ting to de­vel­op new com­puter graph­ics and geo­met­ric mod­el­ing tech­niques: their prop­er­ties de­pend only on the shape and to­po­logy of the ma­ter­i­al ar­range­ment. De­ploy­able struc­tures are shape-shift­ing ob­jects that can trans­ition from a com­pact con­fig­ur­a­tion (ef­fi­cient to fab­ric­ate and trans­port) to a de­sired 3D shape. They have ap­plic­a­tions in many set­tings from emer­gency shel­ters to med­ic­al im­plants and satel­lite an­ten­nas.

06.03.2020 Jan Bednarík
Title: Shape Re­con­struc­tion by Learn­ing Dif­fer­en­ti­able Sur­face Rep­res­ent­a­tions
 
Ab­stract: Gen­er­at­ive mod­els that pro­duce point clouds have emerged as a power­ful tool to rep­res­ent 3D sur­faces, and the best cur­rent ones rely on learn­ing an en­semble of para­met­ric rep­res­ent­a­tions. Un­for­tu­nately, they of­fer no con­trol over the de­form­a­tions of the sur­face patches that form the en­semble and thus fail to pre­vent them from either over­lap­ping or col­lapsing in­to single points or lines. As a con­sequence, com­put­ing shape prop­er­ties such as sur­face nor­mals and curvatures be­comes dif­fi­cult and un­re­li­able. In this pa­per, we show that we can ex­ploit the in­her­ent dif­fer­en­ti­ab­il­ity of deep net­works to lever­age dif­fer­en­tial sur­face prop­er­ties dur­ing train­ing so as to pre­vent patch col­lapse and strongly re­duce patch over­lap. Fur­ther­more, this lets us re­li­ably com­pute quant­it­ies such as sur­face nor­mals and curvatures. We will demon­strate on sev­er­al tasks that this yields more ac­cur­ate sur­face re­con­struc­tions than the state-of-the-art meth­ods in terms of nor­mals es­tim­a­tion and amount of col­lapsed and over­lapped patches.

 

13.03.2020 Amir Zamir
Title: Per­cep­tion in the Ac­tion Loop
 
Ab­stract: Ar­ti­fi­cial In­tel­li­gence seeks agents that can per­ceive the world and act ac­cord­ingly. Des­pite re­mark­able pro­gress to­ward this goal, a fun­da­ment­al short­com­ing per­sists on the per­cep­tion front: dif­fi­culty in scal­ing to the com­plex­ity of the real world, and con­sequently, re­du­cing the op­er­a­tion do­main to per­cep­tu­ally sim­pli­fied ones (e.g. video games, con­trolled spaces, tab­letop ma­nip­u­la­tion scen­ari­os). I’ll talk about ef­forts to­ward a visu­al per­cep­tion that could ul­ti­mately scale to real-world com­plex­ity and sup­port the goals of act­ive agents by go­ing bey­ond isol­ated pat­tern re­cog­ni­tion prob­lems.
I’ll present a meth­od for tract­ably learn­ing a large set of per­cep­tion tasks us­ing trans­fer learn­ing (Taskonomy), to­ward form­ing a multi-task com­pos­i­tion­al per­cep­tion dic­tion­ary. I’ll show this dic­tion­ary can be turned in­to an in­ter­me­di­ate per­cep­tion mod­ule for act­ive ro­bot­ic agents (Mid-Level Vis­ion), en­abling them to im­prove their sample ef­fi­ciency and gen­er­al­iz­a­tion. This is ac­com­plished us­ing both real ro­bots as well as a vir­tu­al en­vir­on­ment rooted in real spaces (Gib­son En­vir­on­ment). I will con­clude with dis­cuss­ing cross-task con­sist­ency and quan­ti­fy­ing un­cer­tainty in per­cep­tu­al es­tim­a­tions (X-TaC).
27.03.2020 Benoît Guillard
Title: 3D sur­face re­con­struc­tion from im­age(s)
 
Ab­stract: Us­ing con­vo­lu­tion­al neur­al net­works for single-view re­con­struc­tion has re­cently be­come a prom­ising and trend­ing top­ic in geo­met­ric­al deep learn­ing. Presen­ted with an im­age of a shape, the task is to ac­cur­ately re­con­struct the vis­ible parts of the ob­ject, and to plaus­ibly hal­lu­cin­ate its un­seen por­tion re­ly­ing on a learned pri­or. Typ­ic­al ar­chi­tec­tures com­prise a CNN en­coder and a de­coder. Im­age en­coders map a 2D view to a vec­tor­ized lat­ent space, while the de­coders map a lat­ent vec­tor to a 3D shape (in the form of a point cloud, a mesh de­form­a­tion, the zero-cross­ing of an im­pli­cit func­tion…). To per­form well un­der dif­fer­ent view­points, the whole ar­chi­tec­ture has to im­pli­citly learn non trivi­al 3D geo­met­ric ma­nip­u­la­tions. This has been seen as a very lim­it­ing factor for gen­er­al­iz­a­tion to un­seen shapes and poses. We pro­pose to con­struct a re­gistered 3D lat­ent space, us­ing re­verse cam­era pro­jec­tions. A lat­ent vec­tor con­sists of a 3D grid aligned with the out­put ob­ject. 2D fea­ture maps and depth maps are pushed to 3D space, and the net­work is re­lieved of the bur­den of loc­al­iz­ing spa­tial fea­tures in 3D space. This also al­lows to geo­met­ric­ally fuse codes in the lat­ent space and more ac­cur­ately re­con­struct a sur­face from mul­tiple views. Our second con­tri­bu­tion is a hy­brid 3D de­coder, re­ly­ing both on voxels and point clouds. A re­l­at­ively coarse grid of oc­cu­pancy voxels first pre­dicts a low res­ol­u­tion ap­prox­im­a­tion of the de­sired sur­face. Then, with­in each ac­tiv­ated voxel, a 2D patch is dif­fer­en­ti­ably fol­ded to cap­ture high­er fre­quency de­tails and smooth­er curvatures. This hy­brid solu­tion ex­ploits both the good spa­tial­iz­a­tion of 3D con­vo­lu­tions and the sparsity of point clouds.
03.04.2020 Quentin Becker
Title: Gen­er­at­ive mod­els for solv­ing in­verse design prob­lems
 
Ab­stract: We present a frame­work that trains a deep gen­er­at­ive mod­el to provide mul­ti­far­i­ous solu­tions to a giv­en in­verse prob­lem. In­verse prob­lems ap­pear in sev­er­al en­gin­eer­ing tasks, where one tries to an­swer the ques­tion “How can I design a struc­ture (usu­ally this im­plies a choice of para­met­ers), to achieve a cer­tain tar­get per­form­ance?”. Cur­rently, we study this on the ex­ample of de­ploy­able beam net­works (X-Shells), which are fab­ric­ated in a flat con­fig­ur­a­tion, but can un­fold to a three di­men­sion­al shape. We pro­pose a vari­ant of the Gen­er­at­ive Ad­versari­al Nets (GAN) frame­work wherein the gen­er­at­or is trained to out­put high qual­ity cre­ations (e.g. X-Shells) with re­spect to a giv­en per­form­ance meas­ure (e.g. de­ploy­ab­il­ity, flat­ness in fab­ric­a­tion state, “beauty” of the de­ployed shape, ...). Since we have lim­ited in­sight in how to cre­ate ex­amples of good X-Shells, our train­ing data samples the space of such struc­tures in­suf­fi­ciently. Thus, we ad­apt our train­ing frame­work to rely only on for­ward sim­u­la­tion of gen­er­ated struc­tures and drop the use of a data­set. The first ver­sion of this frame­work is highly prone to severe mode-col­lapse, which is why we in­tro­duced a nov­el di­versity reg­u­lar­iz­a­tion term in­to the loss. In ad­di­tion to en­cour­aging di­versity among the cre­ations, this term seems to sta­bil­ize train­ing for GAN. We demon­strate these res­ults on a toy ex­ample, as the use of this frame­work for X-Shells is still a work in pro­gress.
24.04.2020 Tizian Zeltner
Title: Spec­u­lar Man­i­fold Sampling for Ren­der­ing High-Fre­quency Caustics and Glints
 
Ab­stract: Weav­ing as a tra­di­tion­al craft has been used to ef­fi­ciently con­struct three-di­men­sion­al sur­faces from flat, straight rib­bons. A reg­u­lar weav­ing pat­tern usu­ally has in­ter­laced fam­il­ies of par­al­lel rib­bons. A clas­sic­al bas­ket weav­ing tech­nique is to in­sert to­po­lo­gic­al sin­gu­lar­it­ies in the pat­tern to pro­duce non­planar shapes. However, the weave pat­tern is dic­tated by the geo­metry of the tar­get shape un­der this tech­nique, and to­po­lo­gic­al sin­gu­lar­it­ies of­ten pro­duce con­cen­trated curvature. In this pro­ject, we in­tro­duce a dif­fer­ent ap­proach to con­trol the 3D equi­lib­ri­um shape of a woven struc­ture: weav­ing with rib­bons that are planar, but curved in their fab­ric­ated states. We pro­pose a sim­u­la­tion-based op­tim­iz­a­tion frame­work that solves for the in-plane rest curvature of each in­di­vidu­al rib­bon in the weave to pro­duce a rich vari­ety of double-curved shapes. We high­light the cap­ab­il­ity to pre­cisely ap­prox­im­ate com­plex free­form shapes without im­pos­ing con­straints on the weave pat­tern. We demon­strate the ef­fect­ive­ness of the curved rib­bon ap­proach and the per­form­ance of our op­tim­iz­a­tion frame­work us­ing sim­u­la­tion, phys­ic­al fab­ric­a­tion, and meas­ure­ments from mi­cro CT scans.
01.05.2020 Samara Ren
Title: Com­pu­ta­tion­al Design of Woven Struc­tures
 
Ab­stract: Weav­ing as a tra­di­tion­al craft has been used to ef­fi­ciently con­struct three-di­men­sion­al sur­faces from flat, straight rib­bons. A reg­u­lar weav­ing pat­tern usu­ally has in­ter­laced fam­il­ies of par­al­lel rib­bons. A clas­sic­al bas­ket weav­ing tech­nique is to in­sert to­po­lo­gic­al sin­gu­lar­it­ies in the pat­tern to pro­duce non­planar shapes. However, the weave pat­tern is dic­tated by the geo­metry of the tar­get shape un­der this tech­nique, and to­po­lo­gic­al sin­gu­lar­it­ies of­ten pro­duce con­cen­trated curvature.
 
In this pro­ject, we in­tro­duce a dif­fer­ent ap­proach to con­trol the 3D equi­lib­ri­um shape of a woven struc­ture: weav­ing with rib­bons that are planar, but curved in their fab­ric­ated states. We pro­pose a sim­u­la­tion-based op­tim­iz­a­tion frame­work that solves for the in-plane rest curvature of each in­di­vidu­al rib­bon in the weave to pro­duce a rich vari­ety of double-curved shapes. We high­light the cap­ab­il­ity to pre­cisely ap­prox­im­ate com­plex free­form shapes without im­pos­ing con­straints on the weave pat­tern. We demon­strate the ef­fect­ive­ness of the curved rib­bon ap­proach and the per­form­ance of our op­tim­iz­a­tion frame­work us­ing sim­u­la­tion, phys­ic­al fab­ric­a­tion, and meas­ure­ments from mi­cro CT scans.
08.05.2020 Merlin Nimier-David
Title: Ra­di­at­ive Back­propaga­tion: An Ad­joint Meth­od for Light­ning-Fast Dif­fer­en­ti­able Ren­der­ing
 
Ab­stract: Phys­ic­ally based dif­fer­en­ti­able ren­der­ing has re­cently evolved in­to a power­ful tool for solv­ing in­verse prob­lems in­volving light. Meth­ods in this area per­form a dif­fer­en­ti­able sim­u­la­tion of the phys­ic­al pro­cess of light trans­port and scat­ter­ing to es­tim­ate par­tial de­riv­at­ives re­lat­ing scene para­met­ers to pixels in the rendered im­age. To­geth­er with gradi­ent-based op­tim­iz­a­tion, such al­gorithms have in­ter­est­ing ap­plic­a­tions in di­verse dis­cip­lines, e.g., to im­prove the re­con­struc­tion of 3D scenes, while ac­count­ing for in­ter­re­flec­tion and trans­par­ency, or to design meta-ma­ter­i­als with spe­cified op­tic­al prop­er­ties.
 
We in­tro­duce Ra­di­at­ive Back­propaga­tion, a fun­da­ment­ally new ap­proach to dif­fer­en­ti­able ren­der­ing. Rather than re­ly­ing on auto­mat­ic dif­fer­en­ti­ation, we re­cast gradi­ent com­pu­ta­tion as a con­tinu­ous trans­port prob­lem, which can then be solved ef­fi­ciently. Un­like pre­vi­ous work, our meth­od scales to com­plex scenes rendered at high res­ol­u­tions. We demon­strate its ef­fi­ciency with a GPU im­ple­ment­a­tion which achieves up to three or­ders of mag­nitude spee­dup com­pared to Mit­suba 2.
15.05.2020 Mengshi Qi
Title: Im­it­at­ive Non-Autore­gress­ive Mod­el­ing for Tra­ject­ory Fore­cast­ing and Im­puta­tion
 
Ab­stract: Tra­ject­ory fore­cast­ing and im­puta­tion are pivotal steps to­wards un­der­stand­ing the move­ment of hu­man and ob­jects, which are quite chal­len­ging since the fu­ture tra­ject­or­ies and miss­ing val­ues in a tem­por­al se­quence are full of un­cer­tain­ties, and the spa­tial-tem­por­ally con­tex­tu­al cor­rel­a­tion is hard to mod­el. Yet, the rel­ev­ance between se­quence pre­dic­tion and im­puta­tion is dis­reg­arded by ex­ist­ing ap­proaches. To this end, we pro­pose a nov­el im­it­at­ive non-autore­gress­ive mod­el­ing meth­od to sim­ul­tan­eously handle the tra­ject­ory pre­dic­tion task and the miss­ing value im­puta­tion task. Spe­cific­ally, our frame­work ad­opts an im­it­a­tion learn­ing paradigm, which con­tains a re­cur­rent con­di­tion­al vari­ation­al au­toen­coder (RC-VAE) as a demon­strat­or, and a non-autore­gress­ive trans­form­a­tion mod­el (NART) as a learner. By jointly op­tim­iz­ing the two mod­els, RC-VAE can pre­dict the fu­ture tra­ject­ory and cap­ture the tem­por­al re­la­tion­ship in the se­quence to su­per­vise the NART learner. As a res­ult, NART learns from the demon­strat­or and im­putes the miss­ing value in a non-autore­gress­ive strategy. We con­duct ex­tens­ive ex­per­i­ments on three pop­u­lar data­sets, and the res­ults show that our mod­el achieves state-of-the-art per­form­ance across all the data­sets.
22.05.2020 Kaicheng Yu
Title: How to train your su­per-net: An Ana­lys­is of Train­ing Heur­ist­ics in Weight-Shar­ing NAS
 
Ab­stract: Weight shar­ing prom­ises to make neur­al ar­chi­tec­ture search (NAS) tract­able even on com­mod­ity hard­ware. Ex­ist­ing meth­ods in this space rely on a di­verse set of heur­ist­ics to design and train the shared-weight back­bone net­work, a.k.a. the su­per-net. Since heur­ist­ics and hy­per­para­met­ers sub­stan­tially vary across dif­fer­ent meth­ods, a fair com­par­is­on between them can only be achieved by sys­tem­at­ic­ally ana­lyz­ing the in­flu­ence of these factors. In this pa­per, we there­fore provide a sys­tem­at­ic eval­u­ation of the heur­ist­ics and hy­per­para­met­ers that are fre­quently em­ployed by weight-shar­ing NAS al­gorithms. Our ana­lys­is un­cov­ers that some com­monly-used heur­ist­ics for su­per-net train­ing neg­at­ively im­pact the cor­rel­a­tion between su­per-net and stand-alone per­form­ance, and evid­ences the strong in­flu­ence of cer­tain hy­per­para­met­ers and ar­chi­tec­tur­al choices. Our code and ex­per­i­ments set a strong and re­pro­du­cible baseline that fu­ture works can build on.