Realistic Graphics Lab

Visu­al Com­put­ing Sem­in­ar (Spring 2020)

Food @ 11:50am,
Talk @ 12:15am
Wenzel Jakob

General information

The Visu­al com­put­ing sem­in­ar is a weekly sem­in­ar series on a vari­ety of top­ics in the broad­er area of Visu­al Com­put­ing.

Why: The mo­tiv­a­tion for cre­at­ing this sem­in­ar is that EPFL has a crit­ic­al mass of people who are work­ing on subtly re­lated top­ics in com­pu­ta­tion­al pho­to­graphy, com­puter graph­ics, geo­metry pro­cessing, hu­man–com­puter in­ter­ac­tion, com­puter vis­ion and sig­nal pro­cessing. Hav­ing a weekly point of in­ter­ac­tion will provide ex­pos­ure to in­ter­est­ing work in this area and in­crease aware­ness of our shared in­terests and oth­er com­mon­al­it­ies like the use of sim­il­ar com­pu­ta­tion­al tools — think of this as the visu­al com­put­ing edi­tion of the “Know thy neigh­bor” sem­in­ar series.

Who: The tar­get audi­ence are fac­ulty, stu­dents and postdocs in the visu­al com­put­ing dis­cip­lines, but the sem­in­ar is open to any­one and guests are wel­comed. There is no need to form­ally en­roll in a course. The format is very flex­ible and will in­clude 45 minute talks with Q&A, talks by ex­tern­al vis­it­ors, as well as short­er present­a­tions. In par­tic­u­lar, the sem­in­ar is also in­ten­ded as a way for stu­dents to ob­tain feed­back on short­er ~20min talks pre­ced­ing a present­a­tion at a con­fer­ence. If you are a stu­dent or postdoc in one of the visu­al com­put­ing dis­cip­lines, you’ll prob­ably re­ceive email from me soon on schedul­ing a present­a­tion.

BC 4<span class='ord'>th</span> floor map

Where and when: every Fri­day in BC410.  Food is served at 11:50, and the ac­tu­al talk starts at 12:15.

How to be no­ti­fied: If you want to be kept up to date with an­nounce­ments, please send me an email and I’ll put you on the list. If you are work­ing in LCAV, CVLAB, IVRL, LGG, LSP, IIG, CHILI, LDM or RGL, you are auto­mat­ic­ally sub­scribed to fu­ture an­nounce­ments, so there is noth­ing you need to do.
You may add the sem­in­ar events to Google Cal­en­dar (click the '+' but­ton in the bot­tom-right corner), or down­load the iC­al file.



Date Lecturer Contents
28.02.2020 Julian Panetta

Title: Com­pu­ta­tion­al Design of Metama­ter­i­als and De­ploy­able Struc­tures [prac­tice job talk]

Ab­stract: Mod­ern di­git­al fab­ric­a­tion tech­no­lo­gies like 3D print­ers and CNC ma­chines of­fer great power to pro­duce cus­tom­ized, finely-tuned phys­ic­al ob­jects with ex­cit­ing ap­plic­a­tions in ar­chi­tec­ture, medi­cine, ro­bot­ics, and more. However, search­ing the vast space of geo­met­ries man­u­fac­tur­able by these tech­no­lo­gies for the best solu­tion to a giv­en design prob­lem re­quires soph­ist­ic­ated com­pu­ta­tion­al design tools, and this is the main bot­tle­neck pre­vent­ing us from lever­aging these tech­no­lo­gies' full po­ten­tial.

My talk will present my work on in­verse design prob­lems for di­git­al fab­ric­a­tion: de­vel­op­ing al­gorithms that take users' high-level spe­cific­a­tions of de­sired func­tion­al­it­ies of a de­form­able ob­ject and solve for the op­tim­al man­u­fac­tur­able geo­metry. These prob­lems re­quire new al­gorithms for fast, ac­cur­ate, and dif­fer­en­ti­able phys­ic­al sim­u­la­tion and ro­bust nu­mer­ic­al op­tim­iz­a­tion to pre­dict the func­tion­al­it­ies of can­did­ate designs and de­term­ine how to im­prove them. The highly non­con­vex op­tim­iz­a­tion prob­lems in­volved also ne­ces­sit­ate new com­pu­ta­tion­al strategies to pro­duce good ini­tial guesses (which we de­vel­op us­ing in­sights from dif­fer­en­tial geo­metry) and to es­cape from loc­al op­tima. I will dis­cuss the soft­ware and al­gorithms I have cre­ated for op­tim­iz­a­tion-based in­verse design in the con­text of two spe­cif­ic ap­plic­a­tions: elast­ic metama­ter­i­als and de­ploy­able struc­tures.

Elast­ic metama­ter­i­als are fine-scale spa­tial ar­range­ments of fab­ric­a­tion ma­ter­i­als that can ex­hib­it prop­er­ties not found in any nat­ur­al ma­ter­i­al and achieve great­er strength-to-weight ra­tios than pure solids. Metama­ter­i­als have prom­ising ap­plic­a­tions in fields like pros­thet­ics, soft ro­bot­ics, and aerospace, and are a per­fect set­ting to de­vel­op new com­puter graph­ics and geo­met­ric mod­el­ing tech­niques: their prop­er­ties de­pend only on the shape and to­po­logy of the ma­ter­i­al ar­range­ment. De­ploy­able struc­tures are shape-shift­ing ob­jects that can trans­ition from a com­pact con­fig­ur­a­tion (ef­fi­cient to fab­ric­ate and trans­port) to a de­sired 3D shape. They have ap­plic­a­tions in many set­tings from emer­gency shel­ters to med­ic­al im­plants and satel­lite an­ten­nas.

06.03.2020 Jan Bednarík
Title: Shape Re­con­struc­tion by Learn­ing Dif­fer­en­ti­able Sur­face Rep­res­ent­a­tions
Ab­stract: Gen­er­at­ive mod­els that pro­duce point clouds have emerged as a power­ful tool to rep­res­ent 3D sur­faces, and the best cur­rent ones rely on learn­ing an en­semble of para­met­ric rep­res­ent­a­tions. Un­for­tu­nately, they of­fer no con­trol over the de­form­a­tions of the sur­face patches that form the en­semble and thus fail to pre­vent them from either over­lap­ping or col­lapsing in­to single points or lines. As a con­sequence, com­put­ing shape prop­er­ties such as sur­face nor­mals and curvatures be­comes dif­fi­cult and un­re­li­able. In this pa­per, we show that we can ex­ploit the in­her­ent dif­fer­en­ti­ab­il­ity of deep net­works to lever­age dif­fer­en­tial sur­face prop­er­ties dur­ing train­ing so as to pre­vent patch col­lapse and strongly re­duce patch over­lap. Fur­ther­more, this lets us re­li­ably com­pute quant­it­ies such as sur­face nor­mals and curvatures. We will demon­strate on sev­er­al tasks that this yields more ac­cur­ate sur­face re­con­struc­tions than the state-of-the-art meth­ods in terms of nor­mals es­tim­a­tion and amount of col­lapsed and over­lapped patches.


13.03.2020 Amir Zamir
Title: Per­cep­tion in the Ac­tion Loop
Ab­stract: Ar­ti­fi­cial In­tel­li­gence seeks agents that can per­ceive the world and act ac­cord­ingly. Des­pite re­mark­able pro­gress to­ward this goal, a fun­da­ment­al short­com­ing per­sists on the per­cep­tion front: dif­fi­culty in scal­ing to the com­plex­ity of the real world, and con­sequently, re­du­cing the op­er­a­tion do­main to per­cep­tu­ally sim­pli­fied ones (e.g. video games, con­trolled spaces, tab­letop ma­nip­u­la­tion scen­ari­os). I’ll talk about ef­forts to­ward a visu­al per­cep­tion that could ul­ti­mately scale to real-world com­plex­ity and sup­port the goals of act­ive agents by go­ing bey­ond isol­ated pat­tern re­cog­ni­tion prob­lems.
I’ll present a meth­od for tract­ably learn­ing a large set of per­cep­tion tasks us­ing trans­fer learn­ing (Taskonomy), to­ward form­ing a multi-task com­pos­i­tion­al per­cep­tion dic­tion­ary. I’ll show this dic­tion­ary can be turned in­to an in­ter­me­di­ate per­cep­tion mod­ule for act­ive ro­bot­ic agents (Mid-Level Vis­ion), en­abling them to im­prove their sample ef­fi­ciency and gen­er­al­iz­a­tion. This is ac­com­plished us­ing both real ro­bots as well as a vir­tu­al en­vir­on­ment rooted in real spaces (Gib­son En­vir­on­ment). I will con­clude with dis­cuss­ing cross-task con­sist­ency and quan­ti­fy­ing un­cer­tainty in per­cep­tu­al es­tim­a­tions (X-TaC).
27.03.2020 Benoît Guillard
Title: 3D sur­face re­con­struc­tion from im­age(s)
Ab­stract: Us­ing con­vo­lu­tion­al neur­al net­works for single-view re­con­struc­tion has re­cently be­come a prom­ising and trend­ing top­ic in geo­met­ric­al deep learn­ing. Presen­ted with an im­age of a shape, the task is to ac­cur­ately re­con­struct the vis­ible parts of the ob­ject, and to plaus­ibly hal­lu­cin­ate its un­seen por­tion re­ly­ing on a learned pri­or. Typ­ic­al ar­chi­tec­tures com­prise a CNN en­coder and a de­coder. Im­age en­coders map a 2D view to a vec­tor­ized lat­ent space, while the de­coders map a lat­ent vec­tor to a 3D shape (in the form of a point cloud, a mesh de­form­a­tion, the zero-cross­ing of an im­pli­cit func­tion…). To per­form well un­der dif­fer­ent view­points, the whole ar­chi­tec­ture has to im­pli­citly learn non trivi­al 3D geo­met­ric ma­nip­u­la­tions. This has been seen as a very lim­it­ing factor for gen­er­al­iz­a­tion to un­seen shapes and poses. We pro­pose to con­struct a re­gistered 3D lat­ent space, us­ing re­verse cam­era pro­jec­tions. A lat­ent vec­tor con­sists of a 3D grid aligned with the out­put ob­ject. 2D fea­ture maps and depth maps are pushed to 3D space, and the net­work is re­lieved of the bur­den of loc­al­iz­ing spa­tial fea­tures in 3D space. This also al­lows to geo­met­ric­ally fuse codes in the lat­ent space and more ac­cur­ately re­con­struct a sur­face from mul­tiple views. Our second con­tri­bu­tion is a hy­brid 3D de­coder, re­ly­ing both on voxels and point clouds. A re­l­at­ively coarse grid of oc­cu­pancy voxels first pre­dicts a low res­ol­u­tion ap­prox­im­a­tion of the de­sired sur­face. Then, with­in each ac­tiv­ated voxel, a 2D patch is dif­fer­en­ti­ably fol­ded to cap­ture high­er fre­quency de­tails and smooth­er curvatures. This hy­brid solu­tion ex­ploits both the good spa­tial­iz­a­tion of 3D con­vo­lu­tions and the sparsity of point clouds.
03.04.2020 Quentin Becker
Title: Gen­er­at­ive mod­els for solv­ing in­verse design prob­lems
Ab­stract: We present a frame­work that trains a deep gen­er­at­ive mod­el to provide mul­ti­far­i­ous solu­tions to a giv­en in­verse prob­lem. In­verse prob­lems ap­pear in sev­er­al en­gin­eer­ing tasks, where one tries to an­swer the ques­tion “How can I design a struc­ture (usu­ally this im­plies a choice of para­met­ers), to achieve a cer­tain tar­get per­form­ance?”. Cur­rently, we study this on the ex­ample of de­ploy­able beam net­works (X-Shells), which are fab­ric­ated in a flat con­fig­ur­a­tion, but can un­fold to a three di­men­sion­al shape. We pro­pose a vari­ant of the Gen­er­at­ive Ad­versari­al Nets (GAN) frame­work wherein the gen­er­at­or is trained to out­put high qual­ity cre­ations (e.g. X-Shells) with re­spect to a giv­en per­form­ance meas­ure (e.g. de­ploy­ab­il­ity, flat­ness in fab­ric­a­tion state, “beauty” of the de­ployed shape, ...). Since we have lim­ited in­sight in how to cre­ate ex­amples of good X-Shells, our train­ing data samples the space of such struc­tures in­suf­fi­ciently. Thus, we ad­apt our train­ing frame­work to rely only on for­ward sim­u­la­tion of gen­er­ated struc­tures and drop the use of a data­set. The first ver­sion of this frame­work is highly prone to severe mode-col­lapse, which is why we in­tro­duced a nov­el di­versity reg­u­lar­iz­a­tion term in­to the loss. In ad­di­tion to en­cour­aging di­versity among the cre­ations, this term seems to sta­bil­ize train­ing for GAN. We demon­strate these res­ults on a toy ex­ample, as the use of this frame­work for X-Shells is still a work in pro­gress.
24.04.2020 Tizian Zeltner
01.05.2020 Samara Ren
08.05.2020 Merlin Nimier-David
15.05.2020 Mengshi Qi
22.05.2020 Kaicheng Yu