Realistic Graphics Lab

Visu­al Com­put­ing Sem­in­ar (Fall 2018)

Food @ 11:50am,
Talk @ 12:15am
Delio Vicini

General information

The Visu­al com­put­ing sem­in­ar is a weekly sem­in­ar series on top­ics in Visu­al Com­put­ing.

Why: The mo­tiv­a­tion for cre­at­ing this sem­in­ar is that EPFL has a crit­ic­al mass of people who are work­ing on subtly re­lated top­ics in com­pu­ta­tion­al pho­to­graphy, com­puter graph­ics, geo­metry pro­cessing, hu­man–com­puter in­ter­ac­tion, com­puter vis­ion and sig­nal pro­cessing. Hav­ing a weekly point of in­ter­ac­tion will provide ex­pos­ure to in­ter­est­ing work in this area and in­crease aware­ness of our shared in­terests and oth­er com­mon­al­it­ies like the use of sim­il­ar com­pu­ta­tion­al tools — think of this as the visu­al com­put­ing edi­tion of the “Know thy neigh­bor” sem­in­ar series.

Who: The tar­get audi­ence are fac­ulty, stu­dents and postdocs in the visu­al com­put­ing dis­cip­lines, but the sem­in­ar is open to any­one and guests are wel­comed. There is no need to form­ally en­roll in a course. The format is very flex­ible and will in­clude 45 minute talks with Q&A, talks by ex­tern­al vis­it­ors, as well as short­er present­a­tions. In par­tic­u­lar, the sem­in­ar is also in­ten­ded as a way for stu­dents to ob­tain feed­back on short­er ~20min talks pre­ced­ing a present­a­tion at a con­fer­ence. If you are a stu­dent or postdoc in one of the visu­al com­put­ing dis­cip­lines, you’ll prob­ably re­ceive email from me soon on schedul­ing a present­a­tion.

Where and when: every Wed­nes­day in BC01 (note the changed loc­a­tion!).  Food is served at 11:50, and the ac­tu­al talk starts at 12:15.

How to be no­ti­fied: If you want to be kept up to date with an­nounce­ments, please send me an email and I’ll put you on the list. If you are work­ing in LCAV, CVLAB, IVRL, LGG, LSP, IIG, CHILI, LDM or RGL, you are auto­mat­ic­ally sub­scribed to fu­ture an­nounce­ments, so there is noth­ing you need to do.
You may add the sem­in­ar events to Google Cal­en­dar (click the '+' but­ton in the bot­tom-right corner), or down­load the iC­al file.


Date Lecturer Contents
19.09.2018 Delio Vicini
03.10.2018 Cyrille Favreau

Blue Brain Brayns, A plat­form for high fi­del­ity large-scale and in­ter­act­ive visu­al­iz­a­tion of sci­entif­ic data and brain struc­tures

The Blue Brain Pro­ject has made ma­jor ef­forts to cre­ate mor­pho­lo­gic­ally ac­cur­ate neur­ons to sim­u­late sub-cel­lu­lar and elec­tric­al activ­it­ies, for ex­ample, mo­lecu­lar sim­u­la­tions of neur­on bio­chem­istry or multi-scale sim­u­la­tions of neur­on­al func­tion.

One of the keys to­wards un­der­stand­ing how the brain works as a whole, is visu­al­iz­a­tion of how the in­di­vidu­al cells func­tion. In par­tic­u­lar, the more mor­pho­lo­gic­ally ac­cur­ate the visu­al­iz­a­tion can be, the easi­er it is for ex­perts in the bio­lo­gic­al field to val­id­ate cell struc­tures; photo-real­ist­ic ren­der­ing is there­fore im­port­ant. Brayns is a visu­al­iz­a­tion plat­form that can in­ter­act­ively per­form high-qual­ity and high-fi­del­ity ren­der­ing of neur­os­cience

large data sets. Thanks to its cli­ent/serv­er ar­chi­tec­ture, Brayns can be run in the cloud as well as on a su­per­com­puter, and stream the ren­der­ing to any browser, either in a web UI or a Jupy­ter note­book.

At the Blue Brain pro­ject, the Visu­al­iz­a­tion team makes in­tens­ive use of Blue Brain Brayns to pro­duce ul­tra-high res­ol­u­tion movies (8K) and high-fi­del­ity im­ages for sci­entif­ic pub­lic­a­tions. Brayns is also used to serve im­mers­ive visu­al­iz­a­tion on the large dis­plays, as well as unique devices such as the curved Open­Deck loc­ated at the Blue Brain of­fice.

Brayns is also de­signed to ac­cel­er­ate sci­entif­ic visu­al­iz­a­tion, and to ad­apt to the large num­ber of en­vir­on­ments. Thanks to its mod­u­lar ar­chi­tec­ture, Brayns makes it easy to use vari­ous ren­der­ing back-ends such as In­tel's OS­PRay (CPU) or NVIDIA's Op­tiX for ex­ample. Every sci­entif­ic use-case such as DICOM, DTI, Blue Brain re­search, etc, is a stan­dalone plug-in that runs on top of Brayns, al­low­ing sci­ent­ists and re­searches to be­ne­fit from a high per­form­ance/fi­del­ity/qual­ity ren­der­ing sys­tem, without hav­ing to deal with the tech­nic­al com­plex­ity of it.

Brayns cur­rently im­ple­ments an num­ber of ba­sic prim­it­ives such as meshes, volumes, point clouds, para­met­ric geo­met­ries, and pi­on­eers new ren­der­ing mod­al­it­ies for sci­entif­ic visu­al­iz­a­tion, like signed dis­tance fields.
Dur­ing this talk, I will ex­plain the mo­tiv­a­tions be­hind the cre­ation of the Brayns plat­form, give some tech­nic­al in­sight about the ar­chi­tec­ture of the sys­tem and the vari­ous tech­niques that we already use to render data­sets. I will also de­scribe how new data­sets, as well as ren­der­ing com­pon­ents (en­gines, shaders, ma­ter­i­als, etc), can be ad­ded to the plat­form.

Links: ht­tps://git­hub.com/BlueBrain/Brayns

10.10.2018 Kaicheng Yu

Over­com­ing neur­al brain­wash­ing

We identi­fy a phe­nomen­on, which we dub neur­al brain­wash­ing, that oc­curs when se­quen­tially train­ing mul­tiple deep net­works with par­tially-shared para­met­ers; the per­form­ance of pre­vi­ously-trained mod­els de­grades as one op­tim­izes a sub­sequent one, due to the over­writ­ing of shared para­met­ers. To over­come this, we in­tro­duce a stat­ist­ic­ally-jus­ti­fied weight plas­ti­city loss that reg­u­lar­izes the learn­ing of a mod­el's shared para­met­ers ac­cord­ing to their im­port­ance for the pre­vi­ous mod­els, and demon­strate its ef­fect­ive­ness when train­ing two mod­els se­quen­tially and for neur­al ar­chi­tec­ture search. Adding weight plas­ti­city in neur­al ar­chi­tec­ture search pre­serves the best mod­els to the end of the search pro­cess lead­ing to im­proved res­ults in both nat­ur­al lan­guage pro­cessing and com­puter vis­ion tasks.


17.10.2018 Erhan Gündogdu

Good Fea­tures to Cor­rel­ate for Visu­al Track­ing

In this talk, I will mainly talk about visu­al ob­ject track­ing prob­lem which I worked on dur­ing my Ph.D. stud­ies. As a sec­ond­ary top­ic, I will men­tion my re­cent re­search activ­it­ies about gar­ment vir­tu­al­iz­a­tion by deep learn­ing.

Visu­al Track­ing
Es­tim­at­ing ob­ject mo­tion is one of the key com­pon­ents of video pro­cessing and the first step in ap­plic­a­tions which re­quire video rep­res­ent­a­tion. Visu­al ob­ject track­ing is one way of ex­tract­ing this com­pon­ent, and it is one of the ma­jor prob­lems in the field of com­puter vis­ion. Nu­mer­ous dis­crim­in­at­ive and gen­er­at­ive ma­chine learn­ing ap­proaches have been em­ployed to solve this prob­lem. Re­cently, cor­rel­a­tion fil­ter based (CFB) ap­proaches have been pop­u­lar due to their com­pu­ta­tion­al ef­fi­ciency and not­able per­form­ances on bench­mark data­sets. The ul­ti­mate goal of CFB ap­proaches is to find a fil­ter (i.e., tem­plate) which can pro­duce high cor­rel­a­tion out­puts around the ac­tu­al ob­ject loc­a­tion and low cor­rel­a­tion out­puts around the loc­a­tions that are far from the ob­ject. Nev­er­the­less, CFB visu­al track­ing meth­ods suf­fer from many chal­lenges, such as oc­clu­sion, ab­rupt ap­pear­ance changes, fast mo­tion and ob­ject de­form­a­tion. The main reas­ons of these suf­fer­ings are for­get­ting the past poses of the ob­jects due to the simple up­date stages of CFB meth­ods, non-op­tim­al mod­el up­date rate and fea­tures that are not in­vari­ant to ap­pear­ance changes of the tar­get ob­ject.
To ad­dress the afore­men­tioned dis­ad­vant­ages of CFB visu­al track­ing meth­ods, this work in­cludes three ma­jor con­tri­bu­tions. First, a spa­tial win­dow learn­ing meth­od is pro­posed to im­prove the cor­rel­a­tion qual­ity. For this pur­pose, a win­dow that is to be ele­ment-wise mul­ti­plied by the ob­ject ob­ser­va­tion (or the cor­rel­a­tion fil­ter) is learned by a nov­el gradi­ent des­cent pro­ced­ure. The learned win­dow is cap­able of sup­press­ing/high­light­ing the ne­ces­sary re­gions of the ob­ject, and can im­prove the track­ing per­form­ance in the case of oc­clu­sions and ob­ject de­form­a­tion. As the second con­tri­bu­tion, an en­semble of track­ers al­gorithm is pro­posed to handle the is­sues of non-op­tim­al learn­ing rate and for­get­ting the past poses of the ob­ject. The track­ers in the en­semble are or­gan­ized in a bin­ary tree, which stores in­di­vidu­al ex­pert track­ers at its nodes. Dur­ing the course of track­ing, the rel­ev­ant ex­pert track­ers to the most re­cent ob­ject ap­pear­ance are ac­tiv­ated and util­ized in the loc­al­iz­a­tion and up­date stages. The pro­posed en­semble meth­od sig­ni­fic­antly im­proves the track­ing ac­cur­acy, es­pe­cially when the ex­pert track­ers are se­lec­ted as the CFB track­ers util­iz­ing the pro­posed win­dow learn­ing meth­od. The fi­nal con­tri­bu­tion of this work ad­dresses the fea­ture learn­ing prob­lem spe­cific­ally fo­cused on the CFB visu­al track­ing loss func­tion. For this loss func­tion, a nov­el back­propaga­tion al­gorithm is de­veloped to train any fully deep con­vo­lu­tion­al neur­al net­work. The pro­posed gradi­ent cal­cu­la­tion, which is re­quired for back­propaga­tion, is per­formed ef­fi­ciently in both fre­quency and im­age do­main, and has a lin­ear com­plex­ity with the num­ber of fea­ture maps. The train­ing of the net­work mod­el is ful­filled on care­fully cur­ated data­sets in­clud­ing well-known dif­fi­culties of visu­al track­ing, e.g., oc­clu­sion, ob­ject de­form­a­tion and fast mo­tion. When the learned fea­tures are in­teg­rated to the state-of-the-art CFB visu­al track­ers, fa­vour­able track­ing per­form­ance is ob­tained on bench­mark data­sets against the CFB meth­ods that em­ploy hand-craf­ted fea­tures or deep fea­tures ex­trac­ted from the pre-trained clas­si­fic­a­tion mod­els.

Gar­ment sim­u­la­tion by deep learn­ing
Gar­ment sim­u­la­tion is a use­ful tool for vir­tu­al try-on, on­line shop­ping, gam­ing in­dustry, vir­tu­al real­ity and so forth. Real­ist­ic sim­u­la­tion of gar­ments on dif­fer­ent body shapes and poses by the help of a phys­ic­ally-based sim­u­la­tion (PBS) is a com­pu­ta­tion­ally heavy task which re­quires spe­cial para­met­er tun­ing for dif­fer­ent body shapes and mo­tion types. Hence, data-driv­en meth­ods that mod­el PBS ap­proaches for the fit­ted gar­ments on tar­get bod­ies are prefer­able for both com­pu­ta­tion­al con­cerns and gen­er­al­iz­a­tion pur­poses. Con­cretely, a PBS ap­proach with non-op­tim­al para­met­riz­a­tion can out­put sim­u­la­tion res­ults with un­desir­able cloth-body in­ter­pen­et­ra­tion. However, a data-driv­en mod­el such as a deep neur­al net­work can be trained by con­sid­er­ing ad­di­tion­al loss terms which will pre­vent in­ter­pen­et­ra­tion. Our meth­od presents a solu­tion for 3D gar­ment fit­ting on dif­fer­ent tar­get body shapes and poses without any post pro­cessing step such as cloth-body in­ter­pen­et­ra­tion, tight­ness, smooth­ing which are re­quired for PBS tools such as NvCloth. For the fore­see­able mis­takes of the learned mod­el, the con­straints are in­cluded in the train­ing loss func­tion of the pro­posed net­work mod­el. Hence, the net­work mod­el seam­lessly pre­dicts the fit­ted gar­ment giv­en the in­put tem­plate gar­ment and the tar­get body in a cer­tain pose.


24.10.2018 Krishna Kanth Nakka

Deep At­ten­tion­al Struc­tured Rep­res­ent­a­tion Learn­ing for Visu­al Re­cog­ni­tion

31.10.2018 Kevin Gonyop Kim

Ex­pand­ing ex­per­i­ence of the learners in vo­ca­tion­al edu­ca­tion

Vo­ca­tion­al edu­ca­tion and train­ing (VET) that takes place in dual con­texts of school and work­place is a well-es­tab­lished sec­ond­ary edu­ca­tion sys­tem in Switzer­land. Al­though it is known as an ef­fect­ive sys­tem for de­vel­op­ing vo­ca­tion­al com­pet­ence, there ex­ists some gap between what they are sup­posed to learn and what they prac­tice at work­places. The work­place ex­per­i­ences are usu­ally lim­ited to con­crete situ­ations in par­tic­u­lar en­vir­on­ments and their con­nec­tions to the gen­er­al know­ledge learned from schools are of­ten weak.
The cent­ral hy­po­thes­is of the Dual-T pro­ject is that di­git­al tech­no­lo­gies can serve as “bridges” over this school-work­place gap and en­hance the learn­ing ex­per­i­ences of the learners.  The goal of my re­search in this pro­ject is to design a way to ex­pand the work­place ex­per­i­ence so that the learner can ex­plore broad­er space of prac­tice. It is an ex­plor­at­ory re­search on how the learners in VET ac­cept the so­cially and syn­thet­ic­ally ex­pan­ded ex­per­i­ences and how they ex­plore them. In this present­a­tion, I will present some of the on­go­ing ap­plic­a­tions to flor­ist and garden­er ap­pren­tices as well as our pre­vi­ous work on lo­gist­ics and car­pentry.

07.11.2018 Peng Song

DESIA: A General Framework for Designing Interlocking Assemblies


Interlocking assemblies have a long history in the design of puzzles, furniture, architecture, and other complex geometric structures. The key defining property of interlocking assemblies is that all component parts are immobilized by their geometric arrangement, preventing the assembly from falling apart. Computer graphics research has recently contributed design tools that allow creating new interlocking assemblies. However, these tools focus on specific kinds of assemblies and explore only a limited space of interlocking configurations, which restricts their applicability for design.

In this talk, we present a new general framework for designing interlocking assemblies. The core idea is to represent part relationships with

a family of base Directional Blocking Graphs and leverage efficient graph analysis tools to compute an interlocking arrangement of parts. This avoids the exponential complexity of brute-force search. Our algorithm iteratively constructs the geometry of assembly components, taking advantage of all existing blocking relations for constructing successive parts. As a result, our approach supports a wider range of assembly forms compared to previous methods and provides significantly more design flexibility. We show that our framework facilitates efficient design of complex interlocking assemblies, including new solutions that cannot be achieved by state of the art approaches.

14.11.2018 Wenzel Jakob


21.11.2018 Edoardo Remelli


28.11.2018 Jan Bednarík

De­form­able sur­face re­con­struc­tion from a single view

05.12.2018 Zheng Dang


12.12.2018 Sena Kiciroglu

Active Drone Based Human Pose Estimation

19.12.2018 Helge Rhodin

Self-su­per­vised Rep­res­ent­a­tion Learn­ing for Hu­man Mo­tion Cap­ture