Visual Computing Seminar (Spring 2016)
The Visual computing seminar is a new weekly seminar series on topics in Visual Computing.
Why: The motivation for creating this seminar is that EPFL has a critical mass of people who are working on subtly related topics in computational photography, computer graphics, geometry processing, human–computer interaction, computer vision and signal processing. Having a weekly point of interaction will provide exposure to interesting work in this area and increase awareness of our shared interests and other commonalities like the use of similar computational tools — think of this as the visual computing edition of the “Know thy neighbor” seminar series.
Who: The target audience are faculty, students and postdocs in the visual computing disciplines, but the seminar is open to anyone and guests are welcomed. There is no need to formally enroll in a course. The format is very flexible and will include 45 minute talks with Q&A, talks by external visitors, as well as shorter presentations. In particular, the seminar is also intended as a way for students to obtain feedback on shorter ~20min talks preceding a presentation at a conference. If you are a student or postdoc in one of the visual computing disciplines, you’ll probably receive email from me soon on scheduling a presentation.
Where and when: every Wednesday in BC02 (next to the ground floor atrium). Food is served at 11:50, and the actual talk starts at 12:15.
How to be notified: If you want to be kept up to date with announcements, please send me an email and I’ll put you on the list. If you are working in LCAV, CVLAB, IVRL, LGG, LSP, IIG, CHILI, LDM or RGL, you are automatically subscribed to future announcements, so there is nothing you need to do.
Instant Field-Aligned Meshes
I’ll introduce the seminar and will give a presentation about a technique to generate high quality meshes from many different kinds of input.
Abstract: We present a novel approach to remesh a surface into an isotropic triangular or quad-dominant mesh using a unified local smoothing operator that optimizes both the edge orientations and vertex positions in the output mesh. Our algorithm produces meshes with high isotropy while naturally aligning and snapping edges to sharp features. The method is simple to implement and parallelize, and it can process a variety of input surface representations, such as point clouds, range scans and triangle meshes. Our full pipeline executes instantly (less than a second) on meshes with hundreds of thousands of faces, enabling new types of interactive workflows. Since our algorithm avoids any global optimization, and its key steps scale linearly with input size, we are able to process extremely large meshes and point clouds, with sizes exceeding several hundred million elements. To demonstrate the robustness and effectiveness of our method, we apply it to hundreds of models of varying complexity.
High-contrast Computational Caustic Design (including demo with manufactured surfaces)
Abstract: We present a new algorithm for computational caustic design. Our algorithm solves for the shape of a transparent object such that the refracted light paints a desired caustic image on a receiver screen. We introduce an optimal transport formulation to establish a correspondence between the input geometry and the unknown target shape. A subsequent 3D optimization based on an adaptive discretization scheme then finds the target surface from the correspondence map. Our approach supports piecewise smooth surfaces and non-bijective mappings, which eliminates a number of shortcomings of previous methods. This leads to a significantly richer space of caustic images, including smooth transitions, singularities of infinite light density, and completely black areas. We demonstrate the effectiveness of our approach with several simulated and fabricated examples.
Shapes from Pixels
Abstract: Continuous-domain visual signals are usually captured as discrete (digital) images. This operation is not invertible in general, in the sense that the continuous-domain signal cannot be exactly reconstructed based on the discrete image, unless it satisfies certain constraints. In this talk, we focus on the problem of recovering binary shape images from a set of samples. First, we propose a scheme for sampling and reconstruction of shape images with finite rate of innovation (FRI). More specifically, we model the shape boundaries as a subset of an algebraic curve with an implicit bivariate polynomial. We show that the image parameters (i.e., the polynomial coefficients) are the solutions of a set of linear annihilation equations with the coefficients being the image moments. We then replace conventional 2D moments with more stable generalized moments that are adjusted to the given sampling kernel. This leads to successful reconstruction of shape images with moderate complexities from samples generated with realistic sampling kernels and in the presence of low to moderate noise levels.
The proposed FRI scheme falls short of reconstructing shape images with intricate geometries from realistic samples. In the second part of this talk, we propose a scheme for recovering shape images with smooth boundaries from a set of samples. The reconstructed image is constrained to regenerate the same samples (measurement consistency) as well as forming a bilevel image. We initially formulate the reconstruction technique by minimizing the shape perimeter over the set of consistent binary shapes. Next, we relax the non-convex shape constraint to transform the problem into minimizing the total variation over consistent non-negative-valued images. We also introduce a requirement (called reducibility) that guarantees equivalence between the two problems. We illustrate that the reducibility property effectively sets a requirement on the minimum sampling density. In this scheme, unlike FRI schemes, we do not constrain the boundary curves by any specific model. Instead, we let the sampling kernel and the sample values decide for them. As a result, there is less restriction on the achievable shape geometries.
Principled Parallel Mean-Field Inference for Discrete Random Fields
Who: Joint presentation by Pierre Baqué and Timur Bagautdinov.
Abstract: Mean-field variational inference is one of the most popular approaches to inference in discrete random fields. Standard mean-field optimization is based on coordinate descent and in many situations can be impractical. Thus, in practice, various parallel techniques are used, which either rely on ad-hoc smoothing with heuristically set parameters, or put strong constraints on the type of models. In this paper, we propose a novel proximal gradient-based approach to optimizing the variational objective. It is naturally parallelizable and easy to implement. We prove its convergence, and then demonstrate that, in practice, it yields faster convergence and often finds better optima than more traditional mean-field optimization techniques. Moreover, our method is less sensitive to the choice of parameters.
Image based relighting of cultural artifacts.
Abstract: By incorporating computational methods into the image acquisition pipeline, computational photography has opened up new avenues in the representation and visualization of real world objects in the digital world. Stained glass windows are a dynamic art form that change their appearance constantly, due to the ever-changing outdoor illumination. They are therefore, an exceptional candidate for virtual relighting. However, as they are anchored and very large in size, it is often impossible to sample their entire light transport with controlled illumination. We present a new acquisition and modeling framework for inverse rendering of stained glass windows, by exploiting sparsity inducing priors on the light transport. We show by experiments that our proposed solution preserves volume impurities under both controlled and uncontrolled, natural illuminations. We also present a matrix completion approach to synthesize glass slabs based on learnt priors. Finally, we present an easy-to-use, handheld acquisition framework to acquire relightable photographs of more general, reflective scenes such as paintings. Our acquisition system consists of two synchronised smartphones, one observing the scene while the other is used moved on an arbitrary trajectory by the user. We then propose a compressive sensing reconstruction of the sparsely sampled light transport, which inturn is used for scene relighting.
Beyond Developable: Computational Design and Fabrication with Auxetic Materials.
Abstract: We present a computational method for interactive 3D design and rationalization of surfaces via auxetic materials, i.e., flat flexible material that can stretch uniformly up to a certain extent. A key motivation for studying such material is that one can approximate doubly-curved surfaces (such as the sphere) using only flat pieces, making it attractive for fabrication. We physically realize surfaces by introducing cuts into approximately inextensible material such as sheet metal, plastic, or leather. The cutting pattern is modeled as a regular triangular linkage that yields hexagonal openings of spatially-varying radius when stretched. In the same way that isometry is fundamental to modeling developable surfaces, we leverage conformal geometry to understand auxetic design. In particular, we compute a global conformal map with bounded scale factor to initialize an otherwise intractable nonlinear optimization. We demonstrate that this global approach can handle non-trivial topology and non-local dependencies inherent in auxetic material. Design studies and physical prototypes are used to illustrate a wide range of possible applications.
From spectral radiation stimuli to full color images
Abstract: We describe the color reproduction workflow that enables converting intensity-tunable emitted or reflected spectral radiation stimuli into full color images. This workflow comprises the characterization of the tunable radiation stimuli, the establishment of a model enabling predicting the colors that are achieved by these tunable radiations, the mapping between colors defined in an input color space such as sRGB into colors achievable by these tunable radiations and the calculation of the intensity of these radiations in order to display a given color. As concrete examples, we consider the creation of fluorescent color images visible only under UV light as well as the creation of classical cyan, magenta and yellow prints.
Color Changing Prints on a Metallic Substrate
Abstract: We adapted the classic color-reproduction workflow in order to print color images on highly specular metallic sheets. Because of the strong specular reflection of the metallic prints, the resulting color images have a colorful appearance unmatched by conventional prints on paper.
Additionally, by optimizing both the diffusing white ink and colored inks halftones, we were able to independently control lightness under specular and under non-specular observation conditions. This allows us to hide patterns or grayscale shapes under one viewing condition, specular or non-specular, and reveal them under the other viewing condition.
Finally, the anisotropic line halftones printed on the metallic sheet change color upon in-plane rotation by 90o. This color change occurs due to the directional optical dot-gain effect. By analyzing this phenomenon and modelling it, we were able to create an innovative color reproduction workflow relying on a 6-dimensional spectral prediction model. This workflow enables creating at once two different images at the same position, one viewable without rotation and one viewable after 90° azimuthal rotation of the print.
For a short video please follow the link: http://actu.epfl.ch/news/rotate-an-image-another-one-appears/
Parallel Inverse Kinematics for Multithreaded Architectures
Abstract: In this talk I will present a parallel prioritized Jacobian based inverse kinematics algorithm for multi-threaded architectures. The approach solves damped least squares inverse kinematics using a parallel line search by identifying and sampling critical input parameters. Parallel competing execution paths are spawned for each parameter in order to select the optimum which minimizes the error criteria. The algorithm is highly scalable and can handle complex articulated bodies at interactive frame rates. The results are shown on complex skeletons consisting of more than 600 degrees of freedom while being controlled using multiple end effectors. We implement our algorithm both on multi-core and GPU architectures and demonstrate how the GPU can further exploit fine-grain parallelism not directly available on a multicore processor. The implementations are 10 - 150 times faster compared to state-of-art serial implementations while providing higher accuracy. We also demonstrate the scalability of the algorithm over multiple scenarios and explore the GPU implementation in detail.
Joint work with Pawan Harish, Mentar Mahmudi, Benoît Le Callennec and Ronan Boulic
Consistent Reconstruction in Multiple-View Geometry: Sampling Theory Meets Computer Vision
Where: note that on this day, the seminar takes place in INM 11 at the same time as usual.
Abstract: Consistent reconstruction is a very well-known concept in the signal processing community for sampling and reconstruction of different classes of signals. Consistent algorithms work by enforcing the constraint that the reconstructed signal, after sampling, leads to the same set of samples as the input of the reconstruction algorithm.
In this talk, We will investigate the idea of consistent reconstruction in the area of multiple-view geometry, especially the 3-D triangulation problem. We will analyse different state-of-the-art triangulation algorithms to see whether they satisfy consistency condition or not. Moreover, we show that, under certain conditions, consistent reconstruction can lead to an optimal decay in the reconstruction error rate with respect to the number of cameras.
Moreover, by applying consistent reconstruction, we can obtain a confidence area for the reconstructed result, using which we can efficiently attack some other problems in image processing, including multi-camera tracking in a reconfigurable camera set-up and optimal camera array design. I will briefly explain these directions as well as some results on real and synthetic datasets.
Spectral Filter Array based sensors
Abstract: Spectral Filter Arrays (SFA) is an emerging technology for multispectral image acquisition. SFAs make possible the use of a compact multispectral sensor to acquire still images or/and video. An introduction will present and compare typical multispectral and color image acquisition systems in order to clearly identify the underlying differences. The following will present the different aspects that may be taken into account while designing or using such sensor (filter sensitivity, energy balance, demosaicing, applications, etc.). Then I will present the characteristics of a prototype camera that we developed at the University of Bourgogne.
Bio: Jean-Baptiste Thomas received his Bachelor in Applied Physics in 2004 and his Master in Optics, Image and Vision in 2006, both from the Université Jean Monnet in France. He received his PhD from the University of Bourgogne in 2009. After a stay as a researcher at the Gjovik University College and then at the C2RMF (Centre de Recherche et de Restauration des Musées de France), he is now Maître de Conférences (Associate Professor) at the Université of Bourgogne. His research focuses on color science and on color and multi-spectral imaging.
Direct Prediction of 3D Body Poses from Motion Compensated Sequences
Abstract: We propose an efficient approach to exploiting motion information from consecutive frames of a video sequence to recover the 3D pose of people. Previous approaches typically compute candidate poses in individual frames and then link them in a post-processing step to resolve ambiguities. By contrast, we directly regress from a spatio-temporal volume of bounding boxes to a 3D pose in the central frame.
We further show that, for this approach to achieve its full potential, it is essential to compensate for the motion in consecutive frames so that the subject remains centered. This then allows us to effectively overcome ambiguities and improve upon the state-of-the-art by a large margin on the Human3.6m, HumanEva, and KTH Multiview Football 3D human pose estimation benchmarks.
What Players do with the Ball: A Physically Constrained Interaction Modeling
Abstract: Tracking the ball is critical for video-based analysis of team sports. However, it is difficult, especially in low resolution images, due to the small size of the ball, its speed that creates motion blur, and its often being occluded by players. In this paper, we propose a generic and principled approach to modeling the interaction between the ball and the players while also imposing appropriate physical constraints on the ball’s trajectory. We show that our approach, formulated in terms of a Mixed Integer Program, is more robust and more accurate than several state-of-the-art approaches on real-life volleyball, basketball, and soccer sequences.