Augmenting Physical Avatars Using Projector-Based Illumination
Amit Bermano (Disney Research Zurich)
Philipp Brüschweiler (Disney Research Zürich, ETH Zürich)
Anselm Grundhöfer (Disney Research Zurich)
Daisuke Iwai (Osaka University)
Bernd Bickel (Disney Research Zurich)
Markus Gross (Disney Research Zurich)
In this project, we propose a complete process for augmenting physical avatars using projector based illumination, signiﬁcantly increasing their expressiveness. Given an input animation, the system decomposes the motion into low-frequency motion that can be physically reproduced by the animatronic head and high-frequency details that are added using projected shading. At the core is a spatio-temporal optimization process that compresses the motion in gradient space, ensuring faithful motion replay while respecting the physical limitations of the system. We also propose a complete multi-camera and projection system, including a novel defocused projection and subsurface scattering compensation scheme. The result of our system is a highly expressive physical avatar that features facial details and motion otherwise unreachable due to physical constraints.