Facial Performance Enhancement Using Dynamic Shape Space Analysis

Project Members

Amit Bermano (Disney Research Zurich)
Derek Bradley (Disney Research Zurich)
Thabo Beeler (Disney Research Zurich)
Fabio Zund (Disney Research Zurich)
Derek Nowrouzezahrai (Disney Research Zurich)
Ilya Baran (Disney Research Zurich)
Olga Sorkine-Hornung (ETH Zurich)
Hanspeter Pfister (Harvard University)
Bob Sumner (Disney Research Zurich)
Bernd Bickel (Disney Research Zurich)
Markus Gross (Disney Research Zurich)

 

PROJECT_FacialPerformanceEnhancement_teaser

Given a low-resolution art-directed facial animation (a facial rig, in this case, shown as wireframe), we augment the performance to enhance the subtle details of expression particular to an individual’s face (shown as a surface and textured).

The facial performance of an individual is inherently rich in subtle deformation and timing details. Although these subtleties make the performance realistic and compelling, they often elude both motion capture and hand animation. We present a technique for adding fine-scale details and expressiveness to low-resolution art-directed facial performances, such as those created manually using a rig, via marker-based capture, by fitting a morphable model to a video, or through Kinect reconstruction using recent faceshift technology. We employ a high-resolution facial performance capture system to acquire a representative performance of an individual in which he or she explores the full range of facial expressions. From the captured data, our system extracts an expressiveness model that encodes subtle spatial and temporal deformation details specific to that particular individual. Once this model has been built, these details can be transferred to low-resolution art-directed performances. We demonstrate results on various forms of input; after our enhancement, the resulting animations exhibit the same nuances and fine spatial details as the captured performance, with optional temporal enhancement to match the dynamics of the actor. Finally, we show experimentally that our technique compares favorably to the current state-of-the-art in example-based facial animation



Publications

PROJECT_Facial-Performance-Enhancement_ACMTOG2013_THUMBNAIL

Facial Performance Enhancement Using Dynamic Shape Space Analysis
October 29, 2013
ACM Transactions On Graphics (ACM TOG) 2013
Paper File [pdf, 62.47 MB]

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.