FaceDirector: Continuous Control of Facial Performance in Video
Charles Malleson (Disney Research Zurich)
Jean-Charles Bazin (Disney Research Zurich)
Oliver Wang (Disney Research Zurich)
Derek Bradley (Disney Research Zurich)
Thabo Beeler (Disney Research Zurich)
Adrian Hilton (University of Surrey)
Alexander Sorkine-Hornung (Disney Research Zurich)
International Conference on Computer Vision (ICCV) 2015
December 11, 2015
We present a method to continuously blend between multiple facial performances of an actor, which can contain different facial expressions or emotional states. As an example, given sad and angry video takes of a scene, our method empowers a movie director to specify arbitrary weighted combinations and smooth transitions between the two takes in post-production. Our contributions include (1) a robust nonlinear audio-visual synchronization technique that exploits complementary properties of audio and visual cues to automatically determine robust, dense spatio-temporal correspondences between takes, and (2) a seamless facial blending approach that provides the director full control to interpolate timing, facial expression, and local appearance, in order to generate novel performances after filming. In contrast to most previous works, our approach operates entirely in image space, avoiding the need of 3D facial reconstruction. We demonstrate that our method can synthesize visually believable performances with applications in emotion transition, performance correction, and timing control.
Download File "FaceDirector- Continuous Control of Facial Performance in Video-Paper"
[pdf, 13.22 MB]