Synthetic Prior Design for Real-Time Face Tracking
Steven McDonagh (Disney Research Zurich)
Martin Klaudiny (Disney Research Zurich)
Derek Bradley (Disney Research Zurich)
Thabo Beeler (Disney Research Zurich)
Iain Matthews (Disney Research Pittsburgh)
Kenny Mitchell (Disney Research Zurich)
3D International Conference on 3D Vision (3DV) 2016
October 25, 2016
Real-time facial performance capture has recently been gaining popularity in virtual film production, driven by advances in machine learning, which allows for fast inference of facial geometry from video streams. These learning-based approaches are significantly influenced by the quality and amount of labelled training data. Tedious construction of training sets from real imagery can be replaced by rendering a facial animation rig under on-set conditions expected at runtime. We learn a synthetic actor-specific prior by adapting a state-of-the-art facial tracking method. Synthetic training significantly reduces the capture and annotation burden and in theory allows generation of an arbitrary amount of data. But practical realities such as training time and compute resources still limit the size of any training set. We construct better and smaller training sets by investigating which facial image appearances are crucial for tracking accuracy, covering the dimensions of expression, viewpoint, and illumination. A reduction of training data in 1-2 orders of magnitude is demonstrated whilst tracking accuracy is retained for challenging on-set footage.
Download File "Synthetic Prior Design for Real-Time Face Tracking-Paper"
[pdf, 8.91 MB]