Interactive Region-Based Linear 3D Face Models

Project Members

J.Rafael Tena (Disney Research Pittsburgh)
Fernando De la Torre (Carnegie Mellon University)
Iain Matthews (Disney Research Pittsburgh)

Linear models, particularly those based on principal component analysis (PCA), have been used successfully on a broad range of human face-related applications. Although PCA models achieve high compression, they have not been widely used for animation in a production environment because their bases lack a semantic interpretation. Their parameters are not a natural set for animators to work with. In this paper we present a linear face modelling approach that allows intuitive click-and-drag interaction for animation. Our model is composed of a collection of PCA sub-models that are independently trained but share boundaries. Boundary consistency and user-given constraints are enforced in a soft least mean squares sense to give flexibility to the model while maintaining coherence. Our results also show that the region-based model generalises better than its holistic counterpart when describing previously unseen motion capture data from multiple subjects. The decomposition of the face into several regions, which we determine automatically from training data, gives the user localised manipulation control. This allows the model to be used for face posing and animation in a natural and intuitive style.

Movie [QuickTime, 1280×720, 94.2MB]

Media Coverage

The Tartan, “Disney, CMU pioneer new motion capture software”
PostGazette.com, CMU teams with Disney to refine human-like Animation”
Times of India, “Researchers build 3D face models”
Kidscreen, “Researchers aim to change the face of animation”
Engadget, “Researchers demo 3D face scanning breakthroughs at SIGGRAPH, Kinect crowd squarely targeted”
Business-Standard, “Breakthrough in motion-capture tech”
The Engineer, “Method could lead to realistic facial expressions in animation”
PhysOrg, “Disney, CMU researchers build face models that give animators intuitive control of expressions”
Science Daily, “New Face Model Give Animators Intuitive Control of Expressions”

Publications

Interactive Region-Based Linear 3D Face Models
ACM Transactions on Graphics, Vol. 30, No. 4 (Proc. ACM SIGGRAPH 2011)
Paper File [pdf, 34.51 MB]

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.