Facial Motion Retargeting with Input-Output Temporal RBMs
Retargeting facial motion from one actor to another is a complex task. Current methods include: Blendshape Mapping, where target face is expressed as a combination of key shapes with the weight given by source data and Geometric Mapping, where expressions of the target face are modelled as a source face offset with respect to the base target geometry. These methods, however, do not take into account dynamic aspects of the facial motion itself, making it hard, for example, to model speech.
We present a type of Temporal Restricted Boltzmann Machine that defines a probability distribution over an output sequence conditional on an input sequence. It shares the desirable properties of Restricted Boltzmann Machines: efficient exact inference, an exponentially more expressive latent state than HMMs, and the ability to model nonlinear structure and dynamics. We apply our model to facial expression transfer. Results demonstrate improved performance over several baselines in 2D and 3D retargeting