Live Texturing of Augmented Reality Characters from Colored Drawings


Stéphane Magnenat (Disney Research Zurich)
Dat Tien Ngo (Computer Vision Lab, EPFL, Switzerland)
Fabio Zund (Disney Research Zurich)
Mattia Ryffel (Disney Research Zurich)
Gioacchino Noris (Disney Research Zurich)
Gerhard Röthlin (Disney Research Zurich)
Alessia Marra (Disney Research Zurich)
Maurizio Nitti (Disney Research Zurich)
Pascal Fua (Computer Vision Lab, EPFL, Switzerland)
Markus Gross (Disney Research Zurich)
Bob Sumner (Disney Research Zurich)

IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2015

September 29, 2015

Live Texturing of Augmented Reality Characters from Colored Drawings-Image

Coloring books capture the imagination of children and provide them with one of their earliest opportunities for creative expression. However, given the proliferation and popularity of digital devices, real-world activities like coloring can seem unexciting, and children become less engaged in them. Augmented reality holds unique potential to impact this situation by providing a bridge between real-world activities and digital enhancements. In this paper, we present an augmented reality coloring book App in which children color characters in a printed coloring book and inspect their work using a mobile device. The drawing is detected and tracked, and the video stream is augmented with an animated 3-D version of the character that is textured according to the child’s coloring. This is possible thanks to several novel technical contributions. We present a texturing process that applies the captured texture from a 2-D colored drawing to both the visible and occluded regions of a 3-D character in real time. We develop a deformable surface tracking method designed for colored drawings that uses a new outlier rejection algorithm for real-time tracking and surface deformation recovery. We present a content creation pipeline to efficiently create the 2-D and 3-D content. And, finally, we validate our work with two user studies that examine the quality of our texturing algorithm and the overall App experience.

Download File "Live Texturing of Augmented Reality Characters from Colored Drawings-Paper"
[pdf, 1.72 MB]

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.