Feature Flow (Practical Temporal Consistency)

Project Members

Aljoscha Smolic (Disney Research Zurich)
Manuel Lang (Disney Research Zurich/ETH Zurich)
Oliver Wang (Disney Research Zurich)
Tunç Aydın (Disney Research Zurich)
Markus Gross (Disney Research Zurich)

 

PROJECT_FeatureFlow_teaser

Sparse feature correspondences are computed in a video sequence (top left), along with user strokes (bottom left). Both pieces of information are efficiently propagated using our spatiotemporal filtering approach, yielding temporally consistent optical flow (right, top), and a dense propagation of the user strokes (right, bottom).

We present an efficient and simple method for introducing temporal consistency to a large class of optimization driven image-based computer graphics problems. Our method extends recent work in edge-aware filtering, approximating costly global regularization with a fast iterative joint filtering operation.

We extend a recent advance in edge-aware filtering called the domain transform, that embeds geodesic distance onto a manifold that allows us to perform filtering with a separable Gaussian filter. This fact causes the spatiotemporal extension to have only a linear (instead of quadratic) increase of runtime.

Using this representation, we can achieve tremendous efficiency gains both in terms of memory requirements and running time. This efficiency gain enables us to process entire shots at once, taking advantage of supporting information that exists across far away frames, something that is difficult with existing approaches due to the computational burden of video data.

Our method is able to filter along motion paths using an iterative approach that simultaneously uses and estimates per-pixel optical flow vectors. We demonstrate its utility by creating temporally consistent results for a number of applications including optical flow, disparity estimation, colorization, scribble propagation, sparse data up-sampling, and visual saliency computation

Publications

Practical Temporal Consistency for Image-Based Graphics Applications-Thumbnail

Practical Temporal Consistency for Image-Based Graphics Applications
August 5, 2012
ACM SIGGRAPH 2012
Paper File [pdf, 17.80 MB]

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.