Stereo to Multi-View Conversion
Nikolce Stefanoski (Disney Research Zurich)
Manuel Lang (Disney Research Zurich/ETH Zurich)
Oliver Wang (Disney Research Zurich)
Simon Heinzle (Disney Research Zurich)
Aljoscha Smolic (Disney Research Zurich)
Content creation for autostereoscopic displays is an unresolved problem. Typical methods rely on view synthesis based on depth image based rendering (DIBR). DIBR also forms the core of a corresponding standardization activity in MPEG, which correspondingly aims at efficient coding and transmission of multiview video plus depth (MVD) data to support MADs. However, this approach relies on depth estimation, which is an ill-posed and unresolved task so far. It is highly questionable if automatic depth estimation can be resolved with sufficient accuracy, reliability and robustness in the near future. Our method applies purely image domain warping instead. Input video is analyzed and information about sparse disparity, vertical edges and saliency is extracted. A constrained energy minimization problem is formulated and efficiently solved. The resulting image warping functions are used to synthesize novel views. Our approach is fully automatic, accurate, and reliable. Disocclusions and related artifacts are avoided due to smooth, saliency-driven warping functions. Our method also works well for extrapolation of views in a limited range, thus supporting multiview creation from stereo input, which is the most relevant use case scenario.
Our method was used as part of a proposal to MPEG for a standard on 3D Video Technology. There it was evaluated in the scope of a subjective testing procedure, which was conducted anonymously, and in which more than 600 subjects participated. Our proposal was always ranked among the four best performing approaches where the concrete rank depends on the testing category