Kernel-predicting Convolutional Networks for Denoising Monte Carlo Renderings

Authors

Steve Bako (University of California, Santa Barbara)
Thijs Vogels (ETH Zurich, Disney Research Zurich)
Brian McWilliams (Disney Research Zurich)
Mark Meyer (Pixar Animation Studios)
Jan Novák (Disney Research Zurich)
Alex Harvill (Pixar Animation Studios)
Pradeep Sen (University of California, Santa Barbara)
Tony DeRose (Pixar Research Group)
Fabrice Rousselle (Disney Research Zurich)

ACM SIGGRAPH 2017

July 10, 2017

We introduce a deep learning approach for denoising Monte Carlo-rendered images that produces high-quality results suitable for production. We train a convolutional neural network to learn the complex relationship between noisy and reference data across a large set of frames with varying distributed eects from the film Finding Dory (le). The trained network can then be applied to denoise new images from other films with significantly different style and content, such as Cars 3 (right), with production-quality results.

Regression-based algorithms have shown to be good at denoising Monte Carlo (MC) renderings by leveraging its inexpensive by-products (e.g., feature buffers). However, when using higher-order models to handle complex cases, these techniques often overt to noise in the input. For this reason, supervised learning methods have been proposed that train on a large collection of reference examples, but they use explicit filters that limit their denoising ability. To address these problems, we propose a novel, supervised learning approach that allows the filtering kernel to be more complex and general by leveraging a deep convolutional neural network (CNN) architecture. In one embodiment of our framework, the CNN directly predicts the nal denoised pixel value as a highly non-linear combination of the input features. In a second approach, we introduce a novel, kernel-prediction network which uses the CNN to estimate the local weighting kernels used to compute each denoised pixel from its neighbors. We train and evaluate our networks on production data and observe improvements over state-of-the-art MC denoisers, showing that our methods generalize well to a variety of scenes. We conclude by analyzing various components of our architecture and identify areas of further research in deep learning for MC denoising.

Download File "Kernel-predicting Convolutional Networks for Denoising Monte Carlo Renderings-Paper"
[pdf, 979.30 KB]

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.