Appendix

For details on solving the joint optimization problem and robustness analysis of our algorithm, please see: [appendix.pdf]

 

Additional Results

Move your mouse over captions to switch between input and results. Click for full resolution.
 

Quantitative comparison

Results with PSNR. Our method ("Ours") is compared with several algorithms. "DT-GIF" is the result obtained by the guided filter version of the detail transfer method [Petschnigg et al, He et al]. "DJF" is the imaged produced by the deep joint filter [Li et al]. "Huawei" is the output image by Huawei P9's black box software. Captured ground truth is shown in "GT".


Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw (30.17dB)     DT-GIF (33.55dB)     DJF (31.49dB)     Huawei (31.75dB)     Ours (33.90dB)     GT
 

Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw (28.29dB)     DT-GIF (29.62dB)     DJF (29.19dB)     Huawei (28.82dB)     Ours (30.15dB)     GT
 

Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw (31.39dB)     DT-GIF (35.64dB)     DJF (32.95dB)     Huawei (33.68dB)     Ours (36.12dB)     GT
 

Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw (30.28dB)     DT-GIF (36.23dB)     DJF (34.37dB)     Huawei (34.34dB)     Ours (36.74dB)     GT
 

Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw (27.90dB)     DT-GIF (34.32dB)     DJF (30.73dB)     Huawei (32.72dB)     Ours (34.71dB)     GT
 

Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw (30.67dB)     DT-GIF (37.83dB)     DJF (34.68dB)     Huawei (35.41dB)     Ours (38.48dB)     GT
 

Improving pre-processed images

Here we demonstrate that our method can be used to improve images that were processed by the blackbox image processing stack implemented in the Huawei P9 cell phone. In this scenario we simply removing the Bayer downsampling matrix in our optimization problem and replacing the input image with the camera image (see "Huawei"). Unreliable correspondences are highlighted in "Mono (warped+mask)". "DCRaw" is the demosaicked image by DCRaw without denoising. "Ours" is our method working on raw Bayer images, whereas "Ours+Huawei" means that we work on images that have been preprocessed with the full (proprietary and unknown) Huawei software stack, including demosaicking, donoising etc.


Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw     Ours     Huawei     Huawei+Ours
 

Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw     Ours     Huawei     Huawei+Ours
 

An example where colorization would fail

We also provide an example scene where colorizing the monochrome image would lead to color ambiguity in occluded region (see the colorful beads occluded in the color image "DCRaw" but seen in the unwarped monochrome image "Mono"). Therefore it is more robust to reconstruct images directly on the RGB camera. "DCRaw" is the demosaicked image by DCRaw without denoising. PSNR is computed against the ground truth ("GT") captured by extending exposure time and decreasing ISO accordingly.


Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw (30.94dB)     Ours (38.68dB)     GT
 

Qualitative comparison

Additional examples without ground truth image. Unreliable correspondences are highlighted in "Monochrome (warped+mask)".


Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw     DT-GIF     DJF     Huawei     Ours
 

Mono     Reference (warped Mono)     Mono (warped+mask)     DCRaw     DT-GIF     DJF     Huawei     Ours
 

References

Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Transaction on Graphics (Proceddings SIGGRAPH 2004) 23(3) (August 2004)

He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(6) (2013)

Li, Y., Huang, J.B., Narendra, A., Yang, M.H.: Deep joint image filtering. In: European Conference on Computer Vision. (2016)