Today I come across with a very interesting paper which tries to tackle real-world noisy image dataset. As many papers from Lei Zhang’s group, I really enjoy the introduction, problem description, and their approach. The following is my note from reading this paper.
A real-world noisy image dataset is required because
- Major sources of real-noise (will be covered in more details in another post)
- Photon shot: inevitable, caused by the stochastic arrival of the photons to the sensors, modeled as a Poisson process, proportional to the mean of the intensity of the specific pixel, not stationary across the whole image.
- (Fixed pattern) pixel response non-uniformity (PRNU): each pixel has a slightly different output level or response for a fixed light level. This is because of the sensor loss light and color mixture in the neighboring pixels.
- (Fixed pattern) dark current non-uniformity (DCNU): the sensor chip is not perfect so there is some electronics event in no light condition.
- Readout: genreated due to inaccurate charge to voltage conversion
- Quantization: noise due to quantization analog to digital (ADC) process.
- Other noise: CCD specific source such as transfer efficiency and CMOS specific noise like column noise, etc.
- Additive White Gaussian Noise (AWGN) model is too simple and does not hold for the real-world noisy image. Real-world noise is signal dependent.
- Evaluate the quality of the denoised image is difficult as no “ground truth”
- Subjective quality assessment is time-consuming and much effort
- Blind quality assessments were developed from not-real-world datasets.
The motivation for this work are
- Previous real-world dataset either limited in
- The process of capturing and handling real-world images (especially related to ISO, reduce post-processing which is the source of distortion),
- Estimated ‘ground truth’ image
- Limited in the number of camera brands.
Contributions of this paper are
- Propose new dataset of real-world noisy images with the ground truth with more ISO, images, and various camera brand.
- Define the process of handling, capturing images
- Evaluate existing denoising methods for the real-world denoising.
The proposed dataset 40 scenes with various contents and objects for indoor scenes
- 5 cameras brand: Cannon (Mark 5D, 80D, 600D), Nikon, Sony
- More camera setting: 6 ISOs (800, 1600, 3200, 6400 , 12800 and 25600)
- various lighting: normal/dark and outdoor normal lighting condition.
- Captured scenes: buildings, classrooms, cafe rooms, outdoor scenes
- Objects: books, pens, bottles, boxes, joys, etc.
- Methods designed for AWGN achieves lower performance in PSNR and SSIM (even the Deep learning method) compared to methods developed for real-world noisy image
- Gray image techniques process color channel independently –> create more artifact, cannot handle different noise characteristic for different channels, different local patch
- Training based methods depend on training dataset as well as the resolution of the training dataset. Using real-world dataset for training might be useful.
- Best methods are conventional methods (not deep learning): Guided, MC-WNNM, TWSC (see their paper for more details)
- The data is certainly only for the indoor scene, missing human/person –> outdoor dataset and human data are desired
- Low-light condition dataset will be very interesting. The most advanced cameras are claimed to handle low-light condition better.
- I wonder the similar dataset for smart-phone cameras. However, it is quite difficult since images taken by smartphone camera are heavily processed.
- More and more smartphone adopt multiple cameras, it would be interesting to see the problem of joint images denoising of same or different camera types.
J. Xu, H. Li, Z. Liang, D. Zhang, and L. Zhang, “Real-world noisy image denoising: A new benchmark,” Available at arxiv