A behavioural method for assessing the performance of image denoising algorithms

Numerous algorithms exist for removing unwanted noise from digital images. Typically, these are inspired by biological visual systems and involve bandpass filtering and thresholding to remove the noise. However, the efficacy of such algorithms is usually assessed numerically (eg by calculating the root mean squared error between original and denoised images) with little regard to perceptual consequences for the end user. This can—in principle—lead to situations where two algorithms are equally ‘good’ numerically, yet one may produce highly salient artefacts whilst the other does not. We propose a novel behavioural method for comparing denoised images. Using segments of 308 images from a public image library, we measured contrast thresholds for three observers detecting Gaussian pixel noise added to the image in one interval of a 2IFC experiment; these constitute our baseline. We then repeated the experiment, having passed all of the stimuli (images with or without noise added) through a denoising algorithm (Fischer et al, 2007 International Journal of Computer Vision 75 231–246). The increase in threshold relative to the baseline (typically a factor of ~2) provides an index of the success of the denoising algorithm by giving an indication of the amount of perceptually meaningful (eg visible) noise that is ‘hidden’ by the algorithm. We use this technique to compare polar-separable and cartesian-separable log-Gabor filters, as well as filters of different orientation bandwidths. Thresholds occurred at a constant peak-signal-to-noise ratio for baseline and denoised conditions, linking numerical comparisons with a measure of perceptual validity for the end user.