# Image quality assessment measure based on natural image statistics in the Tetrolet domain

###### Abstract

This paper deals with a reduced reference (RR) image quality measure based on natural image statistics modeling. For this purpose, Tetrolet transform is used since it provides a convenient way to capture local geometric structures. This transform is applied to both reference and distorted images. Then, Gaussian Scale Mixture (GSM) is proposed to model subbands in order to take account statistical dependencies between tetrolet coefficients. In order to quantify the visual degradation, a measure based on Kullback Leibler Divergence (KLD) is provided. The proposed measure was tested on the Cornell VCL A-57 dataset and compared with other measures according to FR-TV1 VQEG framework.

###### Keywords:

RRIQA, Tetrolet transform, natural image statistics, Gaussian Scale Mixture.

## 1 Introduction

Recently, several RR methods have been introduced but few of them are general-purpose. The first general-purpose RR methods was introduced by Wang [1] in the steerable pyramids domain named WNISM. The KLD was used to quantify the difference between two subband coefficient histograms. The first histogram is computed from the distorted image while the second is summarized using the Generalized Gaussian Density (GGD) model parameters instead of sending all histogram bins. Promising results were obtained for five distortions in the LIVE dataset. Tao et al [2] have proposed the contourlet transform which is effective in dealing with directional information like edges. After CSF masking, the JND is applied to remove visually insensitive coefficients. A histogram is formed from the remaining coefficient. Finally, the histogram is normalized and considered as RR feature. Results were presented for two distortions from the LIVE dataset : JPEG and JPEG2000 compressions. Li et al [3] investigated the Divisive Normalization Transform (DNT) to take into account the dependencies between wavelet coefficients which were ignored in the WNISM. The measure based on the DNT improved the WNISM, specially when it was tested on a set formed by different distortions. Nevertheless, its performances can change significantly since it depends on some parameters which need to be trained. In [4] the construction of the Strongest Component Map (SCM) is proposed. The Weibull distribution parameters are estimated from the SCM coefficients histograms. Finally, only the scale parameter is involved in a measure called W-SCM. Experiments with the LIVE dataset show significant correlation between the model predictions and the subjective scores, nearly the same as WNISM. In [5] grouplets have been used to capture image geometric structures and orientations. To incorporate HVS characteristics, a Contrast Sensitivity Function (CSF) is applied before measuring the changes between the reference and the distorted images. Their results show some significant improvements for JPEG distorted images as compared to WNISM.

Inspired by the work of Wang, we have proposed the use of the BEMD (Bi-dimentional Empirical Mode Decomposition) in the general scheme as an adaptive decomposition. Although the BEMD-based method outperforms the WNISM over several distortions in the TID 2008 dataset, low correlations with human judgment were obtained.

In this work, we propose a joint probability distribution of tetrolet coefficients using GSM model. This allows us to exploit the dependencies between tetrolet coefficients. A GSM model is defined as the product of zero mean Gaussian vector and positive random variable called multiplier. Here, we propose Weibull distribution to model the multiplier distribution. Then, assuming the independency between GSM components (the multiplier and the Gaussian vector) we derived an expression for the KLD in order to evaluate the visual quality of a processed image.

The rest of this paper is organized as follows. In section 2 we give a brief review of the tetrolet transform, in section 3 we explain how we model the dependencies between tetrolet coefficients using the GSM model, we present the distortion measure in section 4. Section 5 is reserved for experimental results and finally a conclusion ends the paper.

## 2 Tetrolet transform

Nowadays, a sparse representation is required in image processing techniques. In such representation the energy of the signal is concentrated in few number of coefficients not null. This facilitates the feature extraction step used in image retrieval, image classification and RRIQA algorithms. Although wavelets were introduced for this aim, they can take advantage only of singularity points. Thus directional information like edge is disregarded. The idea of tetrolet transform [6] is to allow more general partitions which capture the image local geometry by bringing the ”tiling by tetrominoes” problem into play.

Tetrominoes are derived from the well know game ”tetris”. They were introduced by Golomb [7]. We can obtain a tetromino by connecting four equal sized square. Disregarding rotation and isometric we have five free tetrominoes as shown in Figure 1.

The Haar transform is a special case, since it considers only the first tetromino (square). To use other tetrominoes we should have at least a blocks (Figure 2) which will give 117 possibility, whereas a blocks gives possibilities. From computational complexity standpoint, it’s clear that the first choice is the reasonable one.

Therefore, tetrominoes ensure more directions when rotations and reflections are considered. To illustrate this let’s take from Figure 2 (Line 4) the third covering (from left to right), eight other coverings are possible with different directions are shown in Figure 3.

### 2.1 Computing Tetrolet transform

The computation of the tetrolet transform consists in two stages. First, the tiling by tetrominoes is achieved ensuring an optimal covering for each block in the image. Second, the Haar transform is applied to the tetrominoes of each covering. More precisely, let us take an image , is a power of , i.e and we suppose that we are in the level. The image is decomposed into blocks, for each block we consider the 117 possible covering . The Haar transform is than applied to the tetrominoes forming the investigated covering. This leads to four low-pass coefficients and 12 tetrolet coefficients as follows :

and refer to the actual covering and the actual level of decomposition respectively, while refers to the tetrominoes of the covering and refers to the three high-pass parts.

The optimal covering is then qualified as the one whose tetrolet coefficients provide the minimal :

(1) |

In other words, the smaller is the magnitude of the 12 tetrolet coefficients, the minimal is the norm. Thus we obtain the optimal covering and a sparse image representation.

Once we get the optimal covering , we store the corresponding four low-pass coefficients and 12 tetrolet coefficients : . Doing this for all blocks in the image we achieve the tetrolet transform. Before applying further levels of the tetrolet transform, we should rearrange the components of the vector into matrix using a reshape function :

(2) |

## 3 Joint statistics of tetrolet coefficients

The tetrolet transform provides a multi-resolution representation with three orientations since it is derived from the Haar wavelet transform. Here, we propose to exploited the dependencies between tetrolet coefficients as it was done for wavelet coefficients [8] as the same as for the curvelet coefficients [9]. The Gaussian Scale mixture (GSM) model has been used to model both marginal and joint statistics of natural image wavelet coefficients [10]. Let us consider a N-length random vector . we assume that in our study is formed from coefficients clustered around a given coefficient at scale and orientation . is a GSM if it can be written as the product of a zero mean Gaussian random vector with covariance matrix and a positive scalar random variable called multiplier:

(3) |

denotes equality in probability. and are independent. If we denote as the density of the variable the density of can be expressed as [10]:

(4) |

To obtain an explicit expression of the PDF of we should specify the density of the multiplier . Since the multiplier variable is positive, several distributions can be considered. Here, we choose Weibull density. To this end, we should estimate first the multiplier. As this later is unknown, we can estimate it by maximum-likelihood method [10] of the observed coefficients given by :

(5) |

where is the covariance matrix of the Gaussian vector estimated from the tetrolet coefficients and is the length of the vector . Figure 4 illustrates Weibull fitting to the estimated multiplier.

The PDF of Weibull distribution is given by:

(6) |

where is the shape parameter and is the scale parameter of the distribution. Inserting the equation (6) in equation (4) the PDF of becomes :

(7) |

## 4 Distortion measure

In the previous section we have represented the joint statistics of tetrolet coefficients using a univariate Weibull distribution and a multivariate Gaussian distribution. Considering a neighborhood of dimension equals to 9 (33).
At the sender side, we apply two levels tetrolet transform to the reference image. This leads to six tetrolet coefficients subbands (2 scales 3 orientations). From each subband three features are extracted : the covariance matrix and the Weibull parameters . The extracted features are considered as RR side information. Similarly, the same features are extracted from the distorted image at the receiver side and we consider them as reduced description (RD).

A dissimilarity measure is required to compare the RR to the RD and thus quantify the visual degradation. According to our knowledge a closed analytical form of the KLD for the proposed joint distribution in equation (7) does not exist. To resolve this problem, let us consider two joint distributions and , where is a GSM vector. Since the components of the GSM (the multiplier and the Gaussian vector) are independent, we can derive an expression for the KLD between two joint distributions as the sum of the KLD between two multivariate Gaussian densities and the KLD between two Weibull distributions. In other words :

(8) |

Now, that we have a closed analytical form for the KLD for both, Weibull distribution and the multivariate Gaussian density we can easily derive the KLD for the proposed joint distribution as:

(9) |

where denotes the Euler-Mascheroni constant () and is the Gamma function.

First, the distance in equation (9) is computed to quantify the dissimilarity between two tetrolet coefficient subbands, the first from the reference image and the second is its correspondent from the distorted image. Finally, the dissimilarities between the subbands are combined to produce a global dissimilarity as follows :

(10) |

where is the number of the subbands, is a constant to control the scale of the distortion measure and it is equal to 0.1.The log function is involved here to reduce the difference between a high values and a low values of , so that we can have values in the same order.

## 5 Experimental results

Our experimental test was carried out using the Cornell VCL-A 57 [11] dataset. It provides 60 distorted images. Three reference images are altered with six distortions labeled : FLT, NOZ, JPG, JP2, DCQ and BLR. The labels refer to quantization of the LH subbands of a five-level DWT of the image using the 9/7 filters, additive Gaussian white noise, baseline JPEG compression, JPEG2000 compression using the 9/7 filters, JPEG2000 compression using the 9/7 filters with the dynamic contrast-based quantization algorithm, blurring by using a Gaussian filter, respectively. Each image in the Cornell VCL-A57 has its Mean Opinion Score (MOS). The subjective scores must be compared in term of correlation with the objective scores. These objective scores are computed from the values generated by the objective measure, using a non linear function according to the Video Quality Expert Group (VQEG) Phase I FR-TV [12]. Here, we use a four parameters logistic function.

(11) |

where .

Thus, the predicted MOS is given by :

(12) |

Once the nonlinear mapping is achieved, we obtain the predicted objective quality scores. To compare the subjective and objective quality scores, several metrics were introduced by the VQEG. In our study, we compute the correlation coefficient to evaluate the accuracy prediction and the Rank order coefficient to evaluate the monotonicity prediction. Table 1 shows the results for the Cornell VCL A-57 dataset.

Dataset | FLT | JPG | JP2 | DCQ | BLR | NOZ | All |

Correlation Coefficient | |||||||

0.71 | 0.96 | 0.83 | 0.95 | 0.91 | 0.86 | 0.70 | |

0.76 | 0.91 | 0.81 | 0.90 | 0.93 | 0.99 | 0.66 | |

Method in [13] | 0.49 | 0.85 | 0.78 | 0.93 | 0.76 | 0.62 | 0.31 |

0.91 | 0.70 | 0.79 | 0.56 | 0.59 | 0.93 | 0.63 | |

0.92 | 0.91 | 0.87 | 0.94 | 0.79 | 0.88 | 0.72 | |

Rank-Order Correlation Coefficient | |||||||

0.46 | 0.96 | 0.81 | 0.90 | 0.90 | 0.80 | 0.74 | |

0.50 | 0.76 | 0.80 | 0.66 | 0.80 | 0.98 | 0.70 | |

Method in [13] | 0.10 | 0.76 | 0.53 | 0.80 | 0.66 | 0.73 | 0.29 |

0.90 | 0.63 | 0.80 | 0.50 | 0.46 | 0.95 | 0.62 | |

0.96 | 0.93 | 0.86 | 0.96 | 0.90 | 0.91 | 0.78 |

As we can see, results reported in table. 1 concern the proposed measure as well as some FR and RR methods. In comparison with RR methods, the proposed measure outperforms the DNT-based methods for JPG, JP2 and DCQ distortions, and the method in [13] for JPG, JP2, DCQ, BLR and NOZ distortions. The proposed measure outperforms also the PSNR for JPG, JP2, DCQ and BLR distortions, and MSSIM [14]for JPG, DCQ and BLR distortions. However, the proposed measure fails for the FLT distortion.

## 6 Conclusion

In this paper we have introduced a RR measure in the tetrolet domain. The GSM model was used to characterize the dependencies between tetrolet coefficients. We have proposed the Weibull distribution to model the multiplier of the GSM model, this leads to a new joint distribution. Assuming the independence between GSM components we have derived a closed expression of the KLD for the propose joint distribution. Significant improvements were remarked for the proposed measure when it was tested on the Cornell VCL-A57 dataset.

## References

- [1] Z. Wang and E.P. Simoncelli, ”Reduced-reference image quality assessment using a wavelet-domain natural image statistic model,” in Proc.of SPIE Human Visionand Electronic Imaging, 2005, vol. 5666, pp. 149-159.
- [2] D. Tao, X. Li, W. Lu, and X. Gao, ”Reduced-reference IQA in contourlet domain,” IEEE Transactions on Systems, Man, and Cybernetics, PartB: Cybernetics, vol. 39, no. 6, pp. 1623-1627,2009.
- [3] Q. Li and Z. Wang, ”Reduced-reference image quality assessment using divisive normalization-based image representation,” IEEE Journal of Selected Topicsin Signal Processing, vol. 3, no. 2, pp. 202-211,2009.
- [4] W. Xue and X. Mou, ”Reduced reference image quality assessment based on weibull statistics,” in The International Workshop on Quality of Multimedia Experience (QoMEx), 2010, pp. 1-6.
- [5] A. Maalouf, M.C. Larabi, and C. Fernandez-Maloigne, ”A grouplet-based reduced reference image quality assessment,” in The International Workshop on Quality of Multimedia Experience(QoMEx), 2009, pp. 59-63.
- [6] J. Krommweh, ”Tetrolet transform: A new adaptive haar wavelet algorithm for sparse image representation,” Journal of Visual Communication and Image Representation, vol.21, no. 4, pp. 364-374, 2010.
- [7] S.W.Golomb, Polyominoes: puzzles, patterns, problems,and packings, Princeton Univ Pr,1996.
- [8] E.P. Simoncelli, ”Modeling the joint statistics of images in the wavelet domain,” in Proc. SPIE, 1999, vol. 3813, pp. 188-195.
- [9] L. Boubchir and J.M. Fadili, ”Multivariate statistical modeling of images with the curvelet transform,” in Proc. IEEE Conf.on Signal Processing and Its Applications, 2005, pp. 747-750.
- [10] M.J. Wainwright and E.P. Simoncelli, ”Scale mixtures of gaussians and the statistics of natural images,” Advances in neural information processing systems, vol. 12, no. 1, pp. 855-861, 2000.
- [11] D. M. Chandler and S. S. Hemami, ”Cornell-vcl a57 database,” Available at: http://foulard.ece.cornell.edu/dmc27/vsnr/vsnr.html.,2007.
- [12] A.M.Rohaly, J.Libert, P.Corriveau, A.Webster et al, ”Final report from the video quality experts group on the validation of objective models of video quality assessment,” ITU-T Standards Contribution COM, pp. 9-80.
- [13] Z. Wang, G. Wu, H.R. Sheikh, E.P. Simoncelli, E.H. Yang, and A.C. Bovik, ”Quality-aware images,” IEEE Transactions on Image Processing, vol. 15, no. 6, pp. 1680-1689, 2006.
- [14] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, ”Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.