Capturing Patterns via Parsimonious $t$ Mixture Models

A colour image is a digital image that includes 24-bit RGB colour information for each pixel. An RGB image is derived from the three primary colours — red (R), green (G), and blue (B) — for which each colour is 8 bits. The technique of image compression plays a central role in image transmission and storage for high quality. There are several unsupervised image compression approaches based on probabilistic models. The MFA model is well recognized as a dominant dimension reduction technique and is useful in block image transform coding (Ueda et al., 2000). In this subsection, we individually apply the PTMMs and PGMM to colour image compression and compare the quality of the reconstructed images. A $512\times 512$ RGB colour image ‘Lena’ (encoded in 24 bits per pixel) is subdivided into $n=16384$ non-overlapping RGB-blocks of $4\times 4\times 3$ pixels and each block is taken as a 48-dimensional data vector $y$. Let $Y$ be a set of collected $y$ vector. Making a slight modification of Ueda et al. (2000), our compression procedure to transform the experimental image is summarized below.

1.

Set the desired number of components $g$ and dimensionality of factors $q$.

2.

Estimate $\Theta$ by fitting a parsimonious mixture model to $Y$.

3.

Perform a model-based clustering according to the component membership of data point $y$, which is decided by maximizing the posterior probability $Pr(Z_{ij}=1|\text{boldmath}y)$.

4.

For each ${\text{boldmath}y}_j\in Y$ classified to $C_i$, calculate

$${\widehat{\text{boldmath}y}}_j={\widehat{\text{boldmath}\mu }}_i+{\widehat{\text{bf B\%}}}_i{\widehat{\text{boldmath}U}}_{ij},$$

(7)

where ${\widehat{\text{boldmath}U}}_{ij}={\widehat{\text{bf B}}}_i^T({\widehat{\text{bf B\%}}}_i{\widehat{\text{bf B}}}_i^T+{\widehat{\text{boldmath}\Psi }}_i{)}^{-1}({\text{boldmath}Y}_j-{\widehat{\text{boldmath}\mu }}_i)$ is the estimated posterior mean of (6).

5.

Reconstruct ${\widehat{\text{boldmath}y}}_j$, $j=1,\dots ,n$, into a compressed color image.

In what follows, we assumed that the characteristic of each components in the mixture model are the same, which leads to the MFA model with common factor loadings (CUU structure). The Lena image data are fitted by the CUU structure with $(g=4,q=4)$ and $(g=8,q=8)$ using the PGMM and PTMM approaches. To evaluate the quality of the reconstructed image, we compute the root mean squared error (RMSE) and peak signal-to-noise ratio (PSNR), which are widely used metrics in the image coding field. For an original color image $\left\{I_R\right(x,y),I_G(x,y),I_B(x,y\left)\right\}$ of size $M\times N$ and a reconstructed image $\left\{{\hat{I}}_R\right(x,y),{\hat{I}}_G(x,y),{\hat{I}}_B(x,y\left)\right\}$, the RMSE is defined as

where

and the other two quantities $MS{E}_G$ and $MS{E}_B$ are defined in the same fashion. The measure RMSE represents the average Euclidean distance from the original image to the output one. For a 24-bit colour $[0,255{]}^3$ image, the PSNR is given by

Note that the higher PSNR value indicates a compressed (i.e., reconstructed) image of better quality.

Experimental results are provided in Table 2, including the log-likelihoods, the number of parameters, and BIC values, together with the RMSE and PSNR for the compressed images. The performance of PTMM1 and PTMM2 are almost the same in terms of RMSE and PSNR for the compressed images. Notably, the BIC and PSNR values obtained from the PTMM approaches are much higher than the PGMM for all cases.

Figure 1 shows the original Lena image and three compressed images obtained by fitted the CUU $(g=8;q=8)$ models. The quality of the reconstructed images using the PTMMs is much better (i.e., smoother) than those reconstructed using the PGMM (for instance, observe the edge of Lena’s shoulder).

Over the past few decades, numerous template matching approaches, along with their modifications and variants, have been proposed for human perception system face recognition; see Zhao et al. (2003) for an extensive survey. Motivated by a technique developed by Sirovitch and Kirby (1987), the eigenface method (Turk and Pentland, 1991) has become the most popular tool for facial feature extraction in computer vision research. Its implementation is primarily based on the use of PCA, also known as the Karhunen-Loève transform. The central idea of PCA is to find a multilinear subspace whose orthonormal basis maximizes the scatter matrix of all the projected samples.

Although the eigenface method is remarkably simple and quite straightforward, its performance depends heavily on pose, lighting and background conditions. Because PCA is a one-sample method, a single set of eigenfaces may not be enough to represent face images having a large amount of variation. Some efforts have been made to generalize it to the mixture-of-eigenfaces method Kim et al. (2002), which generates more than one set of eigenfaces for better representation of the whole face. However, such a method will produce an over-parameterized solution in many circumstances. To capture the heterogeneity among the underlying face images while maintaining parsimony in modeling, we apply the PTMM approach with common factor loadings (CUU) as a novel strategy for robust facial representation.

Consider a training set of $n$ face images $I_1,\dots ,I_n$. Suppose that all images are exactly the same size; $a$ pixels in width by $b$ in height. Let $\text{boldmath}y=\{{\text{boldmath}y}_j=vec(I_j){\}}_{j=1}^n$ be the transformed image vectors taking intensity values in a $p$-dimensional image space, where $p=a\times b$. In what follows, we briefly review the eigenface method. Define the total scatter matrix of the sample images as

where $\overline{\text{boldmath}y}$ is the mean image vector of all samples. In PCA, the optimal orthonormal basis vectors ${\text{boldmath}\Omega }_{PCA}\in R^{p\times K}$, where $K<<p$, are chosen to maximize the following objective function

where $\{{\text{boldmath}e}_k{\}}_{k=1}^K$ is the set of $p$-dimensional vectors of ${\text{bf S}}_T$ corresponding to the $K$ largest eigenvectors, which are referred to as eigenfaces in Turk and Pentland (1991) because they have exactly the same dimension as the original images. Each face image vector ${\text{boldmath}y}_i$ can be represented as a linear combination of the best $K$ eigenvectors:

where ${\omega }_{jk}={\text{boldmath}e}_k^T({\text{boldmath}y}_j-\overline{\text{\%}\text{boldmath}\$y\$})$.

As another illustration, we compare the face reconstruction performance among the PGMM and PTMM approaches using the image compression algorithm described in the above example and the eigenface method. We use the face images in the Yale face database, which contains 165 grayscale images of 15 individuals in GIF format. There are 11 images per person with pose and lighting variations, but for illustrative purposes and simplicity we consider only 11 images of one individual. For each image, we manually select the centers of eyes, denoted by $(x,y)$. The four boundaries of each cropped central part of the face are $x-60$, $x+60$, $y-100$, and $y+60$, which gives images of $121\times 161$ pixels as a sample in our experiment.

In this experiment, the 11 face images are trained by seven models: the PCA, PGMM, and PTMM1 models with CUU structure, where $K=q=5$ and $g=1,2,3$. The reconstructed images can be obtained by following two steps.

1.

In each model, the fitted vector is obtained as a reconstructed image, say ${\widehat{\text{boldmath}y}}_j$. For the PCA method, ${\widehat{\text{boldmath}y}}_j={\widehat{\text{boldmath}y}}_j^{PCA}$ as defined in (8). As for the PGMM and PTMM1, the solution is obtained by using (7).

2.

If the entries of ${\widehat{\text{boldmath}y}}_j$ do not fall in the color space, such as $[0,1]$, the reproductions ${\widehat{\text{boldmath}y}}_j^{\ast }$ are normalized by

$${\hat{y}}_{jr}^{\ast }=\frac{{\hat{y}}_{jr}-min\left\{{\widehat{\text{boldmath}y}}_j\right\}}{max\left\{{\widehat{\text{boldmath}y}}_j\right\}-min\left\{{\widehat{\text{boldmath}y}}_j\right\}},\text{for}r=1,\dots ,p.$$

Figure 2 shows the original and the reconstructed images obtained from the seven training models. PCA clearly has the worst performance, while the PGMM and PTMM lead to somewhat comparable results. For explicitly measuring the reconstruction quality, we further calculated the RMSE, denoted by $\left[\right({\text{boldmath}y}_j-{\widehat{\text{boldmath}y}}_j^{\ast }{)}^{\top }({\text{\% boldmath}y}_j-{\widehat{\text{boldmath}y}}_j^{\ast }){]}^{1/2}$, for each image and each model. Figure 3 displays the patterns of RMSE values. When comparing these models, smaller RMSEs indicate better reconstructions. In general, PTMM1 yields lower RMSE values than those from PGMM. As a result, we conclude that PTMM1 can be a prominent tool for facial coding.