Shape-Color Differential Moment Invariants under Affine Transformations

# Shape-Color Differential Moment Invariants under Affine Transformations

## Abstract

We propose the general construction formula of shape-color primitives by using partial differentials of each color channel in this paper. By using all kinds of shape-color primitives, shape-color differential moment invariants can be constructed very easily, which are invariant to the shape affine and color affine transforms. 50 instances of SCDMIs are obtained finally. In experiments, several commonly used color descriptors and SCDMIs are used in image classification and retrieval of color images, respectively. By comparing the experimental results, we find that SCDMIs get better results.

###### Keywords:
shape-color primitives, affine transform, partial differential, shape-color differential moment invariants

Authors’ Instructions

## 1 Introduction

Image classification and retrieval for color images are two hotspots in pattern recognition. How to extract effective features, which are robust to color variations caused by the changes in the outdoor environment and geometric deformations caused by viewpoint changes, is the key issue. The classical approach is to construct invariant features for color images. Moment invariants are widely used invariant features.

Moment invariants were first proposed by Hu[1] in 1962. He defined geometric moments and constructed 7 geometric moment invariants which were invariant under the similarity transform(rotation, scale and translation). Researchers applied Hu moments to many fields of pattern recognition and achieved good results[2, 3]. Nearly 30 years later, Flusser et al.[4] constructed the affine moment invariants (AMIs) which are invariant under the affine transform. The geometric deformations of an object, which are caused by the viewpoint changes, can be represented by the projective transforms. However, general projective transforms are complex nonlinear transformations. So, it’s difficult to construct projective moment invariants. When the distance between the camera and the object is much larger than the size of the object itself, the geometric deformations can be approximated by the affine transform. AMIs have been used in many practical applications, such as character recognition[5] and expression recognition[6]. In order to obtain more AMIs, researchers designed all kinds of methods. Suk et al.[7] proposed graph method which can be used to construct AMIs of arbitrary orders and degrees. Xu et al.[8] proposed the concept of geometric primitives, including distance, area and volume. AMIs can be constructed by using various geometric primitives. This method made the construction of moment invariants have intuitive geometric meaning.

The above-mentioned moment invariants are all designed for gray images. With the popularity of color images, the moment invariants for color images began to appear gradually. Researchers wanted to construct moment invariants which are not only invariant under the geometric deformations but also invariant under the changes of color space. Geusebroek et al.[9] proved that the affine transform model was the best linear model to simulate changes in color resulting from changes in the outdoor environment. Mindru et al.[10] proposed moment invariants which were invariant under the shape affine transform and the color diagonal-offset transform. The invariants were constructed by using the related concepts of Lie group. Some complex partial differential equations had to be solved. Thus, the number of them was limited and difficult to be generalized. Also, Suk et al.[11] put forward affine moment invariants for color images by combining all color channels. But this approach was not intuitive and did not work well for the color affine transform. To solve these problems, Gong et al.[12, 13, 14] constructed the color primitive by using the concept of geometric primitive proposed in [8]. Combining the color primitive with some shape primitives, moment invariants that are invariant under the shape affine and color affine transforms can be constructed easily, which were named shape-color affine moment invariants(SCAMIs). In [14], they obtained 25 SCAMIs which satisfied the independency of the functions. However, we find that a large number of SCAMIs with simple structures and good properties are missed in [14].

In this paper, we propose the general construction formula of shape-color primitives by using partial differentials of each color channel. Then, we use two kinds of shape-color primitives to construct shape-color differential moment invariants(SCDMIs), which are invariant under the shape affine and color affine transforms. We find that the construction formula of SCAMIs proposed in [14] is a special case of our method. Finally, commonly used image descriptors and SCDMIs are used for image classification and retrieval of color images, respectively. By comparing the experimental results, we find that SCDMIs proposed in this paper get better results.

## 2 Related Work

In order to construct image features which are robust to color variations and geometric deformations, researchers have made various attempts. Among them, SCAMIs proposed in [14] are worthy of special attention. SCAMIs are invariant under the shape affine and color affine transforms. Two kinds of affine transforms are defined by

 (x′y′)=SA⋅(xy)+ST=(α1α2β1β2)⋅(xy)+(OxOy) (1)
 ⎛⎜ ⎜⎝R′(x,y)G′(x,y)B′(x,y)⎞⎟ ⎟⎠=CA⋅⎛⎜⎝R(x,y)G(x,y)B(x,y)⎞⎟⎠+CT=⎛⎜⎝ a1a2a2 b1b2b2 c1c2c2⎞⎟⎠⋅⎛⎜⎝R(x,y)G(x,y)B(x,y)⎞⎟⎠+⎛⎜⎝OROGOB⎞⎟⎠ (2)

where SA and CA are nonsingular matrices.

For the color image , let be three arbitrary points in the domain of . The shape primitive and the color primitive are defined by

 S(p,q)=∣∣∣(xp−¯x)(xq−¯x)(yp−¯y)(yq−¯y)∣∣∣ (3)
 C(p,q,r)=∣∣ ∣ ∣∣(R(xp,yp)−¯R)(R(xq,yq)−¯R)(R(xr,yr)−¯R)(G(xp,yp)−¯G)(G(xq,yq)−¯G)(G(xr,yr)−¯G)(B(xp,yp)−¯B)(B(xq,yq)−¯B)(B(xr,yr)−¯B)∣∣ ∣ ∣∣ (4)

where represents the mean value of , .

Then, using Eq.(3) and (4), the shape core can be defined by

 sCore(n,m;d1,d2,...,dn)=S(1,2)S(k,l)...S(r,n)m (5)

where n and m represent that the sCore is the product of m shape primitives which are constructed by N points . , , . represents the number of point in all shape primitives, .

Similarly, the color core can be defined by

 cCore(N,M;D1,D2,...,DN)=C(1,2,3)C(G,K,L)...C(P,Q,N)M (6)

Where N and M represent that the cCore is the product of M color primitives which are constructed by N points . , , . represents the number of point in all color primitives, .

Suppose the color image is transformed into the image by two transformations defined by Eq.(1) and (2), in are the corresponding points of in . Gong et al.[14] have proved

 S′(p,q)=|SA|⋅S(p,q) (7)
 C′(p,q,r)=|CA|⋅C(p,q,r) (8)

Further results can be concluded

 sCore′(n,m;d1,d2,...,dn)=|SA|m⋅sCore(n,m;d1,d2,...,dn) (9)
 cCore′(N,M;D1,D2,...,DN)=|CA|M⋅cCore(N,M;D1,D2,...,DN) (10)

Therefore, the SCAMIs are constructed by

 SCAMIs(n,m,N,M;d1,...,dn;D1,...,DN)=I(sCore(n,m,d1,...,dn)⋅cCore(N,M,D1,...,DN))I(sCore(1,0))max(n+N)+m−3M2⋅I(cCore(3,2;2,2,2))M2 (11)

Then there is a relation

 SCAMIs′(n,m,N,M;d1,...,dn;D1,...,DN)=SCAMIs(n,m,N,M;d1,...,dn;D1,...,DN) (12)

It must be said that , and are named the degree, the shape order and the color order of SCAMIs, respectively. In fact, Eq.(11) can be expressed as polynomial of shape-color moment. This moment was first proposed in [16] and defined by

 SCMpqαβγ=∬(x−¯x)p(y−¯y)q(R(x,y)−¯R)α(G(x,y)−¯G)β(B(x,y)−¯B)γdxdy (13)

Gong et al.[14] proposed that they constructed all SCAMIs of which degrees , shape orders and color orders . They obtained 24 SCAMIs which are functional independencies using the method proposed by Brown [17]. However, we will point out in the Section 3 that they omitted many simple and well-behaved SCAMIs.

## 3 The Construction Framework of SCDMIs

In this section, we introduce the general definitions of shape-color differential moment and shape-color primitive, firstly. Then, using the shape-color primitive, the shape-color core can be constructed. Finally, according to Eq.(11) and the shape-color core, we obtain the general construction formula of SCDMIs. Also, 50 instances of SCDMIs are given for experiments in the Section 4.

### 3.1 The Definition of The General Shape-Color Moment

Definition 1.

Suppose the color image have the k-order partial derivatives . The general shape-color differential moment is defined by

 SCMkpqαβγ=∬(x−¯x)p(y−¯y)q(R(k)(x,y)−¯Rδ(k))α(G(k)(x,y)−¯Gδ(k))β(B(k)(x,y)−¯Bδ(k))γdxdy (14)

where , represent the k-order partial derivatives of . represent the mean values of . is the impact function which is defined by

 δ(k)={1(k=0)0(k≠0) (15)

We can find that Eq.(14) and Eq.(13) are identical, when . Therefore, the shape-color moment is a special case of the general shape-color differential moment.

### 3.2 The Construction of The General Shape-Color Primitive

Definition 2.

Suppose the color image have the k-order partial derivatives . are three arbitrary points in the domain of . The general shape-color primitive is defined by

 SCPk(p,q,r)=∣∣ ∣ ∣∣FkR(xp,yp)FkR(xq,yq)FkR(xr,yr)FkG(xp,yp)FkG(xq,yq)FkG(xr,yr)FkB(xp,yp)FkB(xq,yq)FkB(xr,yr)∣∣ ∣ ∣∣ (16)

where

 FkC(x,y)=k∑i=0(ki)(x−¯x)i(y−¯y)k−i∂kC(x,y)∂xi∂yk−i,   C∈{R,G,B}. (17)

We can find that defined by Eq.(4) is a special case of , when .

### 3.3 The Construction of The General Shape-Color Core

Definition 3.

Using Definition 2, the general shape-color core is defined by

 scCorek(N,M;D1,D2,...,DN)=SCPk(1,2,3)SCPk(G,K,L)...SCPk(P,Q,N)M (18)

Where , N and M represent that the is the product of M shape-color primitives constructed by N points . , , . represents the number of point in all shape-color primitives, .

Obviously, defined by Eq.(6) is a special case of , when .

### 3.4 The Construction of SCDMIs

Theorem 1.

Let the color image be transformed into the image by Eq.(1) and Eq.(2), in are corresponding points of in , respectively. Suppose that have the k-order partial derivatives . Then there is a relation

 SCP′k(p,q,r)=|CA|⋅SCPk(p,q,r) (19)

where

 SCP′k(p,q,r)=∣∣ ∣ ∣ ∣∣FkR′(x′p,y′p)FkR′(x′q,y′q)FkR′(x′r,y′r)FkG′(x′p,y′p)FkG′(x′q,y′q)FkG′(x′r,y′r)FkB′(x′p,y′p)FkB′(x′q,y′q)FkB′(x′r,y′r)∣∣ ∣ ∣ ∣∣ (20)

Further, the following relation can be obtained

 scCore′k(N,M;D1,D2,...,DN)=|CA|M⋅scCorek(N,M;D1,D2,...,DN) (21)

where

 scCore′k(N,M;D1,D2,...,DN)=SCP′k(1,2,3)SCP′k(G,K,L)...SCP′k(P,Q,N)M (22)

By using Maple2015, the proof of Theorem 1 is obvious. We can find that Eq.(10) is a special case of Eq.(20), when . So, when we replace in Eq.(11) with , Eq.(12) is still tenable. Now, we can define SCDMIs.

Theorem 2.

 SCDMIsk(n,m,N,M;d1,...,dn;D1,...,DN)=I(sCore(n,m,d1,...,dn)⋅scCorek(N,M,D1,...,DN))I(sCore(1,0))max(n+N)+m−3M2⋅I(scCorek(3,2;2,2,2))M2 (23)

Then there is a relation

 SCDMIs′k(n,m,N,M;d1,...,dn;D1,...,DN)=SCDMIsk(n,m,N,M;d1,...,dn;D1,...,DN) (24)

where

 SCDMIs′k(n,m,N,M;d1,...,dn;D1,...,DN)=I(sCore′(n,m,d1,...,dn)⋅scCore′k(N,M,D1,...,DN))I(sCore′(1,0))max(n+N)+m−3M2⋅I(scCore′k(3,2;2,2,2))M2 (25)

Eq.(23) can be expressed as polynomial of . Eq.(11) is a special case of Eq.(23) when . The proof of Eq.(24) is exactly the same as that of Eq.(12) proposed in [14].

### 3.5 The Instances of SCDMIs

We can use Eq.(24) to construct instances of by setting different k values. However, color images are discrete data, the partial derivatives of each order can’t be accurately calculated. With the elevation of order, the error will be more and more, which will greatly affect the stability of SCDMIs. So, we set in this paper.

When , are equivalent to SCAMIs. We construct of which degrees , shape orders and color orders . Gong et al. [14] thought that in order to obtain the of which degrees , shape orders and color orders , must be . This judgment is wrong. In fact, can be , , and . Thus, lots of were missed in [14]. By correcting this shortcoming, we get 25 that satisfy the independence of the function by using the method proposed by [17].

At the same time, when , is defined by

 SCP1(p,q,r)=∣∣ ∣ ∣∣F1R(xp,yp)F1R(xq,yq)F1R(xr,yr)F1G(xp,yp)F1G(xq,yq)F1G(xr,yr)F1B(xp,yp)F1B(xq,yq)F1B(xr,yr)∣∣ ∣ ∣∣ (26)

where

 F1C(x,y)=(x−¯x)∂C∂x+(y−¯y)∂C∂y   C∈{R,G,B}. (27)

By replacing , , and with , , and , 25 can be obtained. Therefore, we can construct the feature vector SCDMI50, which is defined by

 SCDMI50=[SCDMIs10,...,SCDMIs250,SCDMIs11,...,SCDMIs251] (28)

The construction methods of 50 instances are shown in Table 1.