Image Projective Invariants

Image Projective Invariants

Erbo Li, Hanlin Mo, Dong Xu, Hua Li,  Denotes equal contribution. E. Li is with EON Reality Inc, Irvine, CA, 92618.
Email: sophialiuli@gmail.com H. Mo (correspondent author) and H. Li are with Key lab of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, 100190 Beijing; University of Chinese Academy of Sciences, 100049 Beijing. E-mail:{mohanlin, lihua}@ict.ac.cn D.Xu is with Ambry Genetics, Aliso Viejo, CA 92656.
Email: xudong0614@hotmail.com
Abstract

In this paper, we propose relative projective differential invariants () which are invariant to general projective transformations. By using and the structural frame of integral invariant, projective weighted moment invariants () can be constructed very easily. It is first proved that a kind of projective invariants exists in terms of weighted integration of images, with relative differential invariants as the weight functions. Then, some simple instances of are given. In order to ensure the stability and discriminability of , we discuss how to calculate partial derivatives of discrete images more accurately. Since the number of pixels in discrete images before and after the geometric transformation may be different, we design the method to normalize the number of pixels. These ways enhance the performance of . Finally, we carry out some experiments based on synthetic and real image datasets. We choose commonly used moment invariants for comparison. The results indicate that have better performance than other moment invariants in image retrieval and classification. With , one can compare the similarity between images under the projective transformation without knowing the parameters of the transformation, which provides a good tool to shape analysis in image processing, computer vision and pattern recognition.

General projective transformations, relative projective differential invariants, projective weighted moment invariants, object recognition.

1 Introduction

How to measure the similarity or difference of a scene or an object observed from different viewpoints is one of fundamental problems in computer vision and object recognition. As shown in Fig. 1, a general camera model is the ”pin-hole”, a projection of projective transformation. The severe geometric deformation caused by projective transformation brings some difficult to judge whether the two images of a 3D scene contain the same object. So, it’s necessary to construct a kind of image features which are invariant to general projective transformations.

Geometric moment invariant is a good idea, which was first introduced into computer vision community by [5], using moments to define invariants of images and shapes. The definition of moment has a deep background in physics and mathematics. Some geometric moment invariants under transformations of translation, rotation and scaling, or even affine were built up and have been widely used in applications as global features [2, 9, 18, 30]. For the case of small objects with respect to large distance of camera-to-scene, the effect of projective deformation is slight, which can be approached by affine transformation as usual. For computer vision tasks like robotic vision and object recognition that require precise calculation, the effect of projective deformation can no longer be neglected. In the light of the classical theory of geometric moment invariants, it seems easy to seek for projective invariants in the similar way. But in fact, a major problem occurs that the projective transformation is not linear in transform parameters, whose Jacobian performs as a function of the coordinates instead of a constant. Therefore, such quality determines the fact that the traditional way of generating geometric invariants is not longer valid. In order to adapt the general solution to projective transformation, a possible way goes to reconsider the structure of traditional geometric invariants to combine the structural method with the essence of projective transformation. For general projective transformations, [17] attempted to express projective invariants in the form of infinite series of moments. Unfortunately, the definition setting was problematic and unwarranted. The experimental results presented in [17] were also unsatisfactory. [19, 22, 23, 24] gave some sorts of projective invariants under strict constraints or subsets, which lose the generality and therefore can’t to extend for broader applications. It is worth noting that the determinant consisting of partial differentials was designed in [25] which can be used to construct general projective moment invariants. Also, they defined differential moments which are named as D-moments. But we will point out that one of the structural formulas proposed in [25] was wrong. Therefore, all experiments based on this formula didn’t have reference value.


Fig. 1: Projective transformation from a pin-hole camera model

In addition, the studies of differential invariants should be concerned. Unlike moment invariants constructed by integral frames, differential invariants are not global features but local features. Theoretically, they have good invariance to interference such as occlusion. So far, differential invariants have been studied mainly for affine transformations. In [13], Olver constructed affine differential invariants by using the moving frame method. Then, he found that two kinds of affine differential invariants can be use to define the affine gradient [14]. Wang et al. presented a new method to derive a special type of affine differential invariants [21]. Given some functions defined on the plane and the affine group acting on the plane, there were induced actions of the group on the functions and their derivative functions. But we must point out that these methods are not easy to use. Recently, Li et al. found the isomorphism between differential invariants and geometric moment invariants to general affine transformations [7]. If affine moment invariants were known, relative affine differential invariants can be obtained by the substitution of moments by partial derivatives with the same order. This method made the construction of affine differential invariants very easy. Also, they gave a construction formula of relative projective differential invariants. In [10], Mo et al. combined affine differential invariants with affine moment invariants, and obtained affine weighted moment invariants. Compared with the traditional moment invariants, this kind of invariants achieved better results in image retrieval due to the use of both local and global information. This method provides a new way for our research.

The main contributions of this paper are summarized as follows.

  1. The projective differential invariants are explicitly reported, which are invariant under general projective transformations.

  2. It is proved that a kind of projective weighted moment invariants () exist in terms of weighted integration of images, with as the weight functions.

  3. The reasons which cause to be calculated imprecise are analyzed. The appropriate method to calculate partial derivatives of discrete images is selected. Also, the normalized method is used to deal with the change in the number of pixels. These ways make have practical value.

  4. Experimental results obtained by using the projective invariants based on correct structural formulas are obtained for the first time. By comparing with other moment invariants, it’s obviously that have advantages.

In Section 2, some definitions and notations are given. Then, we introduce some related works and point out their mistakes and limitations in Section 3. Section 45 are the major parts of this paper. We give the definitions of and the structural framework of . Experiments and discussions are shown in Section 6. At last, some conclusions are given in Section 7.

2 Basic Definitions and notations

In this section, we will introduce some basic definitions and notations that will be used to construct and .

2.1 Invariants and Integral Invariants

Given a set of parameters , the transformed parameters are under the transformation T with the correspondence of to , to , etc. If there is a function satisfying (1), then is called relative invariant.

(1)

when the power , is called absolute invariant. Integral invariants are defined in the form of multiple integrations [5, 30]. It’s crucial to find correct integral cores remaining the same constructions under corresponding transformations. Then, they are used to calculate the multiple integrations in (2), where is the integral core, .

(2)

2.2 General Projective Transformation

A general projective transformation between points and is defined by (3).

(3)

where all parameters are real, and . If both and are zero, it reduces to an affine transformation. Notice that there are totally 9 parameters in (3). Since the numerators and the denominators can be divided by a nonzero constant, a common way to simplify (3) is to let r equal to constant 1 to eliminate the parameter without losing the generality. Therefore, there are 8 independent parameters for a general projective transformation which includes some other transformations as its special cases. Let A represent the coefficient determinant of (3).

(4)

The Jacobian determinant J of the projective transformation (3) is

(5)

2.3 Moment and Weighted Moment

For an image function , the order geometric moment is defined as an double integral

(6)

If there is another weight function besides the image function in the integrand, it is called weighted moment which is defined by

(7)

where is called the weight function.

3 Related work

From the viewpoint of projective geometry, the only invariant property for general projective transformations is the cross ratio, which can be expressed in several ways. The cross ratio is defined locally for points on a straight line or line bundles and is not easy to directly be applied to images [11]. As a result, the researchers began to construct new projective invariants of images and achieved some results. In this section, some previous work directly related to this paper will be described. Also, we will point out their limitations and mistakes.

3.1 Restricted Projective Transformation

Because of the complex structure of general projective transformations, some researchers focused on restricted projective transformations firstly. In [19], Voss and Susse defined a kind of finite projective invariant by

(8)

where , is coefficient, and .

These invariants fit for a special case of general projective transformations called ”rein transform” with , which means

(9)

In fact, it is obviously that (8) is a kind of weighted moments, with as the weight function.

[22, 23, 24] gave some new results. In [22], Wang et.al extended the moment definition to allow the power of coordinates varies from non-negative integers to arbitrary integers and gave moment-like invariants in rational form for a special case defined by (10). This kind of invariants was constructed by (11), where is a positive even integer and is a positive integer.

(10)
(11)

[24] proposed co-moment to construct projective invariants, under the condition that the correspondence of two reference points in images was known beforehand. It was said to be the first paper to establish a set of easily implemented projective invariants for 2D images. But this method is not free, independent technique and relying on the two correspondent points seriously restricts its application. For example, it is not easy for image retrieval in large image database, the amount of calculation is unconceivable for comparison between the given image and database images.

3.2 General Projective Transformation

Comparatively speaking, the number of studies conducted on general projective transformations is much less. By using Lie Group theory, [20] proved that there are no finite projective invariants. [17] gave another proof by decomposing the general projective transformation (3) into eight one-parameter transformations. In fact, the real meaning of those proofs can be understood as that there are no simple or direct projective moment invariants. Other forms of invariants or weighted moment invariants may still be possible. Suk and Flusser tried to extend their work on affine moment invariants to projective moment invariants in [17]. They noticed that the determinant of three points

(12)

would be changed to , if was transformed into , by the projective transformation (3).

(13)
(14)

The relationship (14) was used to construct a kind of infinite projective moment invariants, which can be expressed as the form of infinite series of moments. The basic idea was that the Jacobian determinant in the transformation (3) contains a denominator of power 3 as in (5). If the frequency of each points appearing in the definition of moment integration is exactly three, the Jacobian determinants and the denominator in (14) would be canceled out. In this way, given three points and in the original image, the projective moment invariants can be defined by

(15)

where is the image intensity, . It is easy to prove that (15) is a kind of projective invariants. By expanding as power series of , Suk and Flusser got an infinite moment series which was called infinite projective invariant. More points involvements were allowed with the definition. They also got two instance of (15) by setting the point number .

However, two problems should be noted here.

  1. The infinite projective invariant was defined in moment, its form was popular and the calculation was straightforward. But this method was difficult to use. Its error limit was hard to evaluate and the calculation may be time-consuming. We had to compute a large number of moments to ensure that the invariants are stable.

  2. More importantly, Suk and Flusser gave three instances by setting the point number , but only is correct. For , the invariants are always zero.

The second problem can be explained simply as following. When we exchange the order of integration, the final result of (15) does not change. So, when , we have

(16)

Meanwhile, after the change of variables, we can obtain

(17)

And from the property of determinant, the change of point order will change its sign.

(18)

Thus, by using (16), (17) and (18), we can find that

(19)

When , the reason is similar. Therefore, the experimental result didn’t have reference value, which was obtained by using the invariant in [18].

Recently, Wang et al. proposed a kind of projective invariants in [25]. Let an image be transformed by (3) into the image . and in are the corresponding points of and in . Suppose that both and have the first-order partial derivatives.

Then, they defined two determinants by

(20)
(21)

There is a relation

(22)

where

(23)

By using (12) and (20), Wang et al. constructed two kinds of projective invariants, which were defined by

(24)
(25)

where .

These projective invariants can be represented as polynomials of D-moment which was defined by

(26)

where . Obviously, (26) is a kind of weighted moments. Also, there are two problems we have to pay attention to.

  1. In [25], all experimental results were obtained by using the instances of , when . Unfortunately, similarly to (15), (24) is always zero. Because we can find that . Therefore, the results in [25] are not valid.

  2. The definition of is theoretically correct. But, only can be used in practice. When , the expansion of contains more than 70 million terms. And as grows, the number of terms increases exponentially. This means that (25) only constructs one projective invariant.

In summary, the projective invariants that have been obtained with practical value are all for restricted projective transformations. So, their application scenarios are greatly limited. Two kinds of invariants for general projective transformations have obvious defects in theories. Thus, the problem of real projective invariants for general projective transformations has been being widely open.

4 Relative Projective differential invariants

In this section, we will give two definitions of . One is proposed by Li et al. in [7]. Another is defined for the first time in this paper.

Definition 1. Support that an image function has the second-order partial derivatives. Then can be defined by

(27)

Theorem 1. Let an image be transformed by (3) into the image . Suppose that and both have the second-order partial derivatives, then we have

(28)

where

(29)

The proof of Theorem 1 is obvious by using . It should be noted that has a geometric meaning. If an image is taken as a ”curved surface” defined on 2D region, the traditional differential geometry methods can be applied on it. There are two movement invariants on curved surfaces in Euclidean space, Gaussian curvature and mean curvature [3]. They are defined by

(30)
(31)

The numerator of (30) is a Hessian determinant, and the numerator of can be separated into two parts, and . is defined by

(32)

It’s well known that Laplace descriptor is a rotation invariant. In [13], Olver pointed out that the numerator of (30) and in (31) were two relative affine differential invariants. With further analysis, Li et al. found the interesting result that is also a relative projective differential invariant [7]. Then, we will define a new structural formula of .

Definition 2. Support that an image has the third-order partial derivatives. Then we have

(33)

where is defined by

(34)

Theorem 2. Let an image be transformed by (3) into the image . Suppose that both and have the third-order partial derivatives, then we have

(35)

where is defined in the way similar to that of . Theorem 2 can be proved very easily by , too.

We believe that there must be other structural formulas of . Especially, [7] proposed that the isomorphism between differential invariants and geometric moment invariants under general affine transformations. This makes it very easy to obtain affine differential invariants. As we all know, the affine transformation group is a subgroup of the projective transformation group. A projective invariant must be invariant to affine transformations. Thus, while affine differential invariants are well established, could be screened out from affine ones.

5 The Structural framework of PIs

In this section, we will present how to construct , firstly. Then, a new weighted moment, which can be used to calculate , will be defined. Finally, some instances of will be given for experiments in Section 6.

5.1 The Construction of PIs

By using (12), (20), (27) and (33), we can obtain two construction methods of .

Theorem 3. Let an image be transformed by (3) into the image . , and in are the corresponding points of , and . Suppose that and both have the second-order partial derivatives. We have

(36)

where . Then, there is a relation

(37)

where

(38)

Proof. According to (5), (14), (22) and (28) in Section 2.2, Section 3.2 and Section 4, we have

(39)

Therefore, it is proved that has invariance to general projective transformations. Similarly, we can use to construct .

Theorem 4. Let an image be transformed by (3) into the image . , and in are the corresponding points of , and . Suppose that and both have the third-order partial derivatives. We have

(40)

where . Then, there is a relation

(41)

where

(42)

The proof of Theorem 4 is similar to that of Theorem 3, by using (5), (14), (22) and (35). Firstly, we must point out that there are many other structural formulas which can be designed. Here is not listed one by one. Then, it is obviously that for any n, and are not always zero. Because and . Thus, we don’t make the mistake in [17, 25].

5.2 The Definition of Projective Weighted Moment

In order to calculate invariants more conveniently, have to be represented as polynomials of some kinds of moments. So, we need to give the definitions of these moments.

Definition 3. For an image , we define two kinds of weighted moments, which are named projective weighted moments ().

(43)
(44)

Comparing with (26), it can be found that are used to construct the weight functions.

5.3 The Instances of PIs

We can use (36) and (40) to construct instances of . By setting , we can get 4 instances. Their expanded forms are defined by (45), (46) and (47).

(45)
(46)