Classification of curves in 2D and 3D via affine integral signatures

# Classification of curves in 2D and 3D via affine integral signatures

## Abstract

We propose a robust classification algorithm for curves in 2D and 3D, under special and full groups of affine transformations. To each plane or spatial curve we assign a plane signature curve. Curves, equivalent under an affine transformation, have the same signature. The signatures introduced in this paper are based on integral invariants, which behave much better on noisy images than classically known differential invariants. The comparison with other types of invariants is given in the introduction. Though the integral invariants for planar curves were known before, the affine integral invariants for spatial curves are proposed here for the first time. Using the inductive variation of the moving frame method we compute affine invariants in terms of Euclidean invariants. We present two types of signatures, the global signature and the local signature. Both signatures are independent of parameterization (curve sampling). The global signature depends on the choice of the initial point and does not allow us to compare fragments of curves, and is therefore sensitive to occlusions. The local signature, although is slightly more sensitive to noise, is independent of the choice of the initial point and is not sensitive to occlusions in an image. It helps establish local equivalence of curves. The robustness of these invariants and signatures in their application to the problem of classification of noisy spatial curves extracted from a 3D object is analyzed.

## 1 Introduction

Curves and surfaces are fundamental entities in computer vision and pattern recognition. For example, the features of 3D or 2D objects are often spatial or planer curves, and their classification often reduces to a classification of curves under Euclidean, affine or projective transformations. A direct comparison of curves, such as shape matching, generally requires registration, and the ensuing complexity and difficulty in its application in many important problems, have recently led to a renewed research interest in transformation invariants.

Although geometric invariants have been applied to problems in computer image recognition and processing for decades [21, 22, 6, 29, 24], designing robust algorithms that are tolerant to noise and image occlusion remains an open problem. We start by providing a brief overview of various types of invariants that have appeared in computer vision literature. Euclidian differential invariants, such as Euclidean curvature and torsion for space curves, are the most classical. The affine and projective counterparts of curvature and torsion are well known. The dependence of curvature and torsion on high order derivatives (up to order 3 for the Euclidean group, 6 for the affine group and 9 for the projective group), makes numerical approximation of these invariants highly sensitive to noise, and therefore impractical in computer vision applications. This has motivated a high interest in other types of invariants such as semi-differential, or joint invariants [28, 23, 2] and various types of integral invariants [25, 13, 18, 20, 19]. Integral invariants of a curve in the latter references depend on quantities obtained by integration of various functions along it. Since integration reduces the effect of noise, these invariants hold a clear advantage in practical applications.

While explicit expressions for integral invariants are known for plane curves in 2D, they have thus far remained elusive for spatial curves in 3D, primarily on account of their computational complexity. With an increasing availability of 3D data acquisition systems and subsequent emerging applications, interest in 3D analysis and hence robust integral invariants for curves in 3D is becoming essential.

In [8] a hybrid integro-differential affine invariant which only uses first order derivatives along with integrals were computed. Although a performance improvement over classical differential invariants is obtained, the presence of first order derivatives still affects the performance.

In [9], we obtain for the first time explicit formulae of integral Euclidean and affine invariants for spatial curves in 3D. Hann and Hickman [13] introduced and computed these for plane curves. The type of integral invariants, computed in this paper, may be compared with moment invariants [27, 30]. We emphasize, however, the following difference: a moment invariant corresponds a number to a shape, whereas an integral invariant corresponds a curve to a curve. The standard action of the affine group on induces an action on curves. Following the approach of [13] we prolong this action to certain integral expressions, called potentials, and then compute invariants that depend on these integral variables. A direct extension of [13] to 3D, using a Fels-Olver moving frame construction [7] is conceptually straightforward, but the computational complexity makes the problem intractable. An inductive implementation of the moving frame construction, proposed in [17], dramatically simplifies the algebraic derivations, as it allows one to construct invariants for the entire group from invariants of its subgroups: in our case affine invariants in terms of Euclidean ones.

The integral invariants defined in [13] and [9] are sensitive to parameterization, or sampling of the curve in the discrete case. A uniform parameterization is required for two curves to be compared. In order to overcome this limitation, we propose in this paper local and global 2D/3D signatures for the special affine and full affine group. Signatures based on integral invariants are defined in an analogous way as signatures based on differential and joint invariants (see [3] for example). The global signature of a curve depends on the choice of its initial point and does not allow a comparison of its fragments. It is therefore sensitive to occlusions. The local signature, although slightly more computationally involved, is independent of the choice of the initial point and is not sensitive to the occlusion effects in the image. It allows to establish a local equivalence of curves being compared.

In Section 2, after reviewing the basic facts about group actions and invariants, we define the notion of integral jet bundle and integral invariants. Explicit formulae for affine integral invariants in terms of Euclidean for curves in 2D and 3D are given in Section 3, along with their geometric interpretation. In Section 4 we define a global integral signature which classifies curves with a given initial point up to affine transformations. We also define a local signature that is independent of the initial point of a curve. In Section 5 a discrete approximation of the signature construction is tested on curves extracted from 3D objects. The curves are given as discrete sequences of points, with possibly the additive noise. The experiments show that signature construction gives a robust method for classification of curves under affine transformations. The method can be easily adopted to a smaller Euclidean group.

## 2 Group Action and Invariants

In this section we review the basic terminology for the group actions and invariants, as well as the concept of prolonging the action to jet spaces and the notion of differential invariants. We then introduce the notion of integral jet space and define the corresponding prolongation of the action which gives rise to integral invariants.

### 2.1 Definitions

###### Definition 2.1

An action of a group on a set is a map that satisfies the following two properties:

1. , , where is the identity of the group.

2. , for all and .

For and we write

###### Definition 2.2

The orbit of a point is the set .

###### Definition 2.3

A function is called invariant if

 f(g⋅s)=f(s),∀g∈G and ∀s∈S. (1)

Invariant functions are constant along each orbit and can be used to find equivalence classes of objects undergoing various types of transformations.

Let denote a group of non-degenerate matrices with real entries. Its subgroup of matrices with determinant is denoted by . The orthogonal group is , while the special orthogonal group is . The semi-direct product of and is called the affine group: . Its subgroup is called the special affine group. The Euclidean group is . Its subgroup is called the special Euclidean group.

In the paper we consider the action of the affine group and its subgroups on curves in by a composition of a linear transformation and a translation, for and :

 ⎛⎜ ⎜ ⎜⎝¯¯¯¯¯¯¯¯¯¯¯xi(t)⋮¯¯¯¯¯¯¯¯¯¯¯¯xn(t)⎞⎟ ⎟ ⎟⎠=A⎛⎜ ⎜⎝x1(t)⋮xn(t)⎞⎟ ⎟⎠+⎛⎜ ⎜⎝v1⋮vn⎞⎟ ⎟⎠. (2)

where matrix defines a linear transformations and vector defines a translation.

### 2.2 Prolongation of a group action

Our goal is to obtain invariants that classify curves up to affine transformations. The classical method of obtaining such invariants is to prolong the action to the set of derivatives of a sufficiently high order

 ¯¯¯¯¯¯¯¯x(1)i(t)=d¯¯¯¯¯xi(t)dt,¯¯¯¯¯¯¯¯¯¯¯¯¯x(k+1)i(t)=d¯¯¯¯¯¯¯¯x(k)i(t)dt. (3)
###### Definition 2.4

Functions of that are invariant under the prolonged action (3) are called differential invariants of order .

For the Euclidean action on curves in 3D, the two lowest order invariants are called curvature and torsion, and are classically known in differential geometry. Analogous invariants for the affine and projective groups are also known.

As noted in the introduction, differential invariants are highly sensitive to noise. We extend the approach of [13] from planar curves to curves in a space of arbitrary dimension. Let parametrized by be a curve. We define integral variables

 x[α1,…,αn]i(t)=∫t0x1(t)α1⋯xn(t)αndxi(t), (4)

where the integrals are taken along the curve and are non-negative integers, such that . We call the order of integral variables, and there are totally of variables of order less or equal to . Integration-by-parts formula dictates certain relations among the integral variables, the coordinates of an arbitrary point on a curve , and the coordinates of the initial point . For example

 x[0,1,0,…,0]1(t)=∫t0x2(t)dx1(t)=x2(t)x1(t)−x02x01−∫t0x1(t)dx2(t)=x2(t)dx1(t)−x02x01−x[1,0,0,…,0]2(t).

It is not difficult to show that there are

 Nl=(n−1)(n+l)!n!l!−n−1∑m=1(n−m+l)!(n−m)!l!

independent integral variables of variables of order less or equal to . A canonical choice of such variables is given by:

 x[αj1,…,αjk]i(t)=∫t0xj1(t)αj1⋯xjk(t)αjkdxi(t), where j10 and i>j1. (5)

For example variable is canonical, but is not canonical.

###### Definition 2.5

Let be an -dimensional space of independent integral variables of order and less, then the integral jet space of order (denoted ) is defined to be a direct product of and two copies of , i.e . The coordinates of the first copy of represent an arbitrary point on a curve , and coordinates of the second copy of represent the initial point .

The action (2) can be prolonged to the curves on jet space as follows:

 ⎛⎜ ⎜⎝¯¯¯¯¯xi(t)⋮¯¯¯¯¯¯xn(t)⎞⎟ ⎟⎠ = A⎛⎜ ⎜⎝x1(t)⋮xn(t)⎞⎟ ⎟⎠+⎛⎜ ⎜⎝v1⋮vn⎞⎟ ⎟⎠ ⎛⎜ ⎜ ⎜ ⎜⎝¯¯¯¯¯x0i⋮¯¯¯¯¯¯x0n⎞⎟ ⎟ ⎟ ⎟⎠ = A⎛⎜ ⎜⎝x01⋮x0n⎞⎟ ⎟⎠+⎛⎜ ⎜⎝v1⋮vn⎞⎟ ⎟⎠, (6) ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯x[α1,…,αn]i(t) = ∫t0¯¯¯¯¯x1α1(t)⋯¯¯¯¯¯¯xnαn(t)d¯¯¯¯¯xi(t).

It is important that the integration-by-parts relations among the integral variables are respected by the prolonged action, and therefore the action on the integral jet space is is well defined.

###### Definition 2.6

A function on which is invariant under the prolonged action (2.2) is called integral invariant of order .

By introducing new variables

 Xi(t)=xi(t)−x0i,i=1,…,n (7)

and making the corresponding substitution into the integrals, we reduce the problem of finding invariants under the action (2.2) to an equivalent but simpler problem of finding invariant functions of variables under the action of defined by

 ⎛⎜ ⎜ ⎜⎝¯¯¯¯¯¯Xi(t)⋮¯¯¯¯¯¯¯Xn(t)⎞⎟ ⎟ ⎟⎠ = A⎛⎜ ⎜⎝X1(t)⋮Xn(t)⎞⎟ ⎟⎠ (8) ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯X[α1,…,αn]i(t) = ∫t0¯¯¯¯¯¯¯X1α1(t)⋯¯¯¯¯¯¯¯Xnαn(t)d¯¯¯¯¯¯Xi(t).

Invariants with respect to (2.2) may be obtained from invariants with respect to (8) by making substitution (7).1 Invariants with respect to a very general class of actions of continuous finite-dimensional groups on manifolds can be computed using Fels-Olver generalization [7] of Cartan’s moving frame method (see also its algebraic reformulation [14]). The method consists of choosing a cross-section to the orbits and finding the coordinates of the projection along the orbits of a generic point on a manifold to the cross-setion (see Appendix for more details). It can be, in theory, applied to find the invariants under the action described by (8) for arbitrary . Hann and Hickman [13] used Fels-Olver method to compute integral invariants for planar curves () under affine transformations and a certain subgroup of projective transformations. The corresponding derivation of invariants for spatial curves () remained, however, out of reach due to computational complexity (it is often the case in the computational invariant theory that practical computations become unfeasible as the dimension of the group increases, despite the availability of a theoretical method to compute them [26, 5].) In [9], we derived, for the first time, integral invariants under the Euclidean and affine transformations for spatial curves using an inductive variation of the moving frame method [17], which allowes one to construct invariants for the entire group in terms of invariants of its subgroups: in our case, affine invariants in terms of Euclidean. Explicit derivation of invariants for curves of higher in the space of higher dimension () remains an open problem, which seems at present, to be of more theoretical, than of practical interest.

## 3 Integral invariants in 2D and 3D

In this section we present explicit formulas for integral invariants for (plane curves) and (spatial curves) under the affine action (2.2). The affine invariants are written in terms of the Euclidean invariants. We discuss their properties and geometric interpretation. The inductive derivation of these invariants is outlined in the Appendix.

### 3.1 Integral Affine Invariants for Curves in 2D

The standard affine group action on curves in :

 (¯¯¯x(t)¯¯¯y(t))=(a11a12a21a22)(x(t)y(t))+(v1v2),det(a11a12a21a22)≠0,

prolongs to the action on integral variables up to the third order.

By translating the initial point to the origin and making the corresponding substitution in the integrals, we reduce the problem to computing invariants under the action (8) with . Among 12 integral variables

 X[i,j](t) = ∫t0X(t)iY(t)jdX(t),j≠0,i+j≤3 Y[i,j](t) = ∫t0X(t)iY(t)jdY(t),i≠0,i+j≤3 (9)

we make a canonical choice of 6 independent: as suggested by formula (5). The rest can be expressed in terms of those using integration by parts formulas, as follows:

 X[0,1] = XY−Y[1,0], X[0,2] = XY2−2Y[1,1], X[1,1] = 12X2Y−12Y[2,0], X[1,2] = 12X2Y2−Y[2,1], (10) X[0,3] = Y3X−3Y[1,2], X[2,1] = 13X3Y−13Y[3,0].

This reduces the problem to finding invariants under the following -action on . Denote . The action is defined by the following equations:

 ¯¯¯¯¯X = a11X+a12Y,¯¯¯¯Y=a21X+a22Y, ¯¯¯¯¯¯¯¯¯¯¯¯Y[1,0] = (detA)Y[1,0]+12a11a21X2+a12a21XY+12a12a22Y2, ¯¯¯¯¯¯¯¯¯¯¯¯Y[1,1] = (detA)(a22Y[1,1]+12a21Y[2,0])+13a212a11X3+12a21(a11a22+a12a21)X2Y +a21a12a22XY2+13a222a12Y3 ¯¯¯¯¯¯¯¯¯¯¯¯Y[2,0] = (detA)(a11Y[2,0]+2a12Y[1,1])+13a112a21X3+a11a12a21X2Y+a212a21XY2 +13a11a12a21Y3, ¯¯¯¯¯¯¯¯¯¯¯¯Y[1,2] = (detA)(a222Y[1,2]+13a221Y[3,0]+a21a22Y[2,1]) (11) +14a11a213X4+13a221(2a11a22+a12a21)X3Y +12a21a22(2a12a21+a11a22)X2Y2+a12a21a222XY3+14a12a223Y4, ¯¯¯¯¯¯¯¯¯¯¯¯Y[2,1] = (detA)((a11a22+a12a21)Y[2,1]+2a12a22Y[1,2]+23a11a21Y[3,0]) +14a211a221X4+13a11a21(a11a22+2a12a21)X3Y+12a12a21(2a11a22+a12a21)X2Y2 +a212a21a22XY3+14a122a222Y4, ¯¯¯¯¯¯¯¯¯¯¯¯Y[3,0] = (detA)(a211Y[3,0]+3a212Y[1,2]+3a11a12Y[2,1]) +14a311a21X4+a211a12a21X3Y+32a11a212a21X2Y2+a312a21Y3X+14a312a22Y4.

We restrict the above action to the subgroup of rotation matrices by setting . We use the moving frame method to find invariants as described in the Appendix. Computationally this reduces to the substitution , where in (3.1). The resulting non-constant expressions comprise a set of generating invariants for the action:

 XSE = √X2+Y2=r, Y[1,0]SE = Y[1,0]−XY2, Y[1,1]SE = 1r(Y[1,1]X−12Y[2,0]Y−16X2Y2), (12) Y[2,0]SE = 1r(Y[2,0]X+2Y[1,1]Y−13X3Y−23XY3), Y[1,2]SE = 1r2(Y[1,2]X2−Y[2,1]XY+13Y[3,0]Y2−112X3Y3), Y[2,1]SE = 1r2(Y[2,1](X2−Y2)+2Y[1,2]XY+23Y[3,0]XY−14X2Y4−112X4Y2), Y[3,0]SE = 1r2(Y[3,0]X2+3Y[1,2]Y2+3Y[2,1]XY−14X5Y−34X3Y3−34XY5).

The invariants with respect to the special Euclidean group are obtained by making a substitution of and in the above expressions (3.1): 2 We note that since the denominators in the above formulas are invariant, the numerators are also invariant.

We use the inductive approach, described in the Appendix, to build invariants under the -action defined by Eq.(3.1) with the condition The inductive method yields -invariants in terms of -invariants (3.1):

 Y[1,0]SA = Y[1,0]SE=Y[1,0]−XY2, Y[1,1]SA = XSEY[1,1]SE=Y[1,1]X−12Y[2,0]Y−16X2Y2, Y[1,2]SA = Y[1,2]SEX2SE=Y[1,2]X2−Y[2,1]XY+13Y[3,0]Y2−112X3Y3, (13) Y[2,1]SA = Y[2,1]SE−Y[2,0]SEY[1,1]SEY[1,2]SE, Y[3,1]SA = 1X2SE⎛⎜⎝Y[3,0]SE+32Y[2,0]SEY[1,1]SEY[2,1]SE+34⎛⎝Y[2,0]SEY[1,1]SE⎞⎠2Y[1,2]SE⎞⎟⎠ (14)

By replacing with in Eq.(3.1) we return to the integral jet space coordinates. In particular, .

The following three special affine invariants are used in the next section to solve the classification problem with respect to both special and full affine groups:

 I1 = Y[1,0]SA=Y[1,0]−12XY, I2 = Y[1,1]SA=Y[1,1]X−12Y[2,0]Y−16X2Y2, (15) I3 = Y[1,2]SA=Y[1,2]X2−Y[2,1]XY+13Y[3,0]Y2−112X3Y3.

To obtain invariants with respect to the full affine group we need to consider the effect of reflections and arbitrary scaling on the above invariants. We note that the transformation and induces the transformation , and . The following rational expressions are thus invariant with respect to the full affine group:

 IA2 = I2I21=Y[1,1]X−12Y[2,0]Y−16X2Y2(Y[1,0]−12XY)2, IA3 = I3I31=Y[1,2]X2−Y[2,1]XY+13Y[3,0]Y2−112X3Y3(Y[1,0]−12XY)3 (16)

The first of the above invariants is equivalent to the one obtained in [13].

### 3.2 Geometric Interpretation of Invariants for Plane Curves

The first two integral invariants (3.1) readily lend themselves to a geometric interpretation. Invariants is the signed area between the curve segment and the secant (see Figure 1). Indeed, the term in the invariant is the signed area between the curve (whose initial point is translated to the origin) and the -axis, while is the signed area of the triangle . Their difference is the area . Since the - action preserves areas, is clearly an invariant.

The interpretation of is slightly more subtle. Using that and rearranging the terms we rewrite as

 I2=−13((X2Y2−3XY[1,1])+(X2Y2−3YX[1,1])), where X=x−x0,Y=y−y0. (17)

Further, the curve is lifted from 2D to 3D by defining (similarly to the kernel idea), and (17) is rewritten as

 I2=−13((XYZ−3X∫t0ZdY)+(XYZ−3Y∫t0ZdX)) where Z=XY=(x−x0)(y−y0). (18)

The geometric meaning of is illustrated in Figure 2. The term is the signed area “under” the plane curve in the -plane. Thus is the signed volume C under the surface in Figure 2. Since is the signed volume of a rectangular prism (C+D in Figure 2), then is the signed volume of the rectangular prism (C+D) minus three times the volume C “under” the surface . Interchanging and we obtain a similar interpretation for .

### 3.3 Integral Affine Invariants for Curves in 3D

The standard affine group action on curves in :

 ⎛⎜⎝¯¯¯x(t)¯¯¯y(t)¯¯¯z(t)⎞⎟⎠=⎛⎜⎝a11a12a13a21a22a23a31a32a33⎞⎟⎠⎛⎜⎝x(t)y(t)z(t)⎞⎟⎠+⎛⎜⎝v1v2v3⎞⎟⎠,det⎛⎜⎝a11a12a13a21a22a23a31a32a33⎞⎟⎠≠0,

prolongs to the action to integral variables up to second order. We translate the initial point to the origin, and make the corresponding substitution in the integrals. This reduces the problem to computing -invariants under the action (8) with . Among 21 integral variables

 X[i,j,k](t) = ∫t0X(t)iY(t)jZ(t)kdX(t),j+k≠0,i+j+k≤2, Y[i,j,k](t) = ∫t0X(t)iY(t)jZ(t)kdY(t),i+k≠0,i+j+k≤2, (19) Z[i,j,k](t) = ∫t0X(t)iY(t)jZ(t)kdZ(t),i+j≠0,i+j+k≤2,

we choose 11 independent: 3 The rest can be expressed in terms of those using the integration-by-parts formula. Using the inductive approach, we first compute the invariants with respect to rotations . We find the following independent invariants. We obtain -invariants by replacing with ( See Appendix for details of the derivation.)

 XSE = √X2+Y2+Z2, Z[0,1,0]SE = XYZ−2XZ[0,1,0]+2YZ[1,0,0]−2ZY[1,0,0]2√X2+Y2+Z2, Y[1,0,0]SE = Z[0,2,0]RY[1,0,0]R+2Z[0,1,1]RZ[1,0,0]R√Z[0,2,0]R2+4Z[0,1,1]R2, (20) Y[1,0,1]SE = −