Applying Ricci Flow to High Dimensional Manifold Learning

Applying Ricci Flow to High Dimensional Manifold Learning

Yangyang Li fi Yangyang Li Academy of Mathematics and Systems Science Key Lab of MADIS Chinese Academy of Sciences, Beijing University of Chinese Academy of Sciences, Beijing
33email: liyangyang12@mails.ucas.ac.cn
   Ruqian Lu fi Yangyang Li Academy of Mathematics and Systems Science Key Lab of MADIS Chinese Academy of Sciences, Beijing University of Chinese Academy of Sciences, Beijing
33email: liyangyang12@mails.ucas.ac.cn
Received: date / Accepted: date
Abstract

In machine learning, a high dimensional data set such as the digital image of human face is often considered as a point set distributed on a (differentiable) manifold. In many cases the intrinsic dimension of this manifold is low but the representation dimension of data points is high. In order to ease data processing, one uses manifold learning (ML) techniques to reduce a high dimensional manifold (HDM) to a low dimensional one while keeping its essential geometric properties such as relative distances between points unchanged. Traditional ML algorithms often bear an assumption that the local neighborhood of any point on HDM is roughly equal to the tangent space at that point. This assumption leads to the disadvantage that the neighborhoods of points on the manifold, though with very different curvature, will be treated equally and projected to lower dimensional space in the same way. The curvature indifferent way of manifold processing often makes traditional dimension reduction poorly neighborhood preserving. To overcome this drawback we propose to perform an ‘operation’ on the HDM with help of Ricci flow before manifold’s dimension reduction. More precisely, with Ricci Flow we transform each local neighborhood of the HDM to a constant curvature patch. The HDM as a whole is then transformed to a subset of a Sphere with constant positive curvature. We compare our algorithm with other traditional manifold learning algorithms. Experimental results have shown that our method outperforms other ML algorithms with better neighborhood preserving rate.

Keywords:
Manifold Learning Ricci flow Ricci Curvature Dimension Reduction

1 Introduction

In many machine learning tasks, one is often confronted with redundant dimensions of data representation. Manifolds usually arise from data generated in some continuous process. The generated manifold is often embedded in some high-dimensional Euclidean space. In most of the cases the manifold is represented as a discrete data set. An intuitive example is a set of images generated by a continuous changing of face expressions 37 (). Actually, this set of data points can be represented by just a low dimensional set of features accurately. In order to uncover these features, S. Roweis et al. 13 () and J. Tenenbaum et al. 14 () introduced a new research field called manifold learning (in this paper, manifold learning only refers to nonlinear dimensionality reduction technique). It is a perfect combination of classical geometric analysis and computer science. Traditional manifold learning algorithms aim to reduce manifold’s dimension, so that its lower dimensional representation (i.e. the features) could reflect the intrinsic geometric and topologic structure of the high dimensional data points. In general, the existing manifold learning algorithms can be roughly divided into two classes: one is to preserve the global geometric structure of manifold, such as Isomap 14 (); the other one is to preserve the local neighborhood geometric structure, such as LLE 13 (), LEP 15 (), LPP 18 (), LTSA 17 (), Hessian Eigenmap 16 () et al. Isomap aimed to preserve the geodesic distance between any two high dimensional data points, which can be viewed as an extension of Multidimensional Scaling (MDS) 30 (). Local neighborhood preserving algorithms approximated manifolds with a union of locally linear patches (possibly overlapped). After the local patches are estimated with linear methods such as PCA 19 (), the global representation is obtained by aligning the local patches together. Besides using geometry tools, a lot of manifold learning algorithms used statistical knowledge to uncover the lower dimensional structure of manifold 36 (). Manifold learning algorithms have been applied to many fields such as data dimension compression, computer vision, image recognition and more. Despite the success of manifold learning, there are still several problems remaining.

  • Locally short circuit problem: if the embedded manifold is highly curved, the Euclidean distance between any two points is obviously shorter than the intrinsic geodesic distance. In this case the geodesic distance between these two points is often underestimated.

  • Intrinsic dimension estimation problem: since tangent spaces are simply taken as local patches, the intrinsic dimension of manifold cannot be determined by the latter, in particular in case of strongly varying curvature.

  • Curvature sensitivity problem: if the curvature of original manifold is especially high at some point, quite many patches are needed for representing the neighborhoods around this point. But one may not have as many data points as needed to produce enough patches, especially when the data points are sparse.

Among the above mentioned three problems, the third one is critical. It is the background of the other two. To solve this problem is the main target of this paper.

1.1 Motivation

In manifold learning one usually maps an irregularly curved manifold to a lower dimensional Euclidean space directly. This is often unpractical since care is not taken of the varying geometric structure of the manifold at different points. Ricci flow (also called intrinsic curvature flow) is a very useful tool for evolving an irregular manifold and making it converging to a regular one (constant curvature manifold). It has been used to prove the Poincare conjecture 31 (). In this paper we use Ricci flow to regularize the metric and curvature of the generated manifold before reducing its dimension. Our algorithm RF-ML transforms the irregular manifold to a regular constant curvature manifold. Since Ricci flow preserves local structure of manifold and the Riemannian metric of the manifold after Ricci flow is uniform (see section 2), the local relationships among data points in the whole process of algorithm are preserved. The key thought of our method is to construct a Ricci flow equation directly on discrete data points without parametrization and meshes. Under the premise of non-negative curvature, the Ricci flow process evolves the original irregular manifold to a subset of a sphere. Finally traditional manifold learning algorithms can be used to reduce the dimensionality of the (high dimensional) sphere. It is assumed that the data points are distributed on a single open Riemannian manifold to guarantee that the manifold evolves into a punctured sphere under Ricci flow. In the current presentation we only consider Riemannian manifolds with non-negative Ricci curvature. Manifold learning with negative Ricci curvature will be discussed in our next paper. We call this style of algorithm dynamic manifold learning. This paper is just a first attempt in this direction.

1.2 Related Work

In recent years, there have been several works studying intrinsic curvature of sampled points by combining traditional manifold learning techniques in designing curvature aware manifold learning algorithms. Among them Kwang et al. 3 () studied this problem with redefined group Laplacian operator. The redefined group Laplacian matrix was used to measure the pair-wise similarities between data points regarding point to point distance as well as curvature of the local patches. Their experimental results showed that the redefined Laplacian operator led to lower error rates in spectral clustering than the traditional Laplacian operator. W. Xu et al. 4 () 5 () 6 () used Ricci Flow to rectify the pair-wise non-Euclidean dissimilarities among data points and to combine traditional manifold learning algorithms to come up with a new dimension reduction algorithm. Another nonlinear dimension reduction algorithm is the two dimensional Discrete Surface Ricci Flow 1 (), which mainly applied to three dimensional data points distributed on a two dimensional manifold. Chow and Luo have studied the relations between circle packing metric and surface Ricci flow in a theoretical view. Gu et al. 1 () 2 () constructed discrete triangle meshes on three dimensional data points as well as discrete circle packing metric on the triangle meshes. They proposed the discrete surface Ricci flow algorithm using Newton’s method, which can map any curved surface to a normal two-dimensional surface (Sphere, Hyperbolic space, or Plane). Traditional manifold learning algorithms mainly reduce the dimension of especially high dimensional data points, where it is difficult to construct the polygon meshes. Note that all these works on applying Ricci flow to manifold learning are 2-dimensional, no matter whether they are discrete or continuous. They cannot be directly extended to higher dimensional cases.

2 Basic Knowledge

Given data set , where is the number of data points and their dimension. One fundamental assumption of traditional manifold learning is that lie on a -dimensional Riemannian manifold embedded in the high dimensional Euclidean space , where is the manifold itself, its Riemannian metric defined as the family of all inner products defined on all tangent spaces of and is called the ambient space of . On each the tangent space at point , the Riemannian metric is a Euclidean inner product . All the geometric intrinsic quantities (length, angle, area, volume, Riemannian curvature) of the Riemannian manifold can be computed with the Riemannian metric .

2.1 Riemannian Curvature

In general, the Riemannian curvature tensor of a -dimensional Riemannian manifold is represented by a fourth-order tensor, which measures the curvedness of with respect to its ambient space . The directional derivative defined on a Riemannian manifold is Riemannian connection, represented by . Riemannian curvature tensor is a -tensor defined by:

(1)

on vector fields 26 (). Using the Riemannian metric , the Riemannian curvature tensor can be transformed to a -tensor as follows:

(2)

The trace of the Riemannian curvature tensor is a symmetric -tensor called Ricci curvature tensor 26 ():

(3)

In differential geometry, the Riemannian metric is expressed by the first fundamental form. Regarding Riemannian sub-manifold, the Riemannian curvature tensor is captured by the second fundamental form which is a bilinear and symmetric form defined on tangent vector fields . is used to measure the difference between the intrinsic Riemannian connection on M and the ambient Riemannian manifold connection on , where is embedded into . The relationship between and is shown by the following Gauss formula 27 ():

(4)

The corresponding relationship between of and of is shown by the following Gauss equation 27 ():

(5)

If the ambient space is an Euclidean space then . The Riemannian curvature of can be fully captured by the second fundamental form:

(6)

Under local coordinate system, the second fundamental form can be represented by 27 ():

(7)

where is the normal vector field of and is shown by the second derivative of the embedding map about .

2.2 Ricci Flow

Ricci flow is an intrinsic curvature flow on Riemannian manifold, which is the negative gradient flow of Ricci energy. The Ricci flow is defined by the following geometric evolution time dependent partial differential equation 28 (): , where . The Ricci curvature can be considered as a Laplacian of the metric , making Ricci flow equation a variation of the usual heat equation. A solution of a Ricci flow is a one-parameter family of metrics on a smooth manifold , defined in a time interval . In the time interval the Riemannian metric satisfies the metric equivalence condition 28 (), where and . So the relative geodesic distance between arbitrary two neighborhood points on M is consistent under the Ricci flow. In global, the solution of Ricci flow only exists in a short period of time until the emergence of singular points. In 29 () researchers have worked out that the Riemannian manifold of dimension can be transformed to a sphere under Ricci flow, when the sectional curvature satisfies everywhere.

2.3 Statement of the Spherical Conditions

Now coming back to our problem, note that the above constructed patches on data points can be either elliptic with positive sectional curvature or hyperbolic with negative sectional curvature. In this paper we only consider the former case and use elliptic polynomial functions to evaluate the local patches on every point, so that the curvature operator at every point of the Riemannian manifold M is non-negative. In Ricci flow theory, when the intrinsic dimension of embedded manifold , the Riemannian manifold with non-negative Ricci curvature can flow to a Sphere under Ricci flow. For , when the sectional curvature satisfies everywhere, the Riemannian manifold with positive sectional curvature can flow to a Sphere under Ricci flow. So in the Ricci flow step of our algorithm, we apply Ricci flow process under the spherical conditions.

3 Algorithm

In practice, it is difficult to analyze the global structure of a nonlinear manifold, especially when there is no observable explicit structure. Our key idea is that we can uncover the local structure of manifold and the global structure can be obtained by the alignment of the local structures. In this paper we decompose the embedded manifold into a set of overlapping patches and apply Ricci flow to these overlapping patches independently of each other to avoid singular points, since in general Ricci flow on global manifold may encounter singular points when blowing up. The global structure of deformed manifold under Ricci flow can be obtained from the deformed local patches with a suitable alignment.

3.1 RF-ML algorithm

In this subsection, we first give the algorithm process of RF-ML. Then we give some detailed analysis of these algorithm steps.

Our algorithm RF-ML is mainly divided into five steps:

  • Find a local patch (neighborhood) for each data point by using the -nearest neighbor method. In order to find the -nearest neighborhood, we need to define a distance metric to measure the closeness of arbitrary two points. The distance metric we use in this step is the Euclidean metric.

  • Construct a special local coordinate system on every point . In the neighborhood we estimate the local patch information of with a covariance matrix , , where is the mean vector of the -nearest data points. The first eigenvectors with maximal eigenvalues of form a local orthonormal coordinate system of on . The last eigenvectors form a local orthonormal coordinate system of normal space.

  • Determine the intrinsic dimension of local patches by the number of the principle components. In practice due to the different curvature of local patches, the local dimension of each patch may be in practice not all the same. We choose .
    If , stop the algorithm. The manifold’s dimension is not reducible.
    If , continue the algorithm.

  • Construct Ricci flow equations on local patches and then let the overlapping patches flow independently into local spherical patches with constant positive Ricci curvature . Details will be shown below.

  • Align the discrete sphere patches into a global subset of a sphere with positive curvature , where . is the corresponding representation of . Details will be shown below.

  • Reduce the dimensionality of the subset using traditional manifold learning algorithms, where the distance metric between arbitrary two points on is the metric of the sphere other than the Euclidean. The -dimensional representations are .

The intuitive representation of algorithm RF-ML is shown in Figure 2. And the brief algorithm procedure is shown in Algorithm 1.

Figure 1: The intuitive process of algorithm RF-ML. The sub-images from left to right are respectively: input data points distributed on manifold M; the overlapping patches; overlapping patches flowing into discrete spherical patches using Ricci flow; alignment of spherical patches to a subset of a sphere. On the bottom is the representation of data points in low dimensional Euclidean space

Step 4) above is to minimize the value of the energy function such that the Ricci curvature at every point converges to a constant curvature. The curvature energy function is as follows:

(8)

where is the value of constant non-negative Ricci curvature, the embedded manifold where the data points are distributed on and is called the Ricci curvature. The convergence of Ricci flow proved in the following theorem 3.1 indicates that the energy function can provide the optimal solution. In order to obtain the minimum solution of the curvature energy function, we calculate the solution step by step with the help of Ricci flow. For the purposes of this paper, it is not our interest to learn the structure of a single Riemannian manifold , but rather a one-parameter family of such manifolds , parameterized by a ‘time’ parameter . The process is controlled by Ricci flow, until converges to a Riemannian manifold with constant curvature where the global set of patches (step 5) distributed on. In theorem 3.2 we prove that the Riemannian manifold is diffeomorphic to the original Riemannian manifold .

The discrete Ricci flow equation as well as the corresponding iterative equations defined on the discrete data points is constructed as follows:

Suppose the local coordinates of any point under local ambient coordinates are represented as . The first coordinates can be seen as the local natural parameters of . A smooth representation of local patch under the local coordinate system is described by:

(9)

where is a coordinate chart at .

In this paper, we use least square method to approximate the analytic functions under the local coordinate system constructed above. In order to guarantee the curvature operator on each patch being satisfied the spherical conditions as presented in section 2, we choose second-order elliptic polynomial functions to approximate the structures of local patches. The polynomial form of is depicted as , where is the coefficient vector to be determined by least square method and is the second-order polynomial basis vector.

Under the local smooth representation of , the corresponding tangent vector basis at is given by , where . Then the local Riemannian metric tensor is shown as . The second fundamental form coefficient is shown as: .

So the local Ricci flow equation defined on is represented as follows:

(10)

In order to solve the Ricci flow equation, we need to discretize the differential operators on point clouds. Suppose , Eq.10 can be represented as the function of , that is , where So the local Riemannian metric tensor as well as the corresponding Ricci curvature are iterated under the Ricci flow in the local neighborhood as follows:

(11)
(12)
(13)

Optimizing these update equations until , where is a non-negative constant and is the number of total iterations.

In step 5) above, after the Ricci flow converges at each , the overlapping local patches are flowing to a set of discrete local spherical patches . We denote the global coordinates of as . The relationship between and is linked by a set of global alignment maps, which shift the discrete local spherical patches to a global subset of a sphere. We want the global coordinates satisfying the following equations, such that is obtained by the local affine transformation of , where . We have:

(14)

where is the number of sample points, the nearest neighbor size parameter, the unit of orthogonal transformation and the reconstruction error.

In order to obtain the optimized local affine transformation, we need to minimize the local reconstruction error matrix :

(15)

where .
Minimizing the above least square error is equal to solve the following eigenvalue problem:

(16)

where , , , and the generalized inverse matrix of .

Decomposing matrix using SVD method, we obtain , where the columns of are the unit orthogonal eigenvectors of . is a diagonal matrix and the diagonal elements are the eigenvalues of , which are arranged in ascending order. The optimal solution is given by the eigenvectors of the matrix , corresponding to the to smallest eigenvalues of . However, the optimal data set that we finally need are distributed on a sphere with curvature . So we need to give a set of constraints:

(17)

Under these constraints, we rewrite matrix as: , where is a diagonal matrix, , the rest . The obtained to columns of are the optimal data set , which are distributed on a sphere with curvature .

  Input: Training data points , neighbor size parameter .
  Output: Low dimensional representations 1. for to do 2. Find -nearest neighbors of ; 3. Compute tangent space . 4. end for 5. Repeat 6. Update the Ricci flow equations using Eq.11, Eq.12, Eq.13. 7. Until convergence 8. Align the deformed to a complete subset of sphere. 9. Use traditional manifold learning algorithms to reduce the dimension of spherical data points.
Algorithm 1 Applying Ricci flow to Manifold Learning (RF-ML)

3.2 Theoretical Analysis

In this subsection, we give the theoretical anlysis of our algorithm. First we want to illustrate the convergence of Ricci flow, which has been proved by reaserchers. Second we give the relationship between the original manifold and the deformed space under Ricci flow.

Theorem 3.1. 7 () Assume that has weakly 1/4-pinched sectional curvatures in the sense that for all two-planes . Moreover, we assume that is not locally symmetric. Then the normalized Ricci flow with initial metric exists for all time, and converges to a constant curvature metric as .

This theorem has been proved by S. Brendle and R. Schoen 7 () in 2008. In global, the Ricci flow exists for all time, and converges to a constant curvature metric. For each local patch, the Ricci flow definitely exists and also converges to a postive constant curvature .

Theorem 3.2. The global set of patches P (step 5) is distributed on a subspace of a sphere, where is a Riemannian manifold. In addition, is diffeomorphic to the original Riemannian manifold .

Proof:
The original data points are distributed on . At each data point there is a local neighborhood , such that can fully cover the manifold . Thus has an enumerable set of basis.

According to the local Ricci flow, at each data point there is a diffeomorphism between and . In the global alignment there is a diffeomorphism between and . Thus the map between and is also a local diffeomorphism. Let:

(18)
(19)

Define:

(20)

Let . It is defined on the global manifold and induces a local diffeomorphism in any local patch. Since has an enumerable set of basis, according to the unit decomposition theorem of manifold, is a global diffeomorphism. Therefore is also a Riemannian manifold. It is obvious that the data set is distributed on the manifold which is just the manifold mentioned in the statement of this theorem.

4 Experiments

In this section we compare our algorithm RF-ML with other manifold learning algorithms on several synthetic datasets as well as four real world databases.

4.1 Intrinsic Dimension

One implicit fundamental assumption of traditional manifold learning is that the dimension of data sets at each point is all the same. But in practice the distribution of data points on a manifold may be not uniform, even the manifold’s dimensions at different points may be not exactly the same. An intuitive example illustrating this situation is depicted in table 1. We list the numbers of neighborhoods with different dimensions in six groups of datasets: ORL Face 33 (), Yale Face 35 (), Yale-B Face 35 (), Weizmann 34 (), Swiss Roll 32 () and Sphere 32 (). We choose the neighborhood-size parameter and use PCA projection to obtain the 95% principal components. The latter is approximately viewed as the dimension of the local neighborhoods and may not be the same for all the neighborhoods. Table 1 shows the number of neighborhoods in each dimension.

4.2 Dimensionality Reduction

To evaluate the performance of our curvature-aware manifold learning algorithm RF-ML, we compare our method with several traditional manifold learning algorithms (PCA, Isomap, LLE, LEP, Diffu-Map, LTSA) on four sets of three dimensional data, including: Swiss Roll, Sphere, Ellipsoid and Gaussian. Swiss Roll is a locally flat surface, where the Gauss curvature (i.e. Ricci curvature of two dimensional manifolds) is zero everywhere. However, the Gauss curvatures of the other three datasets are not zero. The objective of this comparison is to map each data set to two dimensional Euclidean space and then to analyze the neighborhood preserving ratios (NPRs) 22 () of different algorithms. Table 2 shows the results of these comparisons with seven different algorithms on these four sets of data, where the K-nearest neighbor parameter is . The neighborhood preserving ratio (NPR) 22 () is defined as follows:

(21)

where is the set of -nearest sample subscripts of and is the set of -nearest sample subscripts of . represents the number of intersection points.

Databases ORL Face Yale Face YaleB Face Weizmann Swiss Roll Sphere
Org-dim 1024* 1024* 1024* 200* 3* 3*
1 0 0 0 0 0 0
2 0 0 0 90 1000 984
3 0 0 38 317 16
4 0 0 238 366
5 0 0 362 315
6 6 0 415 251
7 214 9 772 349
8 182 156 583 387
9 6
Total 400* 165* 2414* 2075* 1000* 1000*
10 10 10 10 10 10
Ratio 0.95 0.95 0.95 0.95 0.95 0.95
Table 1: Numbers of neighborhoods in each dimension. The first row shows the six datasets. The second row shows the original dimension of each dataset. The rows from to respectively list the numbers of neighborhoods in dimension . The ’Total’ row shows the total number of data in each dataset. ’’ denotes the neighborhood-size parameter, and ’Ratio’ denotes the percentage of principal components. Among all the data in table 1, those labeled by ’’ are from the original dataset and the other data comes from our experiment results.
Methods PCA Isomap LLE LEP Diffu-Map LTSA RF-ML
Swiss Roll 0.5137 0.8594 0.6187 0.3981 0.4403 0.6121 0.8594
Ellipsoid 0.4399 0.6914 0.6205 0.7506 0.5965 0.4390 0.8702
Sphere 0.4815 0.6467 0.5213 0.7720 0.5010 0.5465 0.8684
Gaussian 0.9969 0.9261 0.9359 6406 0.9848 0.9970 0.9909
Table 2: Neighborhood Preserving Ratio. In this experiment, we fix the neighborhood-size parameter .
Figure 2: The neighborhood preserving rate of two-dimensional Ellipsoid embedded in 3-dimensional Euclidean space. Compute the NPRs under different neighborhood size parameter values with five manifold learning algorithms.

NPR measures the local structure preserving rate of the dimension reduction algorithms. Table 2 shows that for all but one dataset, the NPR of RF-ML is the best one among all algorithms. The Swiss Roll is a locally flat two-dimensional manifold. Its data structure is unchanged under Ricci flow. Therefore no Ricci flow is needed. As for the Sphere dataset, its Gauss curvature is a unique constant everywhere. There is no need of Ricci flow, too. Regarding Ellipsoid dataset and Gaussian dataset, note that the Gauss curvatures at different points are not always the same. When using RF-ML to reduce their dimensions, the NPR of Ellipsoid dataset is 87.95%, 34.28%, 66.58%, 12.49%, 73.33%, 58.90% respectively better over the other six traditional manifold learning algorithms. For Gaussian database, due to the characteristics of the dataset distribution the NPRs of PCA and LTSA are quite high. Compared with the six algorithms, the NPR of our method is not bad and also quite high. This clearly demonstrates that our curvature-aware algorithm RF-ML is more stable and the Ricci flow process preserves better the local structure of data points.

Here we see another advantage of our algorithm. RF-ML is relatively stable under different values of neighbor-size parameter . The ’stable’ in this section means that the change of NPRs under different values of is relatively gentle. While traditional manifold learning algorithms are more sensitive to the neighbor-size parameter . That is because those traditional algorithms implicitly assume the neighborhoods at each point be flat. Our method tries to find the intrinsic curvature structure of local patches at every point, so it is not sensitive to the neighbor-size parameter . In this experiment, we evaluate the performance of our RF-ML algorithm compared with the other four algorithms (LLE, LEP, Isomap, LTSA) under different values of neighbor-size parameter on the Ellipsoid dataset. The comparison results are showed in Figure 2. As the neighborhood-size parameter values are increasing, the NPR of our algorithm RF-ML grows continuously. But the NPRs of the other four manifold learning algorithms are unstable and especially sensitive to the different values of neighbor-size parameter .

4.3 Real World Databases

In this subsection, we give our experiments on four real world databases: ORL database 33 (), Yale Face database 35 (), Extended Yale Face B database 35 (), and USPS database 38 ().

The ORL Face database contains different images of each of distinct subjects. So this database totally contains images. For some subjects, the images were taken at different times, varying the lighting, facial expressions and facial details.

The Yale Face database contains grayscale images of individuals. There are images per subject, one per different facial expression or configuration: center-light, glasses, happy, left-light, no glasses, normal, right-light, sad, sleepy, surprised, and wink.

The Extended Yale Face database B contains single images of subjects. There are images per subject. For every subject in a particular pose, an image with ambient illumination was also captured.

The USPS database refers to numeric data obtained from the scanning of handwritten digits from envelopes by the U.S. Postal Service. There are classes of these handwritten digits from to . The total number of this database is . The images here have been size normalized, resulting in grayscale images. In our experiment, we respectively choose images for each digital class.

(a) ORL database
(b) Yale database
(c) YaleB database
(d) USPS database
Figure 3: The curvature distributions of ORL face database (a), Yale face database (b), YaleB face database (c), and USPS database (d).

Our algorithm aims to regularize the curvature distribution of each database using Ricci flow. So we first analyze the curvature distributions of these four databases. The intuitive results are shown in Figure 3. From these four subfigures we can see that the curvature distribution of USPS database is more variable than the other three databases. One reason is that USPS database has enough samples to guarantee the samples distribution dense enough. The other reason is that the data points among different classes vary largely. For the other three databases, the curvature distributions are ranged from to . The total numbers of ORL database and Yale Face database are respectively and . Their distributions in high dimensional Euclidean space are especially sparse. So the curvature on each data point is relatively small.

Second, we compare our algorithms with traditional manifold learning algorithms on these four databases to test their classification accuracies. In this experiment, we first respectively use traditional manifold learning algorithms and our RF-ML algorithm to reduce the dimensions of these four databases. Then we use Nearest Neighbor Classifier method to test the recognition accuracies of different algorithms on these four databases. In the classification step, for these four databases, we respectively choose half images of each distinct subject as the training subset and the rest images of each subject as the test subset. The experimental results are shown in Table 3. From Table 3 we can see that our proposed method RF-ML mostly outperforms the other four traditional manifold learning algorithms. Our method analyzes the curvature information of these four databases and uses Ricci flow to regularize the curvature information. So compared with traditional manifold learning algorithms, we uncover the intrinsic curvature information of these databases and add the information into the dimension reduction process.

Finally, we further compare our algorithm RF-ML with traditional manifold learning algorithms on USPS database to test the classification accuracies under different low dimension . Comparing with the other three real world databases, the curvature distribution of USPS database varies higher than the other databases. So we do sufficiently comparative experiments on USPS database to test its performance under different low dimensions . The final experimental results are shown in Figure 4. From Figure 4 we can see that the classification accuracies of our algorithm mostly outperform other traditional manifold learning algorithms under the different low dimension . Overall, using Ricci flow to regularize the curvature distribution of dataset is a significant improvement of manifold learning.

Methods PCA LPP LLE LEP RF-ML
ORL DB
Yale DB
YaleB DB
USPS DB
Table 3: Neighborhood Preserving Ratio. In this experiment, we fix the neighborhood-size parameter . In the dimension reduction step, we respectively choose as the intrinsic dimensions of ORL DB, Yale DB, YaleB DB, and USPS DB. In the recognition step, we respectively choose half of each dataset as the training dataset, the rest as the testing dataset.
Figure 4: The classification accuracies of different manifold learning algorithms under different low dimensions .

5 Conclusions and Future Work

In the field of image recognition, to precisely describe the continuously changing images, one critical step is to assume the image set distributed on a lower dimensional manifold, which is embedded in the high dimension pixel space. Then one uses mathematical knowledge of manifold to deal with these datasets, such as dimensionality reduction, classification, clustering, recognition and so on. Manifold learning is an effective way to link the classical geometry with machine learning. Whether the manifold structure is uncovered exactly or not impacts the learning results directly. Many traditional manifold learning algorithms do not differentiate the varying curvature at different points of the manifold. Our method aims to excavate the power of Ricci flow and to use it to dynamically deform the local curvature to make the manifold’s curvature uniform. The extensive experiments have shown that our method is more stable compared with other traditional manifold learning algorithms. There have been already papers published in the literature regarding applying Ricci flow to manifold learning as we have mentioned above. But to the best of our knowledge this paper is the first try to apply Ricci flow to high dimensional (dimension unlimited) data points for dimensionality reduction. One limitation of our algorithm is that RF-ML works only for manifolds with non-negative curvature. We will discuss the applicability of RF-ML algorithm to manifolds with negative Ricci curvature manifold in our next paper. Generally we believe that there will be lots of Ricci flow applications in manifold learning.

Acknowledgements

This work is supported by National Key Research and Development Program of China under grant 2016YFB1000902; National Natural Science Foundation of China project No.61232015, No.61472412, No.61621003; Beijing Science and Technology Project: Machine Learning based Stomatology; Tsinghua-Tencent- AMSS –Joint Project: WWW Knowledge Structure and its Application.

References

  • (1) Guoqiang Wang, Nianfeng Shi and Yunxing Shu, Embedded manifold-based kernel fisher discriminant analysis for face recognition, Neural Processing Letters, vol.43, pp.1-16 (2016).
  • (2) Wei Zeng, S. Drimitris and Xianfeng David Gu, Ricci flow for 3D shape analysis, IEEE TPAMI, vol.32, No.4 (2010).
  • (3) Wei Zeng, X. Yin, Y. Zeng, Y. Lai and X. Gu, 3D face matching and registration based on hyperbolic Ricci flow, IEEE CVPRW, pp.1-8 (2008).
  • (4) Kwang In Kim, J. Tompkin and C. Theobalt, Curvature-aware regularization on Riemannian submanifolds, ICCV (2013).
  • (5) W. Xu, E.R.Hancock and R.C.Wilson, Ricci flow embedding for rectifying non-Euclidean dissimilarity data, Pattern Recognition, vol.47, pp.3709-3725 (2014).
  • (6) W. Xu, E.R.Hancock and R.C.Wilson, Regularising the Ricci flow embedding, SSPR, pp. 579- 588 (2010).
  • (7) W.Xu, E.R.Hancock and R.C.Wilson, Rectifying non-Euclidean similarity data using Ricci flow embedding (2010).
  • (8) S. Brendle and R. Schoen, Classification of manifolds with weakly 1/4-pinched curvatures, arXiv:0705.3963v3.
  • (9) S. Roweis and L. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science, vol.290(5500), pp.2323-2326 (2000).
  • (10) J. Tenenbaum, V. de Silva, and J.Langford, A global geometric framework for nonlinear dimension reduction, Science, vol.290(5500), pp.2319-2323 (2000).
  • (11) M. Belkin and P. Niyogi, Laplacian eigenmaps and spectral technique for embedding and clustering, Neural Information Processing Systems 14, pp. 585-591 (2001).
  • (12) D.L.Donoho and C.E. Grimes, Hessian eigenmaps: Locally linear embedding techniques for hig- h-dimensional data, Proceedings of the National Academy of Sciences of the United States of America, vol.100(10), pp.5591-5596 (2003).
  • (13) Z, Zhang and H, Zha, Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment, SIAM J. Scientific Computing, vol. 26, no. 1, pp.313-338 (2005).
  • (14) Xiaofei He and Partha Niyogi, Locality Preserving Projections, Int. Conf. Advances in Neural Information Processing Systems (2003).
  • (15) I.T. Jolliffe, Principal Component Analysis, Springer-Verlag, New York (1989).
  • (16) G. Sanguinetti, Dimensionality reduction of clustered data sets, IEEE TPAMI, vol.30(3), pp.535-540 (2008).
  • (17) Kewei Tang, Jie Zhang and Zhixun Su, Bayesian Low-Rank and Sparse Nonlinear Representation for Manifold Clustering, Neural Processing Letters, vol. 44, pp. 719-733 (2016).
  • (18) Peter Petersen, Riemannian Geometry, Springer, New York (1998).
  • (19) J. M. Lee, Riemannian Manifolds, Springer, New York (1997).
  • (20) H. D. Cao and B. Chow, Recent developments on the Ricci flow, Bull. Amer. Math. Soc. 36, pp. 59-74 (1999).
  • (21) S. Brendle and R. Schoen, Riemannian manifolds of positive curvature, Proceedings of the International Congress of Mathematicians, Hyderabad, India (2000).
  • (22) T. Cox and M. Cox, Multidimensional Scaling, London (1990).
  • (23) H.D. Cao and X.P. Zhu, A complete proof of the Poincare and geometrization conjectures-application of the Hamilton-Perelman theory of the Ricci flow, Asian J.Math, Vol. 10, No. 2 (2006).
  • (24) T. Wittman, Mani Matlab demo, http://www.math.umn.edu/ wittman/mani/ (2005).
  • (25) http://www.cs.toronto.edu/ roweis/data/olivettifaces.mat.
  • (26) L. Gorelick, M. Blank, E. Shechtman, M. Irani, and R. Basri, Actions as space-time shapes, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no.c12, pp.2247-2253 (2007).
  • (27) http://www.face-rec.org/databases/.
  • (28) http://www-stat.stanford.edu/ tibs/ElemStatLearn/data.html.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
29030
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description