A Riemanian Approach to Blob Detection in Manifold-Valued Images

A Riemanian Approach to Blob Detection in Manifold-Valued Images

Aleksei Shestov Lomonosov Moscow State University, Faculty of Mechanics and Mathematics, Russia, 119991, Moscow, GSP-1, 1 Leninskiye Gory, Main Building
{shestov.msu,mkumskov}@gmail.com
   Mikhail Kumskov Lomonosov Moscow State University, Faculty of Mechanics and Mathematics, Russia, 119991, Moscow, GSP-1, 1 Leninskiye Gory, Main Building
{shestov.msu,mkumskov}@gmail.com
Abstract

This paper is devoted to the problem of blob detection in manifold-valued images. Our solution is based on new definitions of blob response functions. We define the blob response functions by means of curvatures of an image graph, considered as a submanifold. We call the proposed framework Riemannian blob detection. We prove that our approach can be viewed as a generalization of the grayscale blob detection technique. An expression of the Riemannian blob response functions through the image Hessian is derived. We provide experiments for the case of vector-valued images on 2D surfaces: the proposed framework is tested on the task of chemical compounds classification.

Keywords:
blob detection, image processing, manifold-valued images, vector-valued images, differential geometry

1 Introduction

Blob detection [1] is a widely used method of keypoints detection in grayscale images. Informally speaking, blob detection aims to find ellipse-like regions of different sizes with “similar” intensity inside. Blobs are sought as local extremums of a blob response function. Several color blob detection algorithms were proposed in [2, 3]. Blob detection has applications in 3D face recognition, object recognition, panorama stitching, 3D scene modeling, tracking, action recognition, medical images processing, etc.
Our goal is to propose a blob detection framework for the general setting of an image being a map between Riemannian manifolds. Our approach is based on a definition of blob response functions by means of image graph curvatures. Furthermore, we derive the expression of Riemannian blob response functions through image Hessian. This expression shows that Riemannian blob detection coincides with the classical blob detection framework for the grayscale case. Also this expression provides a more convenient way to calculate Riemannian blob response functions for vector- and manifold-valued images.
Research of connections between image processing methods and image graph geometry is of its own interest. This research helps deeply understand traditional methods, provides insights and gives natural generalizations of classical methods to vector-valued and manifold-valued images [4, 5, 6]. Connections between the blob response functions and image graph curvatures were mentioned in papers [7, 8]. Our work is the first to accurately analyze this question in the general setting.

1.0.1 Contributions:

  1. We are the first to provide a blob detection framework for the general setting of an image being a map between manifolds. This framework can be viewed as a generalization of grayscale blob detection. Our framework provides blob response functions for the previously uncovered problems: blob detection in color images on manifold domain and blob detection in manifold-valued images (both on Euclidian and manifold domains).

  2. We are the first to analyze connections between the blob response functions and curvatures of image graph both for Euclidian and manifold domains.

  3. The experiments on the task of chemical compounds classification show the effectiveness of our approach for the case of vector-valued images on 2d surfaces.

2 The Problem Introduction

Blob detection was firstly proposed for grayscale images on 2D Euclidian domain [1]. In [9] blob detection was generalized to 2D surfaces. Several approaches to generalization of blob detection to color case were proposed in [2, 3]. However, these approaches are based on global or local conversion of a color image to the grayscale, so they can’t be used for manifold-valued images.
Consider a grayscale image on a smooth 2-dimensional manifold . The blob detection framework by [9] is as follows:

  1. Calculate the scale-space . is the solution of the heat equation on the surface , where is the Laplace-Beltrami operator;

  2. Choose a blob response function and calculate it:

    (1)
    (2)

    where is the Hessian of as a function of with fixed ;

  3. Find blobs centers and scales as , where or . Find the blobs radii as .

For the general case of a map between manifolds the Hessian is the covariant differential of the differential: . Consider the straightforward generalization of the blob detection stages:

  1. Scale-space calculation. is calculated as the solution of . Methods of manifold-valued PDEs solution for different cases are discussed in the papers [10, 5, DTI] and others. These methods are out of scope of our work.

  2. Blob response calculation. The determinant blob response is not defined.

  3. Blobs centers calculation. We can’t find maximums or minimums of the trace blob response because it is not scalar-valued: .

We see that there is no straightforward generalization of the blob response functions to the manifold-valued case. How can the problem of blob response generalization be solved? Our key ideas are the following:

  1. Consider the image graph as a submanifold embedded in . The grayscale and manifold-valued cases differ only by a co-dimension of the embedding. Then a formulation of the blob response through notions defined for all co-dimensions will give an immediate generalization to the manifold-valued case.

  2. What notions to use? The scalar and the mean curvatures are defined for all co-dimensions and are close to the determinant and the trace of the image Hessian respectively if tangent planes to and to are ”close”.

3 The Proposed Method

3.1 Used Notations

All functions and manifolds here and further are considered to be smooth. Consider - and -dimensional manifolds and . Denote the -dimensional manifold as . Consider the isometric embeddings . Further we identify (resp. ) and related notions with (resp. ). The letters (resp. ) are used as indices for notions related to (resp. ). The set (resp. ) is an orthonormal basis of (resp. ).
For a map its graph is an -dimensional manifold embedded in . Denote the Hessian of as . Let , , be the manifold with the metric . For a map denote .
We analyze a manifold-valued image . Denote the solution of the heat equation , where is the Laplace-Beltrami operator.
For a manifold and its submanifold denote the mean curvature of as , its scalar curvature as , an exponential map from to as .
Subscripts and superscripts are omitted when they are clear from a context. The definitions of used differential geometric notions can be found in textbooks [11].

3.2 Main Definitions and Theorems

Definition 1

The scalar blob response is defined as:

the mean blob response is defined as:

The next theorem connects and with the scale-space Hessian. The obtained expression provides a more convenient way for calculation of the Riemannian blob response functions.

Theorem 3.1

Let , . Then

The next corollary from Theorem 3.1 states that for the grayscale case Riemannian blob detection coincides with usual blob detection. This corollary allows to consider our method as a generalization of grayscale blob detection.

Corollary 1

Let . Then the scalar blob response is equal to the determinant blob response (1):

the mean blob response is equal to the trace blob response (2):

3.3 Proof of the Theorem 3.1

3.3.1 Additional notations.

Consider maps , , . is a basis (not orthonormal) of , is a basis of . Then is a basis of .
For a manifold denote its metric as or , the Levi-Civita connection as , a connection on a vector bundle over as .
Denote as (resp. ) an orthogonal (resp. along a subspace ) projection on a subspace .
Some minor formal details of the proofs are omitted due to the space constraints.

Proposition 1

Let , . Then
. If is injective then .

Proof

Consider the Hessian as , then
. We apply the Leibniz rule to this expression and obtain the first statement. Recall that if is injective then is an isomorphism between and . This gives the second statement. ∎

Lemma 1

Let . Let be the connection on induced by the isomorphism . Let be the second fundamental form of the submanifold of with respect to the connection . Then .

Proof

As is injective, by Prop. 1: =

Proposition 2

Let , , . Then
.

Proof

By Proposition 1: .
Recall that . Then

Proposition 3

Let be the second fundamental form of the submanifold of with respect to the connection and be the second fundamental form with respect to the connection induced by . Let . Then .

Proof

By properties of a second fundamental form of a normalized manifold: and . Then by simple operations with vectors we obtain the lemma proposition.

Lemma 2

.

Proof

. Then

Proposition 4

Consider as a basis of . is the matrix of in the basis and is the unit matrix.
Then: the induced metric on has the matrix ; the induced metric on has the matrix ; the covariant induced metric on has the matrix ;

Lemma 3

Proof

Write the Gauss equation, the scalar curvature definition and apply
Lemma 2: Then substitute for as tends to 0, apply Prop. 4 and obtain the needed equality. ∎

Lemma 4

.

Proof

For :

Theorem 3.1 follows from Lemmas 3 and 4. The formulation of Theorem 3.1 is obtained by substitution of with .

4 The Experiments

4.0.1 Experimental Setup.

We apply our blob detection framework to a chemical compounds classification problem, called also the QSAR problem [12]. The task is to distinguish active and non-active compounds using their structure. Each compound is represented by a triangulated molecular surface [13] and several physico-chemical and geometrical properties on the surface. So an input data element can be modeled as a 2-dimensional manifold with a vector-valued function . We use the following properties: the electrostatic and the steric potentials, the Gaussian and the mean curvatures. These properties are calculated in each triangulation vertex.

4.0.2 Implementation.

We use Riemannian blob detection for the construction of descriptor vectors. The procedure is the following:

  1. Detect blobs by our method in each compound surface;

  2. Form pairs of blobs on each surface;

  3. Transform the blobs pairs into vectors of fixed length by using the bag of words approach [14].

The Riemannian blob response functions are calculated for each triangulation vertex . The procedure is the following:

  1. Find the directional derivatives by the finite differences approximation, where ̅ are the directions from to its neighbour vertices.

  2. Find the differential by solving the overdetermined linear system , is a matrix which columns are vectors .

  3. Find the covariant derivatives of the differential in the neighbour directions, i.e. find for each as by ). are found by the finite differences approximation.

  4. Find the covariant differential by solving the overdetermined linear system , is a matrix which columns are vectors .

is obtained. Calculate ,
.

4.0.3 The Results.

An example of the algorithm result is presented in Fig. 2.
We compare the prediction models built on the base of the following blob detection methods:

  1. Riemannian blob detection with as a blob response function;

  2. A naive method of applying blob detection to each channel separately;

  3. Riemannian blob detection with as a blob response function. It coincides with the method [2], adapted to the case of 2D surface;

  4. The method of adaptive neighbourhood projection [3]. It is adapted by us to the case of 2D surface.

The feature reduction SVM [16] is used for construction of the prediction model. The cross-validation functional [15] is used as an index of the performance quality. The test data is the following: 3 datasets (bzr, er_lit, cox2) from [17], 3 datasets (glik, pirim, sesq) from Russian Oncology Science Center. The results are presented in Table 2.

  naive Adapt. [3] glik 1.0 0.954 0.975 1.0 pirim 0.99 0.96 0.97 0.98 sesq 1.0 0.98 0.976 1.0 bzr 0.992 0.971 0.975 0.983 er_lit 0.98 0.961 0.956 0.98 cox2 0.991 0.967 0.985 0.986   naive Adapt. [3] glik 1.0 0.954 0.975 1.0 pirim 0.99 0.96 0.97 0.98 sesq 1.0 0.98 0.976 1.0 bzr 0.992 0.971 0.975 0.983 er_lit 0.98 0.961 0.956 0.98 cox2 0.991 0.967 0.985 0.986

Figure 2: A molecular surface with on it and found centers (denoted by white color) of blobs of radii 3.
Figure 1: A molecular surface with on it and found centers (denoted by white color) of blobs of radii 3.
Table 3: The results: the cross-validation of the models, based on feature vectors built by the blob detection methods.
Figure 1: A molecular surface with on it and found centers (denoted by white color) of blobs of radii 3.
Table 2: The results: the cross-validation of the models, based on feature vectors built by the blob detection methods.

Riemannian blob detection with as a blob response function is the best performing method. This shows the effectiveness of our approach. This particular method for vector-valued functions on 2D surfaces wasn’t presented in the literature before.

5 Conclusion and Future work

We propose the Riemannian framework for blob detection in manifold-valued images. This framework is based on the definition of the blob response functions by means of the image graph curvatures. Our approach gives new methods for the uncovered problems and coincides with classical blob detection for the grayscale case. The experiments results show the effectiveness of the proposed approach.
The next direction for the research is a generalization of our framework to the case of sections of non-trivial fiber bundles. In particular, such generalization will cover an important case of tangent vector fields.

Acknowledgments

The authors want to thank Dr. Alexey Malistov for valuable discussions and the help with the article editing.

References

  • [1] Lindeberg, T.: Feature detection with automatic scale selection. International journal of computer vision, 30(2), 79–116 (1998)
  • [2] Khanina, N. A., Semeikina, E. V., Yurin, D. V.: Scale-space color blob and ridge detection. Pattern Recognition and Image Analysis, 22(1), 221–227 (2012)
  • [3] Smirnov, P., Semenov, P., Lyakh, M., Chun, A., Gusev, D., Redkin, A., Srinivasan, S.: GRoM—Generalized robust multichannel feature detector. In Signal and Image Processing Applications, 2011 IEEE International Conference on, 585–590 (2011, November)
  • [4] Saucan, E., Wolansky, G., Appleboim, E., Zeevi, Y. Y.: Combinatorial ricci curvature and laplacians for image processing. In Image and Signal Processing, CISP’09, 2nd International Congress on, 1–6 (2009)
  • [5] Sochen, N., Kimmel, R., Malladi, R.: A general framework for low level vision. IEEE transactions on image processing, 7(3), 310–318 (1998)
  • [6] Batard, T., Berthier, M.: Spinor Fourier transform for image processing. IEEE Journal of Selected Topics in Signal Processing, 7(4), 605–613 (2013)
  • [7] Ferraz, L., Binefa, X.: A sparse curvature-based detector of affine invariant blobs. Computer Vision and Image Understanding, 116(4), 524–537 (2012)
  • [8] Ferraz, L., Binefa, X.: A scale invariant interest point detector for discriminative blob detection. In Iberian Conference on Pattern Recognition and Image Analysis, 233–240 (2009, June)
  • [9] Zaharescu, A., Boyer, E., Varanasi, K., Horaud, R.: Surface feature detection and description with applications to mesh matching. In Computer Vision and Pattern Recognition, 2009, 373–380 (2009, June)
  • [10] Mémoli, F., Sapiro, G., Osher, S.: Solving variational problems and partial differential equations mapping into general target manifolds. Journal of Computational Physics, 195(1), 263–292 (2004)
  • [11] Spivak, M.: Comprehensive introduction to differential geometry. Vol. IV A (1981)
  • [12] Baskin, I., Varnek, I.: Fragment Descriptors in SAR/QSAR/QSPR Studies, Molecular Similarity Analysis and in Virtual Screening. Chemoinformatic Approaches to Virtual Screening, 1–43 (2008)
  • [13] Connolly, M. L.: Analytical molecular surface calculation. Journal of Applied Crystallography, 16(5), 548–558 (1983)
  • [14] Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV, Vol. 1, No. 1-22, 1–2 (2004, May).
  • [15] Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, Vol. 14, No. 2, 1137–1145 (1995, August)
  • [16] Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., Vapnik, V.: Feature selection for SVMs. In Proceedings of the 13th International Conference on Neural Information Processing Systems, 647–653 (2000, January)
  • [17] J. J. Sutherland, L. A. O’Brien, and D. F. Weaver.: Spline-fitting with a genetic algorithm : a method for developing classification structure-activity relationships. J. Chem. Inf. Comput. Sci., 43:1906–1915 (2003)
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
370536
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description