A Novel Active Contour Model for Texture Segmentation
Texture is intuitively defined as a repeated arrangement of a basic pattern or object in an image. There is no mathematical definition of a texture though. The human visual system is able to identify and segment different textures in a given image. Automating this task for a computer is far from trivial.
There are three major components of any texture segmentation algorithm: (a) The features used to represent a texture, (b) the metric induced on this representation space and (c) the clustering algorithm that runs over these features in order to segment a given image into different textures.
In this paper, we propose an active contour based novel unsupervised algorithm for texture segmentation. We use intensity covariance matrices of regions as the defining feature of textures and find regions that have the most inter-region dissimilar covariance matrices using active contours. Since covariance matrices are symmetric positive definite, we use geodesic distance defined on the manifold of symmetric positive definite matrices as a measure of dissimlarity between such matrices. We demonstrate performance of our algorithm on both artificial and real texture images.
Texture is intuitively defined as a repeated arrangement of a basic pattern or object in an image. There is no universal mathematical definition of a texture though. The human visual system is able to identify and segment different textures in a given image without much effort. Automating this task for a computer, though, is far from trivial.
Apart from being a tough academic problem, texture segmentation has several applications. Texture segmentation has been applied to detect landscape changes from aerial photographs in remote sensing and GIS , content based image retrieval  and diagnosing ultrasound images  and others.
There are three major components in any texture segmentation algorithm: (a) The model or features that define or characterize a texture, (b) the metric defined on this representation space, and (c) the clustering algorithm that runs over these features in order to segment a given image into different textures.
There are two approaches of modeling a texture: Structural and Statistical. The structural approach describes a texture as a specific spatial arrangement of a primitive element. Vor-onoi polynomials are used to specify the spatial arrangement of these primitive elements [27, 26]. The statistical approach describes a texture using features that encode the regularity in arrangement of gray-levels in an image. Examples of features used are responses to Gabor filters , graylevel co-occurrence matrices [33, 14], Wavelet coefficients , human visual perception based Tamura features , Laws energy measures , Local binary patterns and Covariance matrices of features [28, 10]. In , the authors compare performance of some of the above mentioned features for the specific goal of image retrieval. In fact, Zhu, Wu & Mumford  propose a mechanism of choosing an optimal set of features for texture modeling from a given general filter bank. Markov random fields, Fractal dimensions and the space of oscillating functions  have also been used to model textures.
Various metrics have been used to quantify dissimilarity of features: Euclidean, Chi-squared, Kullback-Leibler & its symmetrized version , manifold distance on the Gabor feature space  and others. NN, Bayesian inference, means, alongwith active contours algorithms are some of the methods used for clustering/segmentating texture areas in the image with similar features.
In this paper, we use intensity convariance matrices over a region as the texture feature. Since these are symmetric positive definite matrices which form a manifold, denoted by , it is natural to use the intrinsic manifold distance as a measure of feature dissimilarity. Using a novel active contours method, we propose to find the background/foreground texture regions in a given image by maximizing the geodesic distance between the interior and exterior covariance matrices. This is the main contribution of our paper.
In the next subsection we list out some existing texture segmentation approaches using active contours model.
1.1 Related work
Sagiv, Sochen & Zeevi  generalize both, geodesic active contours and Chan & Vese active contours, to work on a Gabor feature space. The Gabor feature space is a parametric D manifold embedded in whose natural metric is used to define an edge detector function for geodesic active contours, and to define the intra-region variance in case of the Chan & Vese active contours. In , the authors use Chan & Vese active contours on Local Binary Pattern features for texture segmentation.
In , the authors propose a Chand & Vese active contour model on probability distribution of the structure tensor of the image as a feature. The closest approach to our algorithm is by Houhou et. al., where the authors find a contour that maximizes the KL-divergence based metric on probability distribution of a feature for points lying inside the contour and outside the contour. The feature used is based on principal curvatures of the intensity image considered as a D manifold embedded in . In particular, the cost function for a curve is defined as
where is the probability distribution of the feature inside and outside the closed contour respectively. Gaussian distribution is assumed as the model for the probability distribution of the feature both inside as well as outside the contour.
In our approach, instead of using some scalar feature to represent texture, we iteratively compute a contour that maximizes the geodesic distance between the interior and exterior intensity covariance matrix of the contour. It can be seen that the maximization process has to be carried out over the manifold of symmetric positive definite matrices, making it fundamentally different from the approach in . Moreover, we can easily extend this approach to covariance matrices of any other texture feature one may want to use.
The paper is organized as follows:In next section we provide a brief review of active contour models for image segmentation. In section refersection label, we describe our active contour model based on geodesic distance between the interior and exterior covariance matrices of a contour. We give our experimental results in Section 3 followed by conclusions and future scope.
1.2 Active contours and Level sets
In classical active contours , user initializes a curve on an intensity image which evolves and stabilizes on the object boundary. The curve evolution is gradient descent of an energy functional, , given by
where and are real positive constants, and are first and second derivatives of and is the image gradient. First two terms are regularizers, while the third term pushes the curve towards the object boundary.
Geodesic active contours  is an active contour model where the objective function can be interpreted as the length of a curve in a Riemannian space with metric induced by image intensity. The energy functional for geodesic active contour is given by
where is a monotonically decreasing edge detector function. One such choice is . The curve evolution equation that minimizes this energy is given by
where is the inward unit normal and is the curvature of the curve .
A convenient computational procedure for curve evolution is the level set formulation . Here the curve is embedded in the zero set of a function and the function is made to evolve so that its zero level set evolves according to the desired curve evolution equation. For a curve evolution equation of the form , the corresponding level set evolution is . See Appendix in . In particular the level set evolution for geodesic active contours is given by
Another active contours approach was introduced by Chan and Vese  where the energy function was based on regional similarity properties of an object, rather then its edges (image gradient). Suppose that is the initial curve defined on the domain of the intensity image . can be divided into two parts, interior (denoted by ) and exterior (denoted by ). Let us represent the mean gray value of the region and by and respectively, then the energy function for which the object boundary is a minima is given by
After adding some regularizing terms the energy functional , is given by
where are fixed scalar parameters. The level set evolution equation is given by
where is a smooth approximation of the Dirac delta function. A nice survey on active contours and level set implementation can be found in . We now describe our active contour model for texture segmentation.
2 Proposed Active contour model for texture segmentation
In what follows, we assume familiarity with concepts from differential geometry like geodesic distance, Riemannian Exponential and Riemannian Logarithm maps. A thorough introduction to these concepts can be found in the books [2, 9]. We are given an intensity image . Our algorithm assumes that the image contains a background and a foreground texture. At every point , let be a vector of intensities over a small neighborhood333Although we use a continuous region to model the image domain, we are implicitly assuming a discrete image domain while defining the concept of a neighborhood . We choose to ignore this discrepancy., say of size . Given a closed contour on , we define the following two covariance matrices:
where denote the interior and exterior of respectively. Note that and both belong to the set of symmetric positive definite matrices, which is a Riemannian manifold henceforth denoted by . Let denote the geodesic distance between two points of this manifold. Since the image contains two different texture regions, it is evident that the two covariance matrices(points on this manifold) defined in (1) will be furthest away (in terms of geodesic distance) from each other when the contour lies on the boundary between the two textures. We justify this claim with an empirical evidence in Figure 1.
For a given texture image , we propose the following cost function on the set of all closed contours defined on :
where are defined in Equation (1). We find the contour that maximizes this cost, using the gradient ascent approach giving us a novel active contour scheme. Instead of working on parametric representations of , we work with its level set representation which has several benefits as discussed in .
The curve is represented as the zero level set of the function , i.e., . A typical choice of is the signed distance function of :
with as the usual Euclidean distance on between and . The sets can then be defined in terms of the level set function as
Using the level set function and the Heaviside function
we redefine the covariance matrices from Equation (1), as
Re-writing our cost function from Equation (2) in terms of the level set function gives us
To maximize this cost function we use gradient ascent algorithm, and the gradient is computed as follows
where and are the Riemannian inner products defined on the Tangent space of at points and , respectively. Specific details on this inner product can be found in . The derivatives of the geodesic distance is given by444A simpler explanation for this can be given in case we are working with instead of . In this case and . This is exactly what is done by the Riemannian Log map on manifolds.
where Log denotes the Riemannian log map defined on . Derivatives of the covariance matrices defined in Equation (4) are given by
where and are given by
The gradient ascent as a level set evolution equation is therefore given by
In the next section, we provide necessary implementation details and results on various images.
The Heaviside and the Dirac delta functions are not continuous, instead, we use the following smooth approximation as given in :
We add curvature flow as a regularizer to obtain the following level set evolution equation:
where is the curvature of the curve and is the weight assigned to the curvature term.
We evolve an initial contour given by the user till the cost function given in Equation (2) increases, we stop the evolution the moment it decreases. We re-initialize the level set function when required following the algorithm given in . All images shown in this section are of size pixels. The results shown in this section were obtained on a Intel Core2Duo, 2GB RAM machine using MATLAB. The time required for computing these results was under minutes, most of the time taken in re-initializing the level set function.
We begin by first validating our algorithm on artificially created images. Results are shown in Figure 2.
Topology of the evolving contour can change due to the level set implementation. Next, we give results on real texture images, that of a zebra and the Europe night sky image, in Figure 3. Texture being a neighborhood property rather than a pixel property, the segmented boundary will lie a pixel or two away from the actual boundary.
We next compare our results with the results generated by the algorithm in , on some images from the Berkeley Segmentation dataset , in Figure 4. We have used images from  to display their results. One can clearly see that our algorithm gives a better texture segmentation. Small noise-like artifacts are in fact regions where texture similar to the object texture is present, for instance, in the tiger image, there are reflection of the tiger strips in the water that our algorithm is able to successfully segment.
With our algorithm, one can also segment usual gray level images, as explained next. Let the neighborhood size to be , i.e. . The covariance matrices defined in Equation (1), will simply be the mean of squared intensities in the interior and exterior of the closed contour , respectively. Also note that the covariance matrices now belong to , i.e., the set of positive real numbers , of course with a metric different from the usual Euclidean one on . Our algorithm will then find the contour that maximizes the difference (geodesic distance on ) between the two numbers and . Typical image segmentation results using this approach and results using the Chan & Vese active contours  is shown in Figure 5.
Of course, with , the covariance matrix can capture textures of that scale only. If we have large scale textures, our algorithm will over-segment the image, as can be seen in Figure 6. Simply increasing the neighborhood size may not solve the problem, as the detected boundary may not be properly localized near the actual texture boundary.
In this paper, we propose a novel active contour based unsupervised texture segmentation algorithm. The algorithm finds a contour with maximum geodesic distance between its interior and exterior intensity covariance matrices. The results from previous section are in favor of our algorithm. With the least possible neighborhood size , the process successfully segments gray-level images.
In its current state, the method depends on the size of the neighborhood . Efforts are on to make it independent of , either using a semi-supervised approach or using other multi-scale methods. Instead of intensity covariance matrices, one may also use covariance matrices of well-chosen multi-scale texture features. The method is able to capture even a slight deviation in a texture. This may be an advantage in some cases, but generally one may want the algorithm to be more invariant to small deviations.
-  Gilles Aubert and Pierre Kornprobst. Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations (Applied Mathematical Sciences). Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2006.
-  William M. Boothby. An introduction to differentiable manifolds and Riemannian geometry / William M. Boothby. Academic Press New York, 1975.
-  Vicent Caselles, Ron Kimmel, and Guillermo Sapiro. Geodesic active contours. International Journal of Computer Vision, 22(1):61–79, February 1997.
-  Tony F. Chan and Luminita A. Vese. Active contours without edges. IEEE Transactions on Image Processing, 10(2):266–277, 2001.
-  B. B. Chaudhuri and Nirupam Sarkar. Texture segmentation using fractal dimension. IEEE Trans. Pattern Anal. Mach. Intell., 17(1):72–77, January 1995.
-  Chen Sagiv, N. A. Sochen, and Yehoshua Y. Zeevi. Integrated active contours for texture segmentation. IEEE Trans. Image Process., 15(6):1633–1646, 2006.
-  George R. Cross and Anil K. Jain. Markov random field texture models. IEEE Trans. Pattern Anal. Mach. Intell., 5(1):25–39, 1983.
-  M.N. Do and M. Vetterli. Wavelet-based texture retrieval using generalized gaussian density and kullback-leibler distance. Image Processing, IEEE Transactions on, 11(2):146–158, 2002.
-  Manfredo P. do Carmo. Riemannian Geometry. Birkhäuser, Boston, MA, 1992.
-  Michael Donoser and Horst Bischof. Using covariance matrices for unsupervised texture segmentation. In ICPR, pages 1–4, 2008.
-  Dennis F. Dunn and William E. Higgins. Optimal gabor filters for texture segmentation. IEEE Transactions on Image Processing, 5(7):947–964, 1995.
-  Mohammad Faizal Ahmad Fauzi and Paul H. Lewis. Automatic texture segmentation for content-based image retrieval application. Pattern Anal. Appl., 9(4):307–323, October 2006.
-  P. Thomas Fletcher and Sarang C. Joshi. Riemannian geometry for the statistical analysis of diffusion tensor data. Signal Processing, 87(2):250–262, 2007.
-  R.M. Haralick. Statistical and structural approaches to texture. Proceedings of the IEEE, 67(5):786–804, 1979.
-  Nawal Houhou, Jean-Philippe Thiran, and Xavier Bresson. Fast texture segmentation model based on the shape operator and active contour. In CVPR, 2008.
-  Peter Howarth and Stefan RÃ¼ger. S.: Evaluation of texture features for content-based image retrieval. In In: Proceedings of the International Conference on Image and Video Retrieval, Springer-Verlag, 2004.
-  Michael Kass, Andrew Witkin, and Demetri Terzopoulos. Snakes: Active contour models. International Journal of Computer Vision, 1(4):321–331, 1988.
-  K. I. Laws. Textured image segmentation. PhD thesis, Univ. Southern California, Los Angeles, CA, USA, 1980.
-  D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int’l Conf. Computer Vision, volume 2, pages 416–423, July 2001.
-  R. Muzzolini, Y.-H. Yang, and R. Pierson. Multiresolution texture segmentation with application to diagnostic ultrasound images. Medical Imaging, IEEE Transactions on, 12(1):108–123, 1993.
-  Timo Ojala, Matti PietikÃÂ¤inen, and David Harwood. A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 29(1):51 – 59, 1996.
-  Mikaël Rousson, Thomas Brox, and Rachid Deriche. Active unsupervised texture segmentation on a diffusion based feature space. In CVPR (2), pages 699–706, 2003.
-  M. A. Savelonas, D. K. Iakovidis, and D. E. Maroulis. An LBP-Based Active Contour Algorithm for Unsupervised Texture Segmentation. In Proc. 18th International Conference on Pattern Recognition ICPR 2006, volume 2, 2006.
-  Mark Sussman, Peter Smereka, and Stanley Osher. A level set approach for computing solutions to incompressible two-phase flow. Journal of Computational Physics, 114(1):146 – 159, 1994.
-  Hideyuki Tamura, Shunji Mori, and Takashi Yamawaki. Textural Features Corresponding to Visual Perception. IEEE Transaction on Systems, Man, and Cybernetics, 8(6):460–472, June 1978.
-  Sinisa Todorovic and Narendra Ahuja. Texel-based texture segmentation. In ICCV, pages 841–848, 2009.
-  M. Tüceryan and Anil K. Jain. Texture segmentation using voronoi polygons. IEEE Trans. Pattern Anal. Mach. Intell., 12(2):211–216, February 1990.
-  Oncel Tuzel, Fatih Porikli, and Peter Meer. Region covariance: a fast descriptor for detection and classification. In Proceedings of the 9th European conference on Computer Vision - Volume Part II, ECCV’06, pages 589–600, Berlin, Heidelberg, 2006. Springer-Verlag.
-  Luminita A. Vese and Stanley Osher. Modeling textures with total variation minimization and oscillating patterns in image processing. J. Sci. Comput., 19(1-3):553–572, 2003.
-  Zhizhou Wang and Baba C. Vemuri. Dti segmentation using an information theoretic tensor dissimilarity measure. IEEE Trans. Med. Imaging, 24(10):1267–1277, 2005.
-  Linjiang Yu and Georgy L. Gimel’farb. Separating a colour texture from an arbitrary background. In DICTA, pages 489–498, 2003.
-  Song Chun Zhu, Yingnian Wu, and David Mumford. Filters, random fields and maximum entropy (frame) – towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2):1–20, 1998.
-  S. W. Zucker and D. Terzopoulos. Finding Structure in Co-Occurrence Matrices for Texture Analysis. Computer Graphics and Image Processing, 12:286–308, dec 1980.