Singular fibers of the Gelfand–Cetlin system on
Abstract.
In this paper, we show that every singular fiber of the Gelfand–Cetlin system on coadjoint orbits of unitary groups is a smooth isotropic submanifold which is diffeomorphic to a stage quotient of a compact Lie group by free actions of two other compact Lie groups. In many cases, these singular fibers can be shown to be homogeneous spaces or even diffeomorphic to compact Lie groups. We also give a combinatorial formula for computing the dimensions of all singular fibers, and give a detailed description of these singular fibers in many cases, including the socalled (multi)diamond singularities. These (multi)diamond singular fibers are degenerate for the Gelfand–Cetlin system, but they are Lagrangian submanifolds diffeomorphic to direct products of special unitary groups and tori. Our methods of study are based on different ideas involving complex ellipsoids, Lie groupoids, and also general ideas coming from the theory of singularities of integrable Hamiltonian systems.
2010 Mathematics Subject Classification:
37J35, 17B08, 57R45Contents
1. Introduction
The GelfandCetlin system is a famous integrable Hamiltonian system on the coadjoint orbits of unitary groups, which was found and studied by Guillemin and Sternberg in early 1980s [GS83gelfand, GS83collective], using the socalled Thimm’s method of collective motions [ThimmCollective], and related to the classical work of Gelfand and Cetlin in representation theory [GC1950]. A result of Alekseev and Meinrenken [alekseev] says that this system is also equivalent to an integrable system found by Flaschka and Ratiu [Ratiu], via the socalled GinzburgWeinstein transformation. Compared to many other integrable systems, especially those arising in classical mechanics and physics (see, e.g., [bolsinovfomenko, zung1996symplectic]), the GelfandCetlin has some very special topological and geometric properties:

Its base space (i.e. the space of connected fibers of the momentum map) is (affinely equivalent to) a convex polytope, similar to the case of toric systems (whose base spaces are the socalled Delzant polytopes [delzant]), even though the system is not toric.

In fact, the GelfandCetlin momentum map of the system (which consists of eigenvalue functions of a chain of matrices) generates a toric action, but only on a dense open set of the symplectic manifold in question. This momentum map is not globally smooth, though it can be changed into a smooth momentum map with the same fibers (i.e. preimages) by taking the symmetric functions of the eigenvalue functions. However, this smooth momentum map does not generate a torus action.

Unlike the toric case, where the singularities are all elliptic nondegenerate (in the sense of Vey–Eliasson; see, e.g., [eliasson1990normal, zung1996symplectic, MirandaZung_NF2004, Miranda_CEJM2014, mirandathesis] for nondegenerate singularities), the GelfandCetlin system admits many degenerate singularities.

The degenerate singular fibers of the GelfandCetlin system are very peculiar in the sense that they are all smooth isotropic submanifolds, as will be shown in this paper (see also Cho–Kim–Oh [CKO_GC2018] where the same result is obtained by different methods), while many degenerate singular fibers of other integrable Hamiltonian systems are singular varieties. (See, e.g., [bolsinov, bolsinovfomenko, Zung_CorankOne2000, Zung_CRAS2003] for various results about degenerate singularities).

It turns out that the GelfandCetlin systems can be obtained by the method of toric degenerations, see Nishinou–Nohara–Ueda [NNU_GC2010]. This method, which comes from algebraic geometry, is now known to generate a lot of “artificial” integrable Hamiltonian systems (see, e.g. Harada–Kaveh [harada2015integrable] and references therein). This toric degeneration nature of the GelfandCetlin system may be strongly related to its topological and geometrical particularities.
This paper is the result of a project dating back to 2006 to study the singularities of the GelfandCetlin system, from the point of view of the general topological theory of integrable Hamiltonian systems and their singularities. Unlike some other papers on the subject like [NNU_GC2010, CKO_GC2018], which are mainly motivated by considerations from algebraic geometry, our work is mainly motivated by considerations from dynamical systems. The first results of this project appeared in the form of a PhD thesis in 2009 of Iman Alamiddine [alamiddine2009GelfandCeitlin], who did it under the supervision of N.T. Zung and with the help of E. Miranda. The main result of this thesis is a complete description of a degenerate singular fiber in , which is a Lagrangian submanifold diffeomorphic to , together with a neighborhood of it (i.e. a symplectic normal form for the system around this degenerate fiber). At that time, we conjectured that all the fibers of the GelfandCetlin system (in any dimension) are smooth isotropic. In particular, N.T. Zung gave a talk on this subject at IMPA in 2009, where a method for studying the topology of singular fibers of the GelfandCetlin system using the complex ellipsoids was presented and some results were announced^{1}^{1}1See http://strato.impa.br/videos/workshop_geometry/geometry_070809_03.avi; this talk contains some good ideas and some errors..
Inspired by ideas coming from the GelfandCetlin system, D. Bouloc proved in 2015 [bouloc2015singular] a similar conjecture for all singular fibers of the KapovichMillson system of bending flows of 3D polygons [kapovich1996symplectic], and also of another similar integrable system studied by Nohara and Ueda [nohara2014toric] on the 2Grassmannian manifold. Namely, he showed that all these fibers are isotropic submanifolds (if the ambient symplectic variety itself is a manifold) or orbifolds (in special situations when the ambient symplectic spaces are orbifolds but not manifolds). Remark that these systems of KapovichMillson and Nohara–Ueda can also be obtained by toric degenerations (see Foth–Hu [FH_ToricDegeneration2005]).
Encouraged by the results of [bouloc2015singular], we have a more general conjecture about the singular fibers of integrable systems which can be obtained via toric degenerations, and have worked out the case of GelfandCetlin system for the present paper. In particular, we will show in this paper that every singular fiber of the Gelfand–Cetlin system on coadjoint orbits of unitary groups is a smooth isotropic submanifold which is diffeomorphic to a stage quotient of a compact Lie group by free actions of two other compact Lie groups (Theorem 4.16 and Corollary LABEL:cor:isotropic). In many cases, these singular fibers can be shown to be homogeneous spaces or even diffeomorphic to compact Lie groups. We also give a combinatorial formula for computing the dimensions of all singular fibers (Proposition 5.3), and give a detailed description of these singular fibers in many cases, including the socalled (multi)diamond singularities. These (multi)diamond singular fibers are degenerate for the Gelfand–Cetlin system, but they are Lagrangian submanifolds diffeomorphic to direct products of special unitary groups and tori.
We remark that Cho, Kim and Oh in a recent preprint [CKO_GC2018] already proved the smooth isotropic character of the singular fibers of the GelfandCetlin system and gave a combinatorial formula for the dimensions of these fibers. We found out [CKO_GC2018] by chance during the preparation of our paper. Their paper and ours are independent, and complement each other, because the motivations are completely different (Cho, Kim and Oh came to the problem from the point of view of pure symplectic geometry and mirror symmetry, while our project was motivated by problems coming from dynamical systems), and the methods used are also very different. In particular, in our work we use the variety of complete flags of complex ellipsoids which is not present in [CKO_GC2018]. It is precisely a kind of duality between such a variety of complete flags of complex ellipsoids and a coadjoint orbit of the unitary group (see Figure 2) that gives us a geometric description of the fibers of the GelfandCetlin system.
We notice that some related partial results on the topology of collective integrable systems have been obtained by Lane in [lane]. We remark also that, even though the singularities of the GelfandCetlin system are rather special from the point of view of general integrable Hamiltonian systems, there are still many similarities with other singularities that we encountered before. In particular, there is still a topological decomposition into direct products of simpler singularities, as will be seen in Section 5. One can also talk about the (real) toric degree of these singularities (see [Zung_Integrable2016] and references therein for the notion of toric degree), a topic that we will not discuss in this paper.
Organization of this paper
The rest of this paper is organized as follows:
In section 2 we give the basic notions of Gelfand–Cetlin systems.
In Section 3 we construct a variety of complete flags of complex ellipsoids, which is dual to a symplectic phase space of the GelfandCetlin system (i.e., a coadjoint oorbit of the unitary group), and “move” the GelfandCetlin from the coadjoint orbit to a “dual system” on the variety of complete flags of complex ellipsoids. The “dual GelfandCetlin momentum map” on this “dual Grassmannian” also consists of eigenvalue functions of appropriate matrices, and the two momentum maps (the GelfandCetlin momentum map and its “dual map” have the same image).
In Section 4 we study the symmetry group of an ellipsoid flag with fixed eigenvalues, which gives a new geometrical interpretation of the Gelfand–Cetlin system, and use this machinery to define the symmetry groupoid of ellipsoid flags. These objects yield a good understanding of the geometry of the Gelfand–Cetlin system: The main result of this section is Theorem 4.16 where we prove that the fibers of Gelfand–Cetlin system are smooth submanifolds and identify these manifolds with a quotient manifold constructed using this symmetry groupoid of ellipsoid flags.
Based on the results of Section 3 and Section 4, in Section 5 we give a combinatorial formula for the dimensions of the fibers of the GelfandCetlin system, and show a necessary and sufficient condition for a fiber to be of maximal possible dimension, or equivalently, to be a Lagrangian submanifold (Proposition 5.3). We also give many concrete examples of the fibers, together with their topological description. In particular, there is an interesting family of degenerate singular fibers, which we call (multiple)diamond singularities because the equalities in the corresponding GelfandCetlin triangles form “diamonds”, and which are Lagrangian submanifolds diffeomorphic to compact Lie groups of the type
(1.1) 
where is the number of diamonds and are their sizes. A particular case of diamond singularities worked out in detail in this section is the case of a degenerate fiber diffeomorphic to the 3dimensional sphere in , which has been studied before in Alamiddine’s thesis [alamiddine2009GelfandCeitlin].
Finally, in Section 6 we prove the isotropic character of the fibers of the GelfandCetlin theorem (Proposition 6.1) by direct computations. We believe that this is a general phenomenon, which should be true not only for the GelfandCetlin system and the systems studied by Bouloc [bouloc2015singular], but also for many other systems obtained via the toric degeneration method as well. In this relation, we present in Proposition 6.3 a general sufficient condition for the isotropicness of the singular fibers of an integrable Hamiltonian system. Unfortunately, this proposition is still not general enough: there are some degenerate singular fibers of the GelfandCetlin system (especially those which are Lagrangian) which do not satisfy the conditions of this last proposition. So one will need a more general proposition in order to avoid direct casebycase computations.
2. The Gelfand–Cetlin system on coadjoint orbits of
2.1. Coadjoint orbits of
Consider the unitary group
(2.1) 
and its Lie algebra
(2.2) 
The dual space is identified with the space of Hermitian matrices of size via the map defined for all and by
(2.3) 
Under this identification, the coadjoint representation of is simply the matrix conjugation: for all and . The spectrum of a Hermitian matrix is real. Therefore in this paper, by a coadjoint orbit of we always mean its identification with the set of all Hermitian matrices with a fixed real spectrum with .
Each coadjoint orbit is equipped with a natural symplectic form, called the Kirillov–Kostant–Souriau form, of which we recall here the expression. Note that if , then the tangent space of the coadjoint orbit at can be written as
(2.4) 
and the Kirillov–Kostant–Souriau form on take the form
(2.5) 
for . Moreover, each coadjoint orbit is diffeomorphic to a model given by the following proposition.
Proposition 2.1 ([audin2012torus, Prop. II.1.15]).
Suppose that the spectrum consists of distinct eigenvalues of respective multiplicities , that is
(2.6) 
Then the coadjoint orbit is diffeomorphic to the homogeneous space , where
(2.7) 
is the subgroup of blockdiagonal unitary matrices of size , with diagonal blocks of respective sizes .
From the above proposition we obtain that the orbit has dimension . In particular, we say that (and the corresponding orbit ) is generic if all the eigenvalues are distinct, that is
(2.8) 
In this case, is a dimensional torus and the generic orbit has maximal dimension .
2.2. The Gelfand–Cetlin system
Fix a spectrum . For any matrix and any integer , denote by the upperleft submatrix of of size . Since is a Hermitian matrix of size , it admits real eigenvalues
(2.9) 
The family of functions
(2.10) 
satisfies the Gelfand–Cetlin triangle (diagram) of inequalities [GS83gelfand, Prop. 5.3] shown in Figure 1.
Moreover, these functions commute pairwise under the Poisson bracket induced by the Kirillov–Kostant–Souriau form on , and we have the following theorem due to Guillemin and Sternberg [GS83collective]:
Theorem 2.2 ([GS83collective]).
On any coadjoint orbit , the nonconstant functions in define a completely integrable Hamiltonian system. The singular values of this system are the values for which there is a nontrivial equality in the Gelfand–Cetlin diagram on Figure 1.
Indeed, the family contains functions, which is precisely half the dimension of a generic coadjoint orbit. If the coadjoint orbit is not generic because some of the eigenvalues are equal, then the Gelfand–Cetlin inequalities imply that some of the functions in are constant. But one can check that the number of remaining nonconstant functions is, again, half the dimension of .
Remark that we can also define the functions on in the same way; they will be constant on : .
By abusing the notation, for any given spectrum we will call momentum map of the Gelfand–Cetlin system, or GelfandCetlin map, the following socalled collective function, which will be denoted by the same letter : is given:
(2.11) 
Even though this map has redundant components when is nongeneric, it has the same level sets as the usual momentum map of the Gelfand–Cetlin integrable system, which is enough for the matter of the present paper.
3. Geometric interpretation of the fibers
Fix an ordered real spectrum , and denote by
(3.1) 
the diagonal matrix with entries . We have a natural projection given by
(3.2) 
3.1. Partial and complete flag manifolds
Recall that a flag in is a sequence of vector subspaces
(3.3) 
with increasing dimensions (we will often omit ). The tuple is called the signature of the flag . A flag is complete if and , otherwise it is partial.
Fix a signature . One can associate to any flag with signature a basis of such that for any , the space is generated by the first vectors of this basis. Up to a Gram–Schmidt process, the basis can be supposed unitary, we then identify it with the matrix with columns . Conversely, any matrix represents a unique flag with signature such that for any , is generated by the first columns of .
Note that two matrices represent the same flag if and only if there exists a block diagonal matrix such that , where (with convention ). It is then standard to identify the set of all flags with signature in with the homogeneous space
(3.4) 
In particular, the set of complete flags in is identified with , where is the usual dimensional torus.
Remark 3.1.
In particular, Proposition 2.1 states that every coadjoint orbit is diffeomorphic to a flag manifold , with signature determined by the redundancies among the values .
3.2. Complex ellipsoids
Let be a finitedimensional (complex) vector space with a Hermitian product (the convention chosen in this paper for a Hermitian product is to be linear in the first variable and antilinear the second variable). Recall that a linear transformation is called Hermitian if for any , i.e., . A Hermitian transformation is diagonalizable in a unitary basis: there exists a basis of , with , such that . Moreover, its eigenvalues are real numbers. In particular, for any in we have
(3.5) 
By analogy with the real Euclidean case, we will set the following definition:
Definition 3.2.
A complex ellipsoid in is a subset of the form
(3.6) 
where is a positive definite Hermitian transformation (recall that is positive definite if for any , or equivalently if all its eigenvalues are positive). If is a basis of eigenvectors for and are the associated eigenvalues, then we say that has axes and radii , , .
When and , we simply write
(3.7) 
The complex ellipsoids satisfy the following immediate properties:
Lemma 3.3.
Let be positive definite Hermitian tranformations of , and a unitary map. Then:

for any in , there exists such that ,

if and only if ,

,

if , then preserves if and only if and commute.
Proof.
Fix . Since is positive definite, . Then is welldefined and we have:
which proves (1). But if , then we also have , hence . That is
for all . Since is again Hermitian, it is diagonalizable. But the above condition implies that all its eigenvalues are zero, so we conclude that , which proves (2). The equality (3) comes from:
Then (4) follows from (2) and (3). ∎
The following proposition deals with the intersection of a complex ellipsoid with a lower dimensional vector subspace.
Proposition 3.4.
Let be a complex ellipsoid in , and two linear subspaces of . Then:

is a complex ellipsoid in : there exists a positive definite Hermitian map such that .

In particular, if , then .

If , then .
Proof.
For (2), it suffices to remark that for every , where is the canonical identification. Now for (1), choose unitary such that . Define by , hence if and only if . Now define by . We obtain if and only if , which is equivalent to .
For (3), consider . By Lemma 3.3, there exists such that . If then, , and hence , lie in and so . With a symmetric argument we conclude that . ∎
The above properties motivate the following definitions:
Definition 3.5.
An ellipsoid flag in is a triple where:

is a (vector space) flag in ,

is a positive definite Hermitian matrix of size ,

is the increasing sequence of ellipsoids defined by .
The signature of is the signature of , and we say that is complete if is complete.
We will sometimes denote the ellipsoid flag by , or simply by .
Recall that, for each , the complex ellipsoid is defined by a unique positive definite Hermitian map . For brevity, we will say that the flag is defined by the family .
Definition 3.6.
Let be a complete ellipsoid flag. We call eigenvalues of the tuple
(3.8) 
where for any ,
(3.9) 
are the eigenvalues of the defining maps of .
Fix a positive spectrum and consider a matrix . Let such that . As above, for denote by the upperleft submatrix of and by the canonical inclusion. Consider
(3.10) 
the standard complete flag of , and its image by the linear map . By Proposition 3.4, the complete ellipsoid flag , and hence its image under denoted by flag , have eigenvalues
(3.11) 
Note that where is the map which sends each nondegenerate matrix to the flag defined by its columns, so the ellipsoid flag depends on the diagonalization chosen for .
Introduce the eigenvalue map
(3.12) 
and denote by the involution . Then the above remarks are summed up in the commutative diagram given in Figure 2.
Definition 3.7.
This this paper, by a fiber, we will mean either a preimage of the map on (i.e. a fiber of the GelfandCetlin system on a given coadjoint orbit), or a preimage of the map in the diagram in Figure 2.
4. The geometry of ellipsoid flags with fixed eigenvalues
Fix a positive spectrum. As before, denote by the diagonal matrix with entries , and by the corresponding ellipsoid in .
4.1. Symmetry group of an ellipsoid flag
Let in be a given complete flag and the ellipsoid flag obtained by intersecting with the “standard” ellipsoid . For a fixed , let us describe the subgroup of consisting of those elements which the dimensional complex ellipsoid . We recall the following result from linear algebra.
Lemma 4.1.
Let be a finite dimensional inner product space, a positive definite Hermitian transform, and the decomposition of into eigenspaces of . Then the subgroup of unitary transformations which commute with is exactly
(4.1) 
Remark 4.2.
In the above expression, by with we mean the map defined by
(4.2) 
for any .
This implies the following:
Proposition 4.3.
The subgroup of unitary transformations of that preserves is exactly
(4.3) 
where is the decomposition of into eigenspaces of .
Each has dimension determined by the eigenvalues
(4.4) 
of . (“Horizontal” equalities in the Gelfand–Cetlin diagram).
Proof.
For later use, we will need the subgroup of unitary transformations of which preserve not only but also (or equivalently, ), and the subgroup of transformations in whose restriction to is the identity, where denotes the orthogonal complement of in . Note that .
Lemma 4.4.
Fix a vector of length 1 and write with , . Then for any we have
(4.5) 
Proof.
Using the decomposition , write with and . Since and are orthogonal, we have
It remains to show that . To do so, remark that for any ,
hence . By Lemma 3.3, it follows that . ∎
We are now able to describe the groups and .
Proposition 4.5.
Let be the decomposition of into the eigenspaces of , and
(4.6) 
the induced decomposition of , where is the orthogonal complement of in . Denote by the orthogonal projection on . Then the group of unitary maps on which preserve and is the set of elements
(4.7) 
such that for any and we have .
The subgroup is the set of all above elements with equal to .
Proof.
Fix a unit vector (the orthonormal complement of in ), and write
with and . Then, by Lemma 4.4, for any ,
Note that depends of the choice of , but can be determined intrinsically as the subspace .
Let . Suppose preserves , and more precisely . Then
On the other hand preserves if and only if commutes with . For we have
Similarly,
It follows that preserves if and only if for any , . ∎
Remark 4.6.
The space is trivial if is orthogonal to . In this case, for any we have , so is also an eigenspace of associated to the same eigenvalue, and there must be a vertical equality in the Gelfand–Cetlin diagram for .
By contraposition, if the eigenvalue associated to appears only once in the Gelfand–Cetlin diagram, then the space has necessarily dimension .
However, note that the converse is not true. A vertical inequality in the Gelfand–Cetlin diagram does not imply that the corresponding space is trivial.
Remark 4.7.
If denotes the orthogonal complement of in , then can be written as the group of all elements of the form
This group is clearly isomorphic to
and each has codimension at most 1 in . In particular, the subgroup is isomorphic to
Consider the group
(4.8) 
where for each , is the group of unitary transformations of preserving . Consider also the subgroups defined by
(4.9) 
where for each , is the group of unitary transformations of preserving both and (or equivalently, both and ) and is the group of transformations in whose restriction to is the identity.
Definition 4.8.
The group is called the coarse symmetry group of the ellipsoid flag . The quotient manifold
(4.10) 
with respect to the free right action on defined by
(4.11) 
for any , , is called the reduced symmetry space of .
4.2. Pushforward of flags
In this subsection, is a complete flag in and is its intersection with the standard ellipsoid. Fix an element such that , i.e. columns of form a unitary basis of .
Definition 4.9.
Let in .

The pushforward of by is the complete flag
(4.12)