Separability and complete reducibility of subgroups of the Weyl group of a simple algebraic group of type
Abstract
Let be a connected reductive algebraic group defined over an algebraically closed field . The aim of this paper is to present a method to find triples with the following three properties. Property 1: is simple and has characteristic . Property 2: and are closed reductive subgroups of such that , and is a reductive pair. Property 3: is completely reducible, but not completely reducible. We exhibit our method by presenting a new example of such a triple in . Then we consider a rationality problem and a problem concerning conjugacy classes as important applications of our construction.
Keywords: algebraic groups, separable subgroups, complete reducibility
1 Introduction
Let be a connected reductive algebraic group defined over an algebraically closed field of characteristic . In [15, Sec. 3], J.P. Serre defined that a closed subgroup of is completely reducible (cr for short) if whenever is contained in a parabolic subgroup of , is contained in a Levi subgroup of . This is a faithful generalization of the notion of semisimplicity in representation theory since if , a subgroup of is cr if and only if acts complete reducibly on [15, Ex. 3.2.2(a)]. It is known that if a closed subgroup of is cr, then is reductive [15, Prop. 4.1]. Moreover, if , the converse holds [15, Prop. 4.2]. Therefore the notion of complete reducibility is not interesting if . In this paper, we assume that .
Completely reducible subgroups of connected reductive algebraic groups have been much studied [9], [10], [15]. Recently, studies of complete reducibility via Geometric Invariant Theory (GIT for short) have been fruitful [1], [2], [3]. In this paper, we see another application of GIT to complete reducibility (Proposition 3.6).
Here is the main problem we consider. Let and be closed reductive subgroups of such that . It is natural to ask whether being cr implies that is cr and vice versa. It is not difficult to find a counterexample for the forward direction. For example, take and where and sits inside via the adjoint representation. Another such example is [1, Ex. 3.45]. However, it is hard to get a counterexample for the reverse direction, and it necessarily involves a small . In [3, Sec. 7], Bate et al. presented the only known counterexample for the reverse direction where , , , and , which we call “the example”. The aim of this paper is to prove the following.
Theorem 1.1.
Let be a simple algebraic group of type defined over of characteristic . Then there exists a connected reductive subgroup of type of and a reductive subgroup (the dihedral group of order ) of such that is a reductive pair and is cr but not cr.
Our work is motivated by [3]. We recall a few relevant definitions and results here. We denote the Lie algebra of by . From now on, by a subgroup of , we always mean a closed subgroup of .
Definition 1.2.
Let be a subgroup of acting on by inner automorphisms. Let act on by the corresponding adjoint action. Then is called separable if .
Recall that we always have . In [3], Bate et al. investigated the relationship between complete reducibility and separability, and showed the following [3, Thm. 1.2, Thm. 1.4].
Proposition 1.3.
Suppose that is very good for . Then any subgroup of is separable in .
Proposition 1.4.
Suppose that is a reductive pair. Let be a subgroup of such that is a separable subgroup of . If is cr, then it is also cr.
Recall that a pair of reductive groups and is called a reductive pair if is an module direct summand of . This is automatically satisfied if . Propositions 1.3 and 1.4 imply that the subgroup in Theorem 1.1 must be nonseparable, which is possible for small only.
Now, we introduce the key notion of separable action, which is a slight generalization of the notion of a separable subgroup.
Definition 1.5.
Let and be subgroups of where acts on by group automorphisms. The action of is called separable in if . Note that the condition means that the fixed points of acting on , taken with their natural scheme structure, are smooth.
Here is a brief sketch of our method. Note that in our construction, needs to be .

Pick a parabolic subgroup of with a Levi subgroup of . Find a subgroup of such that acts nonseparably on the unipotent radical of . In our case, is generated by elements corresponding to certain reflections in the Weyl group of .

Conjugate by a suitable element of , and set . Then choose a connected reductive subgroup of such that is not cr. Use a recent result from GIT (Proposition 2.4) to show that is not cr. Note that is cr in our case.

Prove that is cr.
Remark 1.6.
It can be shown using [17, Thm. 13.4.2] that in Step 1 is a nonseparable subgroup of .
First of all, for Step 1, cannot be very good for by Proposition 1.3 and 1.4. It is known that and are bad for . We explain the reason why we choose , not (Remark 2.9). Remember that the nonseparable action on was the key ingredient for the example to work. Since is isomorphic to a subgroup of the Weyl group of , we are able to turn a problem of nonseparability into a purely combinatorial problem involving the root system of (Section 3.1). Regarding Step 2, we explain the reason of our choice of and explicitly (Remarks 3.4, 3.5). Our use of Proposition 2.4 gives an improved way for checking complete reducibility (Remark 3.7). Finally, Step 3 is easy.
In the and examples, the cr and noncr subgroups are finite. The following is the only known example of a triple with positive dimensional such that is cr but not cr. It is obtained by modifying [1, Ex. 3.45].
Example 1.7.
In view of this, it is natural to ask:
Open Problem 1.8.
Is there a triple of connected reductive algebraic groups such that is a reductive pair, is nonseparable in , and is cr but not cr?
Beyond its intrinsic interest, our example has some important consequences and applications. For example, in Section 6, we consider a rationality problem concerning complete reducibility. We need a definition first to explain our result there.
Definition 1.9.
Let be a subfield of an algebraically closed field . Let be a defined closed subgroup of a defined reductive algebraic group . Then is called cr over if whenever is contained in a defined parabolic subgroup of , it is contained in some defined Levi subgroup of .
Note that if is algebraically closed then cr over means cr in the usual sense. Here is the main result of Section 6.
Theorem 1.10.
Let be a nonperfect field of charecteristic , and let be a defined split simple algebraic group of type . Then there exists a defined subgroup of such that is cr over , but not cr over .
As another application of the example, we consider a problem concerning conjugacy classes. Given , we let act on by simultaneous conjugation:
In [16], Slodowy proved the following fundamental result applying Richardson’s tangent space argument, [12, Sec. 3], [13, Lem. 3.1].
Proposition 1.11.
Let be a reductive subgroup of a reductive algebraic group defined over . Let , let and let be the subgroup of generated by . Suppose that is a reductive pair and that is separable in . Then the intersection is a finite union of conjugacy classes.
Proposition 1.11 has many consequences. See [1], [16], and [18, Sec. 3] for example. In [3, Ex. 7.15], Bate et al. found a counterexample for showing that Proposition 1.11 fails without the separability hypothesis. In Section 7, we present a new counterexample to Proposition 1.11 without the separability hypothesis. Here is the main result of Section 7.
Theorem 1.12.
Let be a simple algebraic group of type defined over an algebraically closed of characteristic . Let be the connected reductive subsystem subgroup of type . Then there exists and a tuple such that is an infinite union of conjugacy classes. Note that is a reductive pair in this case.
Now, we give an outline of the paper. In Section 2, we fix our notation which follows [4], [8], and [17]. Also, we recall some preliminary results, in particular, Proposition 2.4 from GIT. After that, in Section 3, we prove our main result, Theorem 1.1. Then in Section 4, we consider a rationality problem, and prove Theorem 1.10. Finally, in Section 5, we discuss a problem concerning conjugacy classes, and prove Theorem 1.12.
2 Preliminaries
2.1 Notation
Throughout the paper, we denote by an algebraically closed field of positive characteristic . We denote the multiplicative group of by . We use a capital roman letter, , , , etc., to represent an algebraic group, and the corresponding lowercase gothic letter, , , , etc., to represent its Lie algebra. We sometimes use another notation for Lie algebras: , , and are the Lie algebras of , , and respectively.
We denote the identity component of by . We write for the derived group of . The unipotent radical of is denoted by . An algebraic group is reductive if . In particular, is simple as an algebraic group if is connected and all proper normal subgroups of are finite.
In this paper, when a subgroup of acts on , always acts on by inner automorphisms. The adjoint representation of is denoted by or just Ad if no confusion arises. We write and for the global and the infinitesimal centralizers of in and respectively. We write and for the set of characters and cocharacters of respectively.
2.2 Complete reducibility and GIT
Let be a connected reductive algebraic group. We recall Richardson’s formalism [14, Sec. 2.1–2.3] for the characterization of a parabolic subgroup of , a Levi subgroup of , and the unipotent radical of in terms of a cocharacter of and state a result from GIT (Proposition 2.4).
Definition 2.1.
Let be an affine variety. Let be a morphism of algebraic varieties. We say that exists if there exists a morphism (necessarily unique) whose restriction to is . If this limit exists, we set .
Definition 2.2.
Let be a cocharacter of . Define
Note that is a parabolic subgroup of , is a Levi subgroup of , and is a unipotent radical of [14, Sec. 2.12.3]. By [17, Prop. 8.4.5], any parabolic subgroup of , any Levi subgroup of , and any unipotent radical of can be expressed in this form. It is well known that .
Let be a reductive subgroup of . Then, there is a natural inclusion of cocharacter groups. Let . We write or just for the parabolic subgroup of corresponding to , and for the parabolic subgroup of corresponding to . It is obvious that and .
Definition 2.3.
Let . Define a map by
Note that the map is the usual canonical projection from to . Now, we state a result from GIT (see [1, Lem. 2.17, Thm. 3.1], [2, Thm. 3.3]).
Proposition 2.4.
Let be a subgroup of . Let be a cocharacter of with . If is cr, there exists such that for every .
2.3 Root subgroups and root subspaces
Let be a connected reductive algebraic group. Fix a maximal torus of . Let denote the set of roots of with respect to . We sometimes write for . Fix a Borel subgroup containing . Then is the set of positive roots of defined by . Let denote the set of simple roots of defined by . Let . We write for the corresponding root subgroup of and for the Lie algebra of . We define .
Let be a subgroup of normalized by some maximal torus of . Consider the adjoint representation of on . The root spaces of with respect to are also root spaces of with respect to , and the set of roots of relative to , , is a subset of .
Let . Let be the coroot corresponding to . Then is a homomorphism such that for some . We define . Let denote the reflection corresponding to in the Weyl group of . Each acts on the set of roots by the following formula [17, Lem. 7.1.8]: By [5, Prop. 6.4.2, Lem. 7.2.1], we can choose homomorphisms so that
(2.1) 
We define . Then we have
(2.2) 
Now, we list four lemmas which we need in our calculations. The first one is [17, Prop. 8.2.1].
Lemma 2.5.
Let be a parabolic subgroup of . Any element in can be expressed uniquely as
where the product is taken with respect to a fixed ordering of .
The next two lemmas [8, Lem. 32.5 and Lem. 33.3] are used to calculate .
Lemma 2.6.
Let . If no positive integral linear combination of and is a root of , then
Lemma 2.7.
Let be the root system of type spanned by roots and . Then
The last result is used to calculate .
Lemma 2.8.
Suppose that . Let be a subgroup of generated by all the where (the group is isomorphic to the Weyl group of ). Let be a subgroup of . Let be the set of orbits of the action of on . Then,
Proof.
When , (2.2) yields Then an easy calculation gives the desired result. ∎
Remark 2.9.
Lemma 2.8 holds in but fails in .
3 The example
3.1 Step 1
Let be a simple algebraic group of type defined over of characteristic . Fix a maximal torus of . Fix a Borel subgroup of containing . Let be the set of simple roots of . Figure 1 defines how each simple root of corresponds to each node in the Dynkin diagram of .
From [6, Appendix, Table B], one knows the coefficients of all positive roots of . We label all positive roots of in Table 1 in the Appendix. Our ordering of roots is different from [6, Appendix, Table B], which will be convenient later on.
The set of positive roots is Note that and are precisely the roots of such that the coefficient of is and respectively. We call the roots of the first type weight1 roots, and the second type weight2 roots. Define
Then is a parabolic subgroup of , and is a Levi subgroup of . Note that is of type . We have Define
Let be simple roots of . From the Cartan matrix of [7, Sec. 11.4] we have
From this, it is not difficult to calculate for all and for all . These calculations show how , and act on . Let be the corresponding homomorphism. Then we have
It is easy to see that . The orbits of in are
Thus Lemma 2.8 yields
Proposition 3.1.
The following is the most important technical result in this paper.
Proposition 3.2.
Let . Then must have the form,
Proof.
By Lemma 2.5, can be expressed uniquely as By (2.1), we have Thus we have
(3.1) 
A calculation using the commutator relations (Lemma 2.6 and Lemma 2.7) shows that
(3.2) 
Since and centralize , we have Set Then (3.2) simplifies to
Since centralizes , comparing the arguments of the term on both sides, we must have
which is equivalent to Then we obtain the desired result. ∎
Proposition 3.3.
acts nonseparably on .
Proof.
In view of Proposition 3.1, it suffices to show that . Suppose the contrary. Since by [17, Cor. 14.2.7] is isomorphic as a variety to for some , there exists a morphism of varieties such that and . By Lemma 2.5, can be expressed uniquely as Differentiating the last equation, and evaluating at , we obtain Since , we have
Then we have
Then from Proposition 3.2, we obtain This is a contradiction. ∎
3.2 Step 2
Let , pick any , and let . Now, set
Remark 3.4.
Remark 3.5.
In this case is the unique simple root not contained in . was chosen so that is generated by a Levi subgroup containing and all root subgroups of weight .
We have Note that Since is generated by all root subgroups of even weight, it is easy to see that is a closed subsystem of , thus is reductive by [3, Lem. 3.9]. Note that is of type .
Proposition 3.6.
is not cr.
Proof.
Let We have
So
It is easy to see that is of type , so is isomorphic to either or . We rule out the latter. Pick such that . Then since . Also, we have . Therefore . It is easy to check that the map is separable, so we have .
Let be the homomorphism as in Definition 2.3. In order to prove that is not cr, by Theorem 2.4 it suffices to find a tuple which is not conjugate to . Set Then
Now suppose that is conjugate to . Then there exists such that
Thus we have Note that So, by Lemma 2.5, can be expressed uniquely as Then we have
This contradicts Proposition 3.2. ∎
Remark 3.7.
Remark 3.8.
Instead of using to define , we can take , , or . In each case, a similar argument goes through and gives rise to a different example with the desired property.
3.3 Step 3
Proposition 3.9.
is cr.
Proof.
First note that is conjugate to , so is cr if and only if is cr. Then, by [1, Lem. 2.12, Cor. 3.22], it suffices to show that is cr. We can identify with the image of the corresponding subgroup of under the permutation representation . It is easy to see that . A quick calculation shows that this representation of is a direct sum of a trivial dimensional and irreducible dimensional subrepresentations. Therefore is cr. ∎
4 A rationality problem
We prove Theorem 1.10. The key here is again the existence of a dimensional curve such that is contained in but not contained in . The same phenomenon was seen in the example.
Proof of Theorem 1.10.
Let , , and be as in the hypothesis. We choose a defined split maximal torus such that for each the corresponding root , coroot , and homomorphism are defined over . Since is not perfect, there exists such that . We keep the notation of Section 3. Let
Now it is obvious that is defined. We already know that is cr by Proposition 3.9. Since and are split, and are defined by [4, \@slowromancapv@.20.4, \@slowromancapv@.20.5]. Suppose that there exists a Levi subgroup