Invariant Subsets of Scattered Trees and the Tree Alternative Property of Bonato and Tardif
Abstract.
A tree is scattered if it does not contain a subdivision of the complete binary tree as a subtree. We show that every scattered tree contains a vertex, an edge, or a set of at most two ends preserved by every embedding of T. This extends results of Halin, Polat and Sabidussi.
Calling two trees equimorphic if each embeds in the other, we then prove that either every tree that is equimorphic to a scattered tree is isomorphic to , or there are infinitely many pairwise nonisomorphic trees which are equimorphic to T. This proves the tree alternative conjecture of Bonato and Tardif for scattered trees, and a conjecture of Tyomkyn for locally finite scattered trees.
Key words and phrases:
graphs, trees, equimorphy, isomorphy2000 Mathematics Subject Classification:
Partially ordered sets and lattices (06A, 06B)1. Introduction
In this paper we deal with trees, alias connected graphs with no cycle. We follow Diestel [3] for most of the standard graph theory notions. A ray is a oneway infinite path.
We first generalizes theorems dealing with automorphisms of trees obtained by Polat and Sabidussi [16]. They proved that every rayless tree has either a vertex or an edge preserved by every automorphism, and that a scattered tree having a set of at least three ends of maximal order contains a rayless tree preserved by every automorphism. (Scattered trees are defined in the abstract and at the end of Section 2. Ends are equivalence classes of rays and defined in Section 5, just before Subsection 5.1. The order of an end, which we usually call rank, is defined and discussed in Subsection 5.4.)
Our first result pertains to scattered trees for which there is no end of maximal order. We prove that if T is such a tree then it contains a rayless tree preserved by every embedding (injective endomorphism) of T. Together with a result of Halin [4], who showed that every rayless tree contains a vertex or an edge preserved by every embedding, we obtain the first main contribution of this paper:
Theorem 1.1.
If a tree is scattered, then either there is one vertex, one edge, or a set of at most two ends preserved by every embedding of .
This is proved in Section 7 as a consequence of Lemmas 7.4 and 7.5. Of course the fact that every end in a scattered tree has an order (or rank), which has been proven by Jung in [10], is needed. However, for our proofs we need a more structural development, with a finer differentiation of the types of preserved elements. In this context we will also reprove Jung’s theorem (cf. Section 6.1).
The above results pertain to scattered trees with no end of maximal order or at least 3 ends of maximal order. If there are only two ends of maximal order, they must be preserved. However, the situation of two ends preserved by every embedding is quite easy to characterize:
Proposition 1.2.
A tree has a set of two distinct ends preserved by every embedding if and only if it contains a twoway infinite path preserved by every embedding.
The case of an end preserved by every embedding is more subtle. Indeed, this does not imply that the end contains a ray preserved by every embedding.
We provide examples in Subsection 5.8 of scattered trees with such an end but which do not contain an infinite path nor a vertex nor an edge preserved by every embedding.
In order to present our next results, we introduce ^{1}^{1}1We will use an alternative presentation in Section 9 based on the notion of level function (see Subsection 5.1). the following notions.
Let be a tree and be an end. We say that is preserved forward, resp. backward, by an embedding of if there is some ray such that , resp. . We say that is almost rigid if it is preserved backward and forward by every embedding, that is every embedding fixes pointwise a cofinite subset of every ray belonging to .
We recall that two trees and are equimorphic, or twins, if is isomorphic to an induced subtree of and is isomorphic to an induced subtree of . As we will see (cf. Lemma 4.1), if is a ray, then decomposes as a sum of rooted trees indexed by , that is where is the tree, rooted at , whose vertex set is the connected component of in if and in if . If the number of pairwise nonequimorphic rooted trees is finite, we say that is regular. We say that an end is regular if it contains some regular ray (in which case all other rays that it contains are regular).
Theorem 1.3.
Let be a tree. If is scattered and contains exactly one end
of maximal order, then is preserved forward by every
embedding.
If contains a regular end preserved forward by
every embedding, then contains some ray preserved by every
embedding provided that is scattered or is not almost rigid.
Theorem 1.3 will be proved in Subsection 5.7; it follows from Corollary 5.24, Proposition 5.8 and Proposition 5.12.
Corollary 1.4.
Let be a scattered tree. If contains a regular almost rigid
end , then contains a ray fixed pointwise by every embedding.
In particular, the set of embeddings of has a common fixed point.
Examples of trees containing an end preserved backward by every embedding and no end preserved forward by every embedding were obtained independently by Hamann [7] (see Example 5 in Subsection 5.8) and Lehner [14] (see Example 6 in Subsection 5.8). They are reproduced with their permission.
Definition 1.5.
A tree is called stable if either:

There is a vertex or an edge or a twoway infinite path or a oneway infinite path preserved by every embedding or an almost rigid end;

Or there exists a nonregular end which is preserved forward by every embedding.
Theorem 1.6.
Every scattered tree is stable.
We will apply these results to conjectures of BonatoTardif and Tyomkyn.
Definition 1.7.
The tree alternative property holds for a tree if either every tree equimorphic to is isomorphic to , or else there are infinitely many pairwise nonisomorphic trees which are equimorphic to .
Conjecture 1.8 (BonatoTardif [1]).
The tree alternative property holds for every tree.
Bonato and Tardif [1] proved that their conjecture holds for rayless trees, and their result was extended to rayless graphs by Bonato et al, see [2].
Now let denote the set of twins of up to isomorphism. Note that if is a tree for which every embedding is an automorphism, that is surjective, then ; in particular this is the case for any locally finite rooted tree. On the other hand a star with infinitely many vertices is an example of a tree having embeddings which are not automorphisms and with . Another such example is the tree consisting of two disjoint stars with infinitely many vertices whose roots are adjacent.
As this will be developed in Subsection 5.1, if is an end in a tree , there is an orientation of the edges of which is the (oriented) covering relation of an ordering on the set of vertices of . Indeed, for every vertex there is unique ray starting at and belonging to ; denoting this ray by we may set if . If is a vertex, let be the induced rooted tree rooted at with .
We are now ready to state the tree alternative result for stable trees.
Theorem 1.9.

The tree alternative conjecture holds for stable trees. In particular:

If is a stable tree which does not contain a vertex or an edge preserved by every embedding or an almost rigid end and which has a nonsurjective embedding, then unless is the oneway infinite path.

If has a vertex or an edge or an almost rigid end preserved by every embedding and is locally finite, then every embedding of is an automorphism of .

If has an almost rigid end, then if and only if for every vertex . Otherwise .

has a nonregular and not almost rigid end preserved forward by every embedding, then .
Theorem 1.9 is proved in Sections with the help of results of Section . Note that Theorems 1.9 and 1.6 together with Definition 1.5 imply:
Corollary 1.10.
Let be a scattered tree with . Then there exists a vertex or an edge or a twoway infinite path or a oneway infinite path or an almost rigid end preserved by every embedding of .
Tyomkyn [18] proved that the tree alternative property holds for rooted trees and conjectured that is infinite at the exception of the oneway infinite path, for every locally finite tree which has a nonsurjective embedding.
We obtain from Theorem 1.6 and from Theorem 1.9 we now obtain the second main contribution of the paper:
Theorem 1.11.
The tree alternative conjecture holds for scattered trees, and Tyomkyn’s conjecture holds for locally finite scattered trees.
Theorem 1.9 clearly holds for finite trees. That Theorem 1.6 holds for finite trees can either be seen directly by successively removing endpoints until an edge or a single vertex remains, or by just applying the Polat, Sabidussi, Halin results mentioned above. Hence, in the process of proving Theorems 1.6 and 1.9, we will mostly assume that our trees are infinite.
The results of this paper have been presented in part at the workshop on Homogeneous Structures, Banff, Nov. 813, 2015 and to the seminar on Discrete Mathematics, Hamburg, Feb. 19. 2016. The paper benefited of remarks from the audiences and we are pleased to thank them; in particular we thank M. Hamann and F. Lehner for their examples of trees.
We further thank M. Hamann for the information he provided on his recent result generalizing Theorem 1.1. This generalization [9] has two parts, and the first one reads as follows:
Theorem 1.12.
Let be a monoid of embeddings of a tree . Then either:

There is a vertex, an edge or a set of at most two ends preserved by each member of ;

Or contains a submonoid freely generated by two embeddings.
Note that if is a group of automorphisms, the conclusion is similar: in the second case contains a subgroup freely generated by two automorphisms (see Theorem 2.8 and Theorem 3.1 of [8]). For anterior versions of this result, see Tits [17], Pays and Valette [15] and Woess [19].
In his second part, M. Hamann proves that if the first case of the alternative above does not hold, then contains a subdivision of the binary tree. For doing so, he shows that there are infinitely many embeddings with different directions (the direction of an embedding being an end preserved forward by with a positive period (see Section 5 for the definition of period)). Then, he observes that there are two embeddings and such that none fixes the direction of the other. From that, he builds a subdivision of the binary tree.
We wish to warmly thank the referees for their truly valuable and appreciated contributions.
2. Basic definitions
Let be a tree. For we write if is adjacent to . An embedding of into a tree is an injection of to for which if and only if for all . An embedding of is an embedding of into . We write if is a subtree of . That is if is connected and . If and is an embedding of then is the subtree of induced by the set of vertices. Hence an embedding is an isomorphism of to . An embedding preserves a subtree of if . Note that if is finite or a twoway infinite path and preserves then , in which case restricted to is an automorphism of . The embedding fixes a vertex if it preserves it, that is if . Two trees and are isomorphic, resp. equimorphic or twins, and we set , resp. , if there is an isomorphism of onto , resp. an embedding of to and an embedding of to . A rooted tree is a pair consisting of a tree and a vertex , the root. If is a rooted tree with root and is a rooted tree with root , then an embedding of to is an embedding of to which maps to . The definitions above extend to rooted trees.
Let be an ordered set (poset), that is a set equipped with an order relation, denoted by . We say that is an ordered forest if for each element , the set is totally ordered. If furthermore, two arbitrary elements have a common lower bound, then is an ordered tree.
Rooted trees can be viewed as particular types of ordered trees. Indeed, let be a rooted tree; if we order by setting if is on the shortest path joining to we get an ordered tree with least element , such that is finite for every . Conversely suppose that is an ordered tree, with a least element , such that is finite for every . Then is a meetsemilattice, that is every pair of elements and has a meet (a largest lower bound) that we denote by and the unoriented graph of the covering relation of is a tree (we recall that an element covers , and we note , if and there is no element such that ).
In the sequel, we will rather consider the dual of the above order on a rooted tree, we will denote it and we will denote by the resulting poset.
Let be the set of finite sequences with entries and . For we denote by and the sequences obtained by adding and on the right of ; we also denote by the empty sequence. There are an order and a graph structure on . Ordering via the initial subsequence ordering, the least element being the empty sequence, we obtain an ordered tree, the binary ordered tree. Considering the (undirected) covering relation we get a tree, the binary tree also known as dyadic tree, which we denote by . A tree is scattered (See Figure 1) if no subdivision of the binary tree is embeddable into .
3. Automorphisms and embeddings
We recall that an automorphism of a tree is called a rotation if it fixes some vertex ; it is called an inversion if it reverses an edge, and it is called a translation if it leaves invariant a twoway infinite path. We recall the following basic result due to Tits:
Theorem 3.1.
[17] Every automorphism of a tree is either a rotation, an inversion or a translation.
The short and beautiful proof is worth recalling.
Proof.
For two elements , let be the length of the shortest path joining to . Let be an automorphism of . Let with minimum. If , then is a rotation. If and , then and is an inversion. If , then , where is the the shortest path joining and , is a twoway infinite path left invariant by . ∎
The case of embeddings is similar. For this Halin proved the following result:
Theorem 3.2.
[5] Let be an embedding of a tree into itself. Then either there is:

A fixed point; or

An edge reversed by ; or

A ray with .
Furthermore, each case excludes the others.
Case of the theorem above can be refined into two parts as follows:
Proposition 3.3.
Let be an embedding of a tree into itself. Then either there is:

A fixed point; or

An edge reversed by ; or

A twoway infinite path preserved by on which fixes no vertex and reverses no edge; or

A ray preserved by with a vertex not in the range of .
Furthermore, each case excludes the others and the edge and path in Cases (2) and Case (3) are unique, whereas in Case (4), the ray has a unique maximal extension to a ray preserved by .
4. Decomposition of trees
The basic idea for the finite case has been to study the effect of reducing the tree by removing the endpoints and study the effect of this operation on the embeddings, of course in this case the automorphisms, of the tree. If the tree is infinite, we will have to decompose the tree into larger pieces and then determine properties of the tree in relation to properties of the parts and the way those parts are put together to reconstitute the tree. Of course some notation, which will then be used throughout the paper, has to be introduced to describe those actions. In order to reduce the length of chains of symbols we will often, if we think that no confusion is possible, use the symbol for a tree to also stand for the set of its vertices and the set of vertices of an induced subtree of a tree to stand for the subtree. So for example if is a path in a tree, then we might write and to indicate that is a vertex of and is a subpath of and so on.
Let be a tree and be a subset of . We denote by the subgraph of induced by . If is a subtree of and a vertex of , we denote by the connected component of in . Note that and that since is a tree, is the set of vertices on the paths of meeting at . Note that whenever the neighbourhood of is included in . We denote by the tree rooted at . The tree then consists of the tree with the rooted trees attached at for every . In general the trees may be given as a set of rooted trees of the form . That is with a function associating the rooted trees to the vertices of and then we identify the root with and let the trees disjointly stick out of .
Here is the formal definition: Let be a tree, be a subset of and be a family of rooted trees. The sum of this family is the tree denoted by , or also simply , which is obtained by first taking isomorphic copies of those trees which are pairwise disjoint and which contain instead of as a root. With the understanding that if the trees are already pairwise disjoint and contain as a root, then we will not take different copies of them. The set of vertices of the tree is the set and the edge set is . If we simply denote this tree by . In some special cases we will simplify even further. For example if is a set of rooted trees, then with the understanding that also denotes the ray indexed naturally by . As to be expected we then have:
Lemma 4.1.
If is a tree and a subtree, then .
Proof.
First, the vertexsets of the trees and for are disjoint and if there is an edge adjacent to a vertex in and also adjacent to a vertex in , then it is the edge . Indeed, by construction, for each . If and have a nonempty intersection or there is an edge not equal to from to , then the path from to would complete a cycle of . ∎
Let be a tree and . For every vertex , the distance from to , denoted by is the least integer such that there is some path of length from to some vertex . Hence iff . If induces a subtree, say , this vertex is unique; we denote it by .
Lemma 4.2.
If is a subtree of , then the map from to is a retraction of reflexive graphs and for every .
Proof.
Clearly, if and only if . Hence and is the range of . This amounts to say that is a setretraction of onto . To conclude that it is a reflexivegraphretraction, we need to prove that it transforms an edge into an edge or identifies its end vertices. For that, let , and . Then, as it is easy to see, . Hence, if and is not an edge, then as required. The fact that the vertices on the shortest path from a vertex to belong to ensures that is connected. It follows that is the sum . ∎
Lemma 4.3.
Let be a tree, and be two infinite paths of and be an embedding of into . Then is embeddable into for every if and only if has an extension to an embedding of into itself such that .
Proof.
For the direct implication, choose for each an embedding of into . Observe that is an embedding of into and . According to Lemma 4.1 , hence the implication holds. For the converse, note that if is an extension of , then induces an embedding of into for every vertex of degree . Thus the converse holds if is a twoway infinite path. If is a oneway infinite path, i.e. a ray, let be its vertex of degree . Due to the condition on the map , it induces an embedding of into . ∎
Corollary 4.4.
Let be an embedding of a tree which preserves a
twowayinfinite path . Then for all .
If is an embedding which preserves a ray starting at with the predecessor of in not in , then
for all .
5. Ends
Let be a tree. Two rays and are called equivalent if the set of vertices induces a ray, equivalently if for some finite subset of . This relation is an equivalence relation, with classes called ends; the equivalence class of a ray is denoted by . The set of ends is denoted by .
We note that since is tree, then for each end and there is a unique ray originating at and belonging to . We denote it by . For we denote by (or if there is a risk of confusion) the vertex of at distance of . For we denote by the tree rooted at and whose vertex set is the connected component of in where and are the predecessor and successor of in . If , this rooted tree will be also denoted by , its vertex set is the connected component of in . We note that .
To each end is attached an order and to a topology. We discuss their properties below.
5.1. Orders on a tree
Let be a tree and be a subtree. Let ; we set if is on the shortest path joining to some vertex of .
This relation is an order on , is the set of maximal elements of . For every edge disjoint from either or . Indeed, since is connected, no path containing and has its end vertices in , hence either the shortest path from to contains or the shortest path from to contains . For each vertex , . As a poset, is the dual of a forest. If reduces to a vertex , we denote by the resulting poset; its dual satisfies properties a) and b) given in Section 2.
Let . Let . We set if . This relation is an order. The unoriented covering relation of this order is the adjacency relation on . As a poset, equipped with this order is the dual of an ordered tree; this dual is a joinsemilattice. We denote by the join of two vertices .
Lemma 5.1.
Let be an embedding of a tree into itself. Then an end , as an equivalence class, is sent by into an equivalence class of and furthermore:
preserves the covering relation, (that is iff
for all ).
In particular is
a joinsemilattice embedding from into .
Proof.
Since we have for every . Let . We have respectively iff and and iff and . Since , implies trivially . Conversely, suppose that . Let . By deleting some elements of we may suppose that . Let , and . Since embeddings are isometric, we have . Since and , is on the shortest path joining to . This path ending at it is the image of the shortest path joining to , hence belongs to this path, proving that ; since , it follows that hence . From this, we have for all . Indeed, let . Let . Since is order preserving, , hence is both on the shortest path joining and and the shortest path joining and . Since is a oneto one isometry, . ∎
Corollary 5.2.
An embedding of preserves , that is , iff is a joinsemilattice embedding of .
5.2. Valuation, level function and period
Let be a tree and be an end of . To each embedding preserving we attach an integer , the period of . In order to do so we introduce the notion of valuation.
A valuation is any map such that
(1) 
for every .
Lemma 5.3.
Let be an end of . For every , there is a unique valuation such that . In particular two valuations differ by some constant.
Proof.
We order by . Let . Set where . Let . Observe that . If follows that . Hence is an valuation. If is any valuation, we must have and . Hence, . Thus iff . If is a valuation and , then is a valuation. Hence, if is an other valuation, set . Then is an valuation which coincides with on . Hence . ∎
From this lemma, it follows that if we choose a vertex , there is a unique valuation such that . Such a vertex is called the origin of the valuation.
Lemma 5.4.
Let be an end of , be an valuation on and be an embedding. Then preserves iff there is some so that for all vertices of . Moreover, is nonnegative iff preserves forward; is negative iff preserves backward and not forward.
Proof.
Set . If preserves , then is an valuation. Hence from Lemma 5.3, for some . Conversely, suppose that for all vertices of . Let . We have . In particular, iff , that is iff . Since preserves the covering relation , it preserves (Corollary 5.2). Suppose that preserves forward, then there is some such that . This implies for , hence . Similarly, if preserves backward there is some such that , hence . If , then . We prove that if preserves , then it preserves forward or backward according to the sign of .
Claim 5.5.
Let , and . Then and .
Proof of Claim 5.5. Since we have ; since we have , hence is well defined. We have . Since and , and are comparable. Since , .
Pick any . Let and . According to Claim 5.5, . If , then and hence , proving that is preserved forward. If , then , and is fixed pointwise by . If , then then and is preserved downward. ∎
The number is the period of w.r.t. . According to Lemma 5.3 if is an other valuation, then has the same period.
An embedding preserves a valuation if for all . In this case, preserves every other valuation attached to the same end. A map is a level function if this is the valuation associated with an end preserved forward by every embedding.
Lemma 5.6.
Let be a valuation associated with an end . Then is a level function iff every embedding of has a nonnegative period. In particular, is a level function provided that is almost rigid.
Proof.
The first assertion follows from Lemma 5.4. The second assertion is obvious. ∎
Lemma 5.7.
Let be a tree, be an end, be an embedding with nonnegative period and . Then
(2) 
and
(3) 
for every .
If , then is the set of fixed points of , whereas if , then is either a ray belonging to or a twoway infinite path preserved by .
Proof.
Equations (2) and (3) are equivalent. The second implies trivially the first; since the second equation follows from the first. To prove Equation (2) apply Claim 5.5: there is some such that . This implies hence .
Since and preserves , it follows that preserves that is . Since preserves , preserves some ray, say , belonging to , hence and thus is nonempty. Suppose . We claim that is totally ordered. From this, it follows that this is a path (a ray or a twoway infinite path). If this is not the case, contains at least two incomparable elements, say . Let . Then let . Since is an embedding and are incomparable. This implies that . Since and , we have contradicting the fact that is an isometry. ∎
5.3. Rays preserved by every embedding
We present two results on regular ends, namely Proposition 5.8 and Proposition 5.12. The proofs are based on the same idea, but the proof of the first one is much simpler.
Proposition 5.8.
Le be a scattered tree and be an almost rigid regular end. Then contains a ray preserved by every embedding of .
Proof.
Our aim is to find such that for every embedding . Indeed, since is almost rigid, all embeddings have period zero (Lemma 5.4) hence for every and every embedding . In fact, we prove that under the weaker assumption that is preserved forward by every embedding, then there is some such that for every embedding, from which follows that the ray is preserved by every embedding.
Pick . Let , set in and .
Claim 5.9.
There is such that for each the interval contains two integers such that and are equimorphic.
Proof of Claim 5.9. The existence of is immediate: say that two integers and are equivalent if and are equimorphic (as rooted trees). The fact that is regular means that this equivalence relation has only finitely many blocks. Pick two elements in each equivalence class whenever possible and otherwise one element; since the number of these elements is finite, some integer dominates the elements chosen. This integer has the required property.
Let be given by Claim 5.9. We claim that for every embedding of . Indeed, suppose not. Let in the joinsemilattice . Since , hence there is some with such that . Let such that and are equimorphic as rooted trees. Since , the rays and meet at (and not before), hence embeds into . In particular, the trees and embed into . Set , , , , . According to Claim 5.10, is nonscattered.
Claim 5.10.
Let be a rooted tree ordered by if is on the unique path joining to . Let such that , let be the tree rooted at whose vertex set is the connected component of in (where and are the neighbours of in ) and be the tree rooted at whose vertex set is the connected component of in . If there exist embeddings of the rooted tree into the rooted trees and , then is not scattered.
Proof of Claim 5.10. Let be an embedding of into and be an embedding of into . Then maps the path from to to a path from to and maps the path from to to the path from to to . Implying that there is an oriented path from to . It follows that every copy of contains three distinct vertices and with an oriented path from to excluding and an oriented path from to excluding and embeddings of into and into . Implying that contains a subdivision of the binary tree.
This concludes the proof of Proposition 5.8. ∎
When we have embeddings with positive period, we do not need that is scattered, but the proof is more complex.
Let be a tree and be an end. Let be the set of embeddings of , be the subset of those preserving , be the subset of those with positive period and be the greatest common divisor of the periods of members of . Let and ; we say that is a period of if for every such that . If has a positive period, then there is one which divides all the others, we will call it the period of .
If and is the period of we set . Furthermore, we set .
As we will see below, under the existence of embeddings with positive period, the regularity of an end amounts to the fact that it contains some periodic ray.
Lemma 5.11.
Let be a tree and be a regular end. Then,

for every embedding , the set contains some such that is a period of ;

all rays for have the same period; in particular this period divides ;

If every embedding of has a non negative period, then contains no twoway infinite path; in particular each is a ray.
Proof.
Item (1). Let and . Set and for . Say that two nonnegative integers are equivalent if . Since is a regular end, is a regular ray and this means that the equivalence relation above has only finitely many classes. Let be the period of . Then for every nonnegative integer . Since is an embedding of , is embeddable into for every . Hence, the rooted trees , for , form an increasing sequence w.r.t embeddability. Since the number of equivalence classes is finite, there is some such that all , for are equivalent (pick such that is maximal with respect to equimorphy). This means that on the set , the congruence class of modulo is included into the equivalence class of . By considering an upper bound of , we get an integer such that on the set each congruence class is included into some equivalence class of our relation above. This means is a period of hence and thus .
Item (2) Indeed, let , and be periods of and respectively. Let be the greatest common divisor of and . The rays and intersect on the ray , hence and , and thus , is a period of that ray. But then is a period of and . Taking for and