On the Vere-Jones classification and existence of maximal measures for countable topological Markov chains Pacific J. Math., 209, No. 2, 365–380, 2003.

# On the Vere-Jones classification and existence of maximal measures for countable topological Markov chains 00footnotetext: Pacific J. Math., 209, No. 2, 365–380, 2003.

Sylvie Ruette
###### Abstract

We consider topological Markov chains (also called Markov shifts) on countable graphs. We show that a transient graph can be extended to a recurrent graph of equal entropy which is either positive recurrent of null recurrent, and we give an example of each type. We extend the notion of local entropy to topological Markov chains and prove that a transitive Markov chain admits a measure of maximal entropy (or maximal measure) whenever its local entropy is less than its (global) entropy.

## Introduction

In this article we are interested in connected oriented graphs and topological Markov chains. All the graphs we consider have a countable set of vertices. If is an oriented graph, let be the set of two-sided infinite sequences of vertices that form a path in and let denote the shift transformation. The Markov chain associated to is the (non compact) dynamical system . The entropy of the Markov chain was defined by Gurevich; it can be computed by several ways and satisfies the Variational Principle [9, 10].

In [16] Vere-Jones classifies connected oriented graphs as transient, null recurrent or positive recurrent according to the properties of the series associated with the number of loops, by analogy with probabilistic Markov chains. To a certain extent, positive recurrent graphs resemble finite graphs. In [9] Gurevich shows that a Markov chain on a connected graph admits a measure of maximal entropy (also called maximal measure) if and only if the graph is positive recurrent. In this case, this measure is unique and it is an ergodic Markov measure.

In [13, 14] Salama gives a geometric approach to the Vere-Jones classification. The fact that a graph can (or cannot) be “extended” or “contracted” without changing its entropy is closely related to its class. In particular a graph with no proper subgraph of equal entropy is positive recurrent. The converse is not true [14] (see also [6] for an example of a positive recurrent graph with a finite valency at every vertex that has no proper subgraph of equal entropy). This result shows that the positive recurrent class splits into two subclasses: a graph is called strongly positive recurrent if it has no proper subgraph of equal entropy; it is equivalent to a combinatorial condition (a finite connected graph is always strongly positive recurrent). In [13, 14] Salama also states that a graph is transient if and only if it can be extended to a bigger transient graph of equal entropy. We show that any transient graph is contained in a recurrent graph of equal entropy, which is positive or null recurrent depending on the properties of . We illustrate the two possibilities – a transient graph with a positive or null recurrent extension – by an example.

The result of Gurevich entirely solves the question of existence of a maximal measure in term of graph classification. Nevertheless it is not so easy to prove that a graph is positive recurrent and one may wish to have more efficient criteria. In [8] Gurevich and Zargaryan give a sufficient condition for existence of a maximal measure; it is formulated in terms of exponential growth of the number of paths inside and outside a finite subgraph. We give a new sufficient criterion based on local entropy.

Why consider local entropy? For a compact dynamical system, it is known that a null local entropy implies the existence of a maximal measure ([11], see also [1] for a similar but different result). This result may be strengthened in some cases: it is conjectured that, if is a map of the interval which is , , and satisfies , then there exists a maximal measure [2]. Our initial motivation comes from the conjecture above because smooth interval maps and Markov chains are closely related. If is (i.e. is and is -Hölder with ) with then an oriented graph can be associated to , is connected if is transitive, and there is a bijection between the maximal measures of and those of [2, 3]. We show that a Markov chain is strongly positive recurrent, thus admits a maximal measure, if its local entropy is strictly less that its Gurevich entropy. However this result does not apply directly to interval maps since the “isomorphism” between and its Markov extension is not continuous so it may not preserve local entropy (which depends on the distance).

The article is organized as follows. Section 1 contains definitions and basic properties on oriented graphs and Markov chains. In Section 2, after recalling the definitions of transient, null recurrent and positive recurrent graphs and some related properties, we show that any transient graph is contained in a recurrent graph of equal entropy (Proposition 2.8) and we give an example of a transient graph which extends to a positive recurrent (resp. null recurrent) graph. Section 3 is devoted to the problem of existence of maximal measures: Theorem 3.8 gives a sufficient condition for the existence of a maximal measure, based on local entropy.

## 1 Background

### 1.1 Graphs and paths

Let be an oriented graph with a countable set of vertices . If are two vertices, there is at most one arrow . A path of length is a sequence of vertices such that in for . This path is called a loop if . We say that the graph is connected if for all vertices there exists a path from to ; in the literature, such a graph is also called strongly connected.

If is a subgraph of , we write ; if in addition , we write and say that is a proper subgraph. If is a subset of , the set is denoted by . We also denote by the subgraph of whose vertices are and whose edges are all edges of between two vertices in .

Let be two vertices. We define the following quantities.

• is the number of paths such that and ; is the radius of convergence of the series .

• is the number of paths such that , and for ; is the radius of convergence of the series .

###### Proposition 1.1 (Vere-Jones [16])

Let be an oriented graph. If is connected, does not depend on and ; it denoted by .

If there is no confusion, and will be written and . For a graph these two radii will be written and .

### 1.2 Markov chains

Let be an oriented graph. is the set of two-sided infinite paths in , that is,

 ΓG={(vn)n∈Z∣∀n∈Z,vn→vn+1 in G}⊂(V(G))Z.

is the shift on . The (topological) Markov chain on the graph is the system .

The set is endowed with the discrete topology and is endowed with the induced topology of . The space is not compact unless is finite. A compatible distance on is given by , defined as follows: is identified with and the distance on is given by . If and are two elements of ,

 d(¯u,¯v)=∑n∈ZD(un,vn)2|n|≤3.

The Markov chain is transitive if for any non empty open sets there exists such that . Equivalently, is transitive if and only if the graph is connected. In the sequel we will be interested in connected graphs only.

### 1.3 Entropy

If is a finite graph, is compact and the topological entropy is well defined (see e.g. [5] for the definition of the topological entropy). If is a countable graph, the Gurevich entropy [9] of is given by

 h(G)=sup{htop(ΓH,σ)∣H⊂G,H finite}.

This entropy can also be computed in a combinatorial way, as the exponential growth of the number of paths with fixed endpoints [10].

###### Proposition 1.2 (Gurevich)

Let be a connected oriented graph. Then for any vertices

 h(G)=limn→+∞1nlogpGuv(n)=−logR(G).

Another way to compute the entropy is to compactify the space and then use the definition of topological entropy for compact metric spaces. If is an oriented graph, denote the one-point compactification of by and define as the closure of in . The distance naturally extends to . In [9] Gurevich shows that this gives the same entropy; this means that there is only very little dynamics added in this compactification. Moreover, the Variational Principle is still valid for Markov chains [9].

###### Theorem 1.3 (Gurevich)

Let be an oriented graph. Then

 h(G)=htop(¯¯¯¯ΓG,σ)=sup{hμ(ΓG)∣μ σ-invariant probability measure}.

## 2 On the classification of connected graphs

### 2.1 Transient, null recurrent, positive recurrent graphs

In [16] Vere-Jones gives a classification of connected graphs as transient, null recurrent or positive recurrent. The definitions are given in Table 1 (lines 1 and 2) as well as properties of the series which give an alternative definition.

In [13, 14] Salama studies the links between the classification and the possibility to extend or contract a graph without changing its entropy. It follows that a connected graph is transient if and only if it is strictly included in a connected graph of equal entropy, and that a graph with no proper subgraph of equal entropy is positive recurrent.

###### Remark 2.1

In [13] Salama claims that is independent of , which is not true; in [14] he uses the quantity and he states that if then for all vertices , which is wrong too (see Proposition 3.2 in [7]). It follows that in [13, 14] the statement “R=L” must be interpreted either as “ for some ” or “ for all ” depending on the context. This encouraged us to give the proofs of Salama’s results in this article.

In [14] Salama shows that a transient or null recurrent graph satisfies for all vertices ; we give the unpublished proof due to U. Fiebig [6].

###### Proposition 2.2 (Salama)

Let be a connected oriented graph. If is transient or null recurrent then for all vertices . Equivalently, if there exists a vertex such that then is positive recurrent.

Proof. For a connected oriented graph, it is obvious that for all , thus the two claims of the Proposition are equivalent. We prove the second one.

Let be a vertex of such that . Let for all . If we break a loop based in into first return loops, we get the following formula:

 ∑n≥0pGuu(n)xn=∑k≥0(F(x))k. (1)

Suppose that is transient, that is, . The map is analytic on and thus there exists such that . According to Equation (1) one gets that , which contradicts the definition of . Therefore is recurrent. Moreover by assumption, thus , which implies that is positive recurrent.

###### Definition 2.3

A connected oriented graph is called strongly positive recurrent if for all vertices .

###### Lemma 2.4

Let be a connected oriented graph and a vertex.

1. if and only if .

2. If is recurrent then is the unique positive number such that .

Proof. Use the fact that is increasing.

The following result deals with transient graphs [13].

###### Theorem 2.5 (Salama)

Let be a connected oriented graph of finite positive entropy. Then is transient if and only if there exists a connected oriented graph such that . If is transient then can be chosen transient.

Proof. The assumption on the entropy implies that . Suppose first that there exists a connected graph such that , that is, . Fix a vertex in . The graph is a proper subgraph of thus there exists such that , which implies that

 ∑n≥1fGuu(n)Rn<∑fG′uu(n)R′n≤1.

Therefore is transient.

Now suppose that is transient and fix a vertex in . One has . Let be an integer such that

 ∑n≥1fGuu(n)Rn+Rk<1.

Define the graph by adding a loop of length based at the vertex ; one has and

 ∑n≥1fG′uu(n)R′n≤∑n≥1fG′uu(n)Rn=∑n≥1fGuu(n)Rn+Rk<1. (2)

Equation (2) implies that and also that the graph is transient, so by Proposition 2.2. Then one has thus .

In [14] Salama proves that if for all vertices then there exists a proper subgraph of equal entropy. We show that the same conclusion holds if one supposes that for some . The proof below is a variant of the one of Salama. The converse is also true, as shown by U. Fiebig [6].

###### Proposition 2.6

Let be a connected oriented graph of positive entropy.

1. If there is a vertex such that then there exists a connected subgraph such that .

2. If there is a vertex such that then for all proper subgraphs one has .

Proof.

i) Suppose that . If is followed by a unique vertex, let be this vertex. If is followed by a unique vertex, let be this vertex, and so on. If this leads to define for all then , which is not allowed.

Let be the last built vertex; there exist two distinct vertices such that and . Let be the graph deprived of the arrow and the graph deprived of all the arrows , . Call the connected component of that contains (); obviously . For all one has

 fGuu(n)=fGuku(n−k)=fG1uku(n−k)+fG2uku(n−k),

thus there exists such that . One has

 R≤R(Gi)≤Luku(Gi)=Luu=R,

thus , that is, .

ii) Suppose that and consider . Suppose first that is a vertex of . The graph is positive recurrent by Proposition 2.2 so . Since there exists such that , thus

 ∑n≥1fG′uuRn<1. (3)

Moreover . If is transient then (Proposition 2.2) thus . If is recurrent then thus because of Equation (3). In both cases , that is, .

Suppose now that is not a vertex of and fix a vertex in . Let a path (in ) of minimal length between and , and let be a path of minimal length between and .

If is a loop in then

 (u0=u,u1,…,up=w0,w1,…,wn=v0,v1,…,vq=u)

is a first return loop based in in the graph . For all we get that , thus , that is, .

The following result gives a characterization of strongly positive recurrent graphs. It is a straightforward corollary of Proposition 2.6 (see also [6]).

###### Theorem 2.7

Let be a connected oriented graph of positive entropy. The following properties are equivalent:

1. for all one has (that is, is strongly positive recurrent),

2. there exists such that ,

3. has no proper subgraph of equal entropy.

### 2.2 Recurrent extensions of equal entropy of transient graphs

We show that any transient graph can be extended to a recurrent graph without changing the entropy by adding a (possibly infinite) number of loops. If the series is finite then the obtained recurrent graph is positive recurrent (but not strongly positive recurrent), otherwise it is null recurrent.

###### Proposition 2.8

Let be a transient graph of finite positive entropy. Then there exists a recurrent graph such that . Moreover can be chosen to be positive recurrent if for some vertex of , and is necessarily null recurrent otherwise.

Proof. The entropy of is finite and positive thus and there exists an integer such that . Define . Let be a vertex of and define ; one has . Moreover

 ∑n≥1αn≥∑n≥112n=1,

thus

 ∑n≥k+1αn=αk∑n≥1αn≥αk. (4)

We build a sequence of integers such that . For this, we define inductively a strictly increasing (finite or infinite) sequence of integers such that for all

 k∑i=0αni≤D2nkαn.

– Let be the greatest integer such that . By choice of one has , thus by Equation (4). This is the required property at rank .

– Suppose that are already defined. If then and we stop the construction. Otherwise let be the greatest integer such that

 k∑i=0αni+∑j≥nαj>D2.

By choice of and Equation (4), one has

 αnk+1≤∑j≥nk+1+1αj≤D2−k∑i=0αni.

This is the required property at rank .

Define a new graph by adding loops of length based at the vertex . Obviously one has , and by construction. Therefore

 ∑n≥1fG′uu(n)Rn=∑n≥1fGuu(n)Rn+∑i∈I2(pR)ni=1. (5)

This implies that . If is transient then and by Proposition 2.2, thus and Equation (5) leads to a contradiction. Therefore is recurrent. By Lemma 2.4(ii) one has , that is, . In addition,

 ∑n≥1nfG′uu(n)Rn=∑n≥1nfGuu(n)Rn+∑i∈Iniαni

and this quantity is finite if and only if is finite. In this case the graph is positive recurrent.

If , let be a recurrent graph containing with . Then is null recurrent because

 ∑n≥1nfHuu(n)Rn≥∑n≥1nfGuu(n)Rn=+∞.

###### Example 2.9

We build a positive (resp. null) recurrent graph such that and then we delete an arrow to obtain a graph which is transient and such that . First we give a description of depending on a sequence of integers then we give two different values to the sequence so as to obtain a positive recurrent graph in one case and a null recurrent graph in the other case.

Let be a vertex and a sequence of non negative integers for , with . The graph is composed of loops of length based at the vertex for all (see Figure 1). More precisely, define the set of vertices of as

 V={u}∪+∞⋃n=1{vn,ik∣1≤i≤a(n),1≤k≤n−1},

where the vertices above are distinct. Let for . There is an arrow for and there is no other arrow in . The graph is connected and for .

The sequence is chosen such that it satisfies

 ∑n≥1a(n)Ln=1, (6)

where is the radius of convergence of the series . If is transient then by Proposition 2.2, but Equation (6) contradicts the definition of transient. Thus is recurrent. Moreover, by Lemma 2.4(ii).

The graph is obtained from by deleting the arrow . Obviously one has and

 ∑n≥1fG′uu(n)Ln=1−L<1.

This implies that is transient because . Moreover by Proposition 2.2 thus , that is, .

Now we consider two different sequences .

1) Let for and otherwise. Then and

 ∑n≥1fGuu(n)Ln=∑n≥12n2−n12n2=∑n≥112n=1.

Moreover

 ∑n≥1nfGuu(n)Ln=∑n≥1n22n<+∞,

hence the graph is positive recurrent.

2) Let , for and otherwise. One can compute that , and

 ∑n≥1fGuu(n)Ln=12+∑n≥222n−n122n=12+∑n≥212n=1.

Moreover

 ∑n≥1nfGuu(n)Ln=12+∑n≥22n12n=+∞

hence the graph is null recurrent.

###### Remark 2.10

Let be transient graph of finite entropy. Fix a vertex and choose an integer such that . For every integer let , add loops of length based at the vertex and call the graph obtained in this way. It can be shown that the graph is transient, and . Then Proposition 2.8 implies that every transient graph is included in a null recurrent graph of equal entropy.

###### Remark 2.11

In the more general setting of thermodynamic formalism for countable Markov chains, Sarig puts to the fore a subclass of positive recurrent potentials which he calls strongly positive recurrent [15]; his motivation is different, but the classifications agree. If is a countable oriented graph, a potential is a continuous map and the pressure is the analogous of the Gurevich entropy, the paths being weighted by ; a potential is either transient or null recurrent or positive recurrent. Considering the null potential , we retrieve the case of (non weighted) topological Markov chains. In [15] Sarig introduces a quantity ; is transient (resp. recurrent) if (resp. ). The potential is called strongly positive recurrent if , which implies it is positive recurrent. A strongly positive recurrent potential is stable under perturbation, that is, any potential close to is positive recurrent too. For the null potential, , thus if and only if the graph is strongly positive recurrent (Lemma 2.4 and Theorem 2.7). In [7] strongly positive recurrent potentials are called stable positive.

Examples of (non null) potentials which are positive recurrent but not strongly positive recurrent can be found in [15]; some of them resemble much the Markov chains of Example 2.9, their graphs being composed of loops as in Figure 1.

## 3 Existence of a maximal measure

### 3.1 Positive recurrence and maximal measures

A Markov chain on a finite graph always has a maximal measure [12], but it is not the case for infinite graphs [9]. In [10] Gurevich gives a necessary and sufficient condition for the existence of such a measure.

###### Theorem 3.1 (Gurevich)

Let be a connected oriented graph of finite positive entropy. Then the Markov chain admits a maximal measure if and only if the graph is positive recurrent. Moreover, such a measure is unique if it exists, and it is an ergodic Markov measure.

In [8] Gurevich and Zargaryan show that if one can find a finite connected subgraph such that there are more paths inside than outside (in term of exponential growth), then the graph has a maximal measure. This condition is equivalent to strong positive recurrent as it was shown by Gurevich and Savchenko in the more general setting of weighted graphs [7].

Let be a connected oriented graph, a subset of vertices and two vertices of . Define as the number of paths such that and for all , and put .

###### Theorem 3.2 (Gurevich-Zargaryan)

Let be a connected oriented graph of finite positive entropy. If there exists a finite set of vertices such that is connected and for all vertices in , , then the graph is strongly positive recurrent.

For graphs that are not strongly positive recurrent the entropy is mainly concentrated near infinity in the sense that it is supported by the infinite paths that spend most of the time outside a finite subgraph (Proposition 3.3). This result is obtained by applying inductively the construction of Proposition 2.6(i). As a corollary, there exist “almost maximal measures escaping to infinity” (Corollary 3.4). These two results are proven and used as tools to study interval maps in [4], but they are interesting by themselves, that is why we state them here.

###### Proposition 3.3

Let be a connected oriented graph which is not strongly positive recurrent and a finite set of vertices. Then for all integers there exists a connected subgraph such that and for all , for all , .

###### Corollary 3.4

Let be a connected oriented graph which is not strongly positive recurrent. Then there exists a sequence of ergodic Markov measures such that and for all finite subsets of vertices , .

### 3.2 Local entropy and maximal measures

For a compact system, the local entropy is defined according to a distance but does not depend on it. One may wish to extend this definition to non compact metric spaces although the notion obtained in this way is not canonical.

###### Definition 3.5

Let be a metric space, its distance and let be a continuous map.

The Bowen ball of centre , of radius and of order is defined as

 Bn(x,r)={y∈X∣d(Tix,Tiy)

is a -separated set if

 ∀y,y′∈E,y≠y′,∃0≤k

The maximal cardinality of a -separated set contained in is denoted by .

The local entropy of is defined as , where

 hloc(X,ε)=limδ→0limsupn→+∞1nsupx∈Xlogsn(δ,Bn(x,ε)).

If the space is not compact, these notions depend on the distance. When , we use the distance introduced in Section 1.2. The local entropy of does not depend on the identification of the vertices with .

###### Proposition 3.6

Let be the topological Markov chain on and its compactification as defined in Section 1.2. Then .

Proof. Let , and . By continuity there exists such that, if and then for all . By definition of there is such that , thus , which implies that . Consequently , and . The reverse inequality is obvious.

We are going to prove that, if , then is strongly positive recurrent. First we introduce some notations.

Let be an oriented graph. If is a subset of vertices, a subgraph of and , define

 CH(¯u,V)={(vn)n∈Z∈ΓH∣∀n∈Z,un∈V⇒(vn=un),un∉V⇒vn∉V}.

If and , define

 [S]qp={(vn)n∈Z∈ΓG∣∃(un)n∈Z∈S,∀p≤n≤q,un=vn}.
###### Lemma 3.7

Let be an oriented graph on the set of vertices .

1. If then for all and all ,

2. If and are two paths in such that and for then is -separated.

Proof. (i) Let . If , then for all . Consequently for all

 d(σi(¯u),σi(¯v))=∑k∈ZD(ui+k,vi+k)2|k|≤∑k∈Z2−(p+2)2|k|≤3⋅2−(p+2)<2−p.

(ii) Let such that . By hypothesis, . Suppose that . Then .

###### Theorem 3.8

Let be a connected oriented graph of finite entropy on the set of vertices . If , then the graph is strongly positive recurrent and the Markov chain admits a maximal measure.

Proof. Fix and such that . Let be an integer such that . Let be a finite subgraph such that and let be a finite subset of vertices such that is connected and contains the vertices of and the vertices . Define , and for all .

Our aim is to bound . Choose and let be a path between and with for . Fix . One has .

Fix such that

 ∀δ≤δ0,limsupn→+∞1nsup¯v∈ΓGlogsn(δ,Bn(¯v,