Once reinforced random walk on
Abstract
We revisit Vervoort’s unpublished paper [16] on the once reinforced random walk, and prove that this process is recurrent on any graph of the form , with a finite graph, for sufficiently large reinforcement parameter.
We also obtain a shape theorem for the set of visited sites, and show that the fluctuations around this shape are of polynomial order. The proof involves sharp general estimates on the
time spent on subgraphs of the ambiant graph which might be of independent interest.
Keywords and phrases. Recurrence, Reinforced random walk, selfinteracting random walk, shape theorem.
MSC 2010 subject classifications: 60K35.
1 Introduction
1.1 General overview
The oncereinforced random walk (ORRW) belongs to the large class of selfinteracting random walks, whose future evolution depends on its past history. The study of these processes is usually difficult and basic properties such as recurrence and transience are hard to obtain. One of the most famous example of selfinteracting random walks is the linearly edgereinforced random walk introduced by Coppersmith and Diaconis [4] in the eighties, for which recurrence and transience were only recently proved in a series of papers, see [1, 12, 14, 6].
Even though the definition of ORRW, introduced in 1990 by Davis [5], is simple, it turns out that its study does not seem easier than that of the linearly reinforced RW, and results on graphs with loops are very rare. In this model, the current weight of an edge is if it has never been crossed and otherwise, with . It has been conjectured by Sidoravicius that ORRW is recurrent on for and undergoes a phase transition for , being recurrent when the parameter is large and transient when it is small. These questions on , , are completely open. Until recently, it was not even clear whether this weak reinforcement procedure could indeed change the nature of the walk, so that ORRW could be recurrent on a graph which is transient for simple random walk, as soon as the parameter is large enough. The first example of such phase transition was provided in [9] on a particular class of trees with polynomial growth, which is in contrast with the result of Durrett, Kesten and Limic [8] who showed that the ORRW is transient on regular trees for any (later generalized to any supercritical tree by Collevecchio [2]). More recently, the complete picture on trees has been given in [3]: the critical parameter of ORRW on a locally finite tree is equal to its branchingruin number, which is defined in [3] as a polynomial regime of the branching number (see [10]).
As already mentioned, results on graphs with loops are very rare. Sellke [15] first investigated the case of the ladder , with , and proved that the ORRW is almost surely recurrent on this graph for all . The proof is a simple consequence of a general (and nice) martingale argument, but it does not really face the difficulty of possible drift pushing the walk toward infinity, which can in principle happen in the presence of loops in the graph. In an unpublished paper, Vervoort [16] announced a more difficult result, namely that the ORRW is recurrent on the ladder for all large enough reinforcement parameter . Unfortunately, his proof was never published and the preprint [16] is unpolished, with gaps and mistakes. The general strategy of [16] was to show that the mean drift of the walk, each time it exits its present range, is almost balanced. The reason being that for large enough , all possible exit edges are equally likely to be chosen. Thus, at least at first order, there should be equal probability to get a drift to the right (when the exit edge is an horizontal edge oriented to the left) as an opposite drift (when this edge is oriented to the right). However, an important ingredient, which was missing in [16], is to show first that the ORRW cannot travel a large distance before exiting its present range. Indeed the first order approximation of uniformity for the choice of the exit edge is only valid when the edges taken into consideration are not too far one from each other. One difficulty then is to obtain an estimate, which is uniform over all the possible ranges (or finite subgraphs of ). We prove such general result here, which might be of independent interest, with the help of electrical network techniques. Details can be found in our Proposition 2.6 below.
Furthermore, we improve the lower bound on , and obtain a polynomial bound in the height of the ladder, instead of an exponential one, which was implicit in [16]. For this purpose, one needs to adapt the notion of walls from [16], to ensure their existence with a probability , instead of an exponentially small (in the size of ) one. We also analyze the fluctuations of the range of the walk and provide a shape theorem. Finally we show that the successive return times to the origin have finite expectation.
1.2 Model and results
Let us define a nearestneighbor random walk as a ORRW on a (nonempty, locally finite and undirected) graph , with reinforcement parameter . First, the current weight of an edge is defined as follows: at time , an edge has conductance if it has never been crossed (regardless of any orientation of the edges) and conductance otherwise. For any , let be the set of nonoriented edges crossed up to time , that is
(1.1) 
At time , if , then the walk jumps to a neighbor of with conditional probability
(1.2) 
where is the natural filtration generated by the walk, i.e. .
Our first result is the following:
Theorem 1.1.
There exists a constant , such that for any finite connected graph , the oncereinforced random walk on is recurrent for any reinforcement parameter .
Note that here by recurrent we mean that almost surely every site is visited infinitely often.
Our second result is a shape theorem. Denote by the graph whose vertex set is , the set of visited sites up to time , and whose edges are those crossed by the walk up to this time. Let be the first time when the number of vertices in this graph equals .
Theorem 1.2.
There exists a constant , such that for any finite connected graph and any , the following holds: almost surely for all large enough, there exists , such that
where inclusions here are meant as inclusion of graphs.
Remark 1.3.
We do not expect that the center of the cluster could be taken to be zero. Indeed for the ORRW on (i.e. when is reduced to a single vertex), explicit computations show that, for any , infinitely often and infinitely often.
Remark 1.4.
Concerning the exponent , it is far from being optimal. Our proof would allow to replace the constant by any other constant smaller than , at the cost of imposing larger . But we do not believe that this would be optimal neither. In fact we expect that the correct order of the fluctuations is precisely in , with the asymptotical mean drift per level: , where equals times the number of edges between level and which are crossed for the first time from left to right minus the number of those edges crossed for the first time in the other direction. But here the main issue would be to show that the above limit actually exists. On the other hand, our proofs in this paper show that if the limit indeed exists, then it is larger than , up to lower order terms (and in particular it is positive for large ). We also suspect that should be equivalent to , for some constant , when goes to infinity.
1.3 Organization of the paper
The paper is organized as follows. In Section 2, we prove general estimates on random walks on networks where conductances take only two values: on a finite subgraph of and one elsewhere. Our main results there are estimates, which are uniform on , on the time spent on certain level sets, that is subsets of the form . This section can be read independently of the rest of the paper, and might be interesting on its own. Then in Section 3, we define a notion of wall, that extends the one from Vervoort’s paper [16]. The interest of this new definition is to obtain bounds of polynomial order (in the size of ) in all our results. In section 4 we gather the results proved so far to obtain gambler’s ruin type estimates. These estimates are then used in Section 5 to prove Theorems 1.1, and 1.2, and in Section 6 to prove that the successive return times to the origin have finite expectation.
2 Random walks on subgraphs of .
This section gathers some results concerning (reversible) random walks on subgraphs of where is a finite graph. In particular, we study the position where such a walk exits a given subgraph. As such, the section does not deal specifically with oncereinforced random walk but the results obtained here will play a crucial role during the proof of the main theorems. We also believe that some estimates such as Proposition 2.9 may be found of independent interest.
2.1 Notation
A graph is a collection of vertices and edges . By a small abuse of notation, we shall sometimes identify a graph and its set of vertices when the associated edge set is unambiguous. An undirected edge between two vertices and is denoted by , while a directed edge is denoted by . We write when and we say in this case that and are neighbors. All the graphs considered here are assumed to be nonempty and locally finite, meaning that all vertices have only finitely many neighbors. If is a directed edge, we call the tail and also denote it by , and the head and denote it by . We write for the set of directed edges of .
Given two vertices and , we denote by their graph distance in . For a subgraph , we denote by the induced (also called intrinsic) distance, i.e. the minimal number of edges needed to be crossed to go from to while staying inside . In particular, we have = .
Given a subgraph , we define
(2.1)  
(2.2) 
In words, is the set of directed edges of which do not belong to as an undirected edge but whose tail belongs to . Notice that the head of a directed edge may, or may not, be in . The set is the set of tails of the edges in , or equivalently the set of vertices adjacent to an edge outside .
In this paper, we consider cylinder graphs of the form . In this case, if , we will denote by and the respective projections on and so that
Finally, for , we denote by the subgraph of isomorphic to with edge set consisting of all edges with both endpoints in . We call this subgraph the level set .
2.2 Reversible RW and electrical networks
We recall here some standard results on random walks and electrical networks which we will use repeatedly in the paper. We refer the reader to [7] and [11] for a comprehensive and thorough presentation of the theory.
A network is a graph , endowed with a map . The value of an edge is called its weight or conductance, and its reciprocal is called its resistance. A random walk on a network is the Markov chain which moves only to neighbors of its current position, choosing it with a probability proportional to the weight of the edge traversed. We denote the law of the chain starting from by . This process is reversible with respect to the measure defined by . Given a subset of vertices , a map is said to be harmonic on if it satisfies:
Given a vertex and a subset , a voltage is a function which is harmonic outside , and vanishes on . Given a voltage function, we define the associated current function on the oriented edges by
(Ohm’s law) 
Then, is a flow from to , which means an antisymmetric function on the set of oriented edges such that
(Kirchoff’s node’s law) 
The strength of the flow is defined by . We say that we have a unit current flowing from to when , and one defines similarly a unit flow.
Given a random walk , the hitting time of a set of vertices is defined by
whereas the first return time is defined by
To simplify notations, we just write (resp ) when . Similarly, when is of the form , we also use the notation for the hitting time of the level set .
The effective conductance between a vertex and a subset is defined by the formula
(2.3) 
Its reciprocal is called the effective resistance between and and is denoted by . It follows from the maximum principle that there exists a unique unit current flowing from to . The corresponding voltage is the unique function that is harmonic outside of , vanishes on , and satisfies .
We recall three important results which we will need in later sections.
Proposition 2.1 (Current as edge crossings, Prop. 2.2 in [11]).
Let be a finite connected network. Consider the random walk starting at some vertex and let be a subset of vertices not containing . For , let be the number of crossings of the directed edge by the walk before it hits . We have , where is the unit current flowing from to .
As a consequence of this proposition (c.f. Exercice 2.37 of [11]), if is a unit current from to , then necessarily
(2.4) 
Given a flow on an electrical network, the energy dissipated by the flow is defined by
The following result characterizes the current among all flows on a network.
Proposition 2.2 (Thomson’s principle, p. 35 in [11]).
The unit current has minimal energy among all unit flows:
We say that a flow has a cycle if there exist oriented edges with and and for all . It follows from Thomson’s principle that a current cannot have a cycle because we could otherwise decrease its energy by removing from it a small flow with support on the cycle. Another immediate consequence of Thomson’s principle is that the effective conductance/resistance is a monotone function of the conductances on the edges.
Proposition 2.3 (Rayleigh’s Monotonicity Principle, p. 36 in [11]).
Let be a finite connected graph with two conductances assignments, and such that . Let be a vertex and a subset of vertices not containing . We have .
We end this section with the remarkable CommuteTime Identity, which relates the hitting times between two points of a graph and the effective resistance between these two points.
Proposition 2.4 (CommuteTime Identity, Corollary 2.21 in [11]).
Let be a finite connected network and let and be two vertices of . The commute time between and is
2.3 The exit edge for a random walk on a subgraph of
By definition, the oncereinforced random walk behaves as a usual random walk as long as it stays inside its trace. More precisely, assume that at some time , the ORRW has crossed exactly all the edges of a subgraph (in particular ). Then, from time and until it exits the subgraph , the ORRW behaves as the random walk on the electrical network with conductances given by
(2.5) 
In particular, when is large, the probability to choose a nonreinforced edge is small. Thus, informally, one can visualize the walk as “bumping” on the boundary of its trace many times before exiting, and so it should “mix” a little more than the usual random walk. This remark leads to a key idea which originates from Vervoort [16]: when is large, the distribution of the exit edge gets close (locally, in some sense) to the uniform measure on the boundary .
In this subsection, we give two results in this direction that concern the distribution of the exit edge. They are stated in term of the random walk on the electrical network (2.5) but they translate readily to the ORRW as explained above. The first result states that two edges on the boundary which are not too far away have approximately the same probability to be chosen as the next exit edge.
Proposition 2.5.
Let be a finite connected subgraph of a graph . Fix and consider the electrical network on with conductances . Let denotes a random walk on this electrical network and define as the first time the walk exits the subgraph :
(2.6) 
For any , and for any , we have
(2.7) 
Proof.
Consider the finite connected graph whose vertex set is , with an additional cemetery vertex, and whose edge set consists of all the original edges in plus, for each , one additional edge between and . The edges inside are assigned weight whereas the edges adjacent to are assigned unit weight. Note that this construction may create multiple edges between some vertex in and .
Let with tails and . By construction, the law of up to time , matches the law of the random walk on the network , up to the hitting time of . Thus, according to Proposition 2.1 and using Ohm’s law, for any , we have
where v is the voltage at when a unit current flows from to . By definition there exists a path of length inside going from to and composed of edges with conductance . Applying Ohm’s law along this path and using (2.4), we find that
and the result follows. ∎
The proposition above is fairly general since it does not make any assumption on the graph (it need not be of cylinder type). However, as time increases, so does the size of the boundary of the trace of the walk. Thus, without additional information on the distribution of the exit probabilities, the bound (2.7) applied to the ORRW becomes mostly useless when the number of possible exit edges becomes much larger than .
In order to keep (2.7) relevant, we need to control the number of exit edges which have a non negligible probability of being chosen and show that they are . This estimate which is missing from Vervoort’s paper is the purpose of the next proposition. Unlike Proposition 2.5, it is specific to cylinder graphs.
Proposition 2.6.
Let be a finite connected subgraph of where is a finite connected graph. Fix and consider the random walk on the electrical network with conductances . Fix and suppose that there exist integers and such that

.

For each , there exist with (there is an exit edge at each level).
Recall that defined by (2.6) denotes the first time the walk exits the subgraph and that denotes the first time it reaches level . We have
(2.8) 
The proposition above tells us that the random walk on the network (2.5) cannot travel too far away horizontally without exiting its trace. More precisely, the number of incomplete levels it can cross before exiting is (at most) of order . This means that, for the ORRW, only the exit edges belonging to the nearest incomplete levels have to be taken into account. But then, there are no more than such exit edges (because the underlying graph is a cylinder). Thus, we are now in the case where we can use Proposition 2.5 to control the exit probabilities.
Let us also remark that the ratio in (2.8) is not surprising because the walk on any subgraph of is diffusive. Thus, we can expect that it spends a time of order inside a slice of diameter . On the other hand, each time the walk visits a site on the boundary of , it has a probability proportional to to exit it at the next step. This heuristic is simple but making it rigorous is challenging because the upper bound (2.8) needs to hold uniformly on all possible subgraph . The proof we present here is rather convoluted and will by carried out at the end of the next subsection.
2.4 Proof of Proposition 2.6
In this section, we need to consider random walks on different graphs. In order to distinguish between these processes, we will use superscript that refer to the underlying graph. For instance, given a subgraph of , probabilities relating to a random walk on will be denoted by whereas we will keep the usual notation for a random walk on the whole graph .
In everything that follows, denotes a finite connected graph. We start with a simple lemma which bounds the return time to a given vertex on the same level as the starting position for the simple random walk on .
Lemma 2.7.
Consider the simple random walk on . Let such that . Recall that denotes the hitting time of vertex . We have
Recall that, for a subgraph , , defined in (2.2), denotes the set of vertices of which are adjacent to an edge outside . Since we can couple the simple random walk on and the simple random walk restricted on so that they coincide until they reach a vertex of , the previous lemma directly entails
Corollary 2.8.
Let be a subgraph of . Consider the simple random walk on , i.e. on the network with conductances . For any such that contains at least one vertex at level , we have
Proof of Lemma 2.7.
Without loss of generality, we can assume that . Fix and consider the graph on which we put unit conductances (i.e. we are considering the SRW on the graph). Noticing that the number of oriented edges in is bounded by , Proposition 2.4 (the commutetime identity) shows that
Furthermore, the effective resistance between and is bounded by the graph distance between those two vertices. This may be checked, for instance, from Rayleigh’s monotonicity principle (Proposition 2.3) by putting null conductances everywhere except on a geodesic path between the two vertices. Since and are on the same level, we deduce that . Set . Using Markov’s inequality, we find that
On the other hand, if denotes a simple random on starting from , an application of the reflection principle shows that
Thus, the previous inequality shows that the probability that the simple random walk on starting from hits level or before time is at most . But, until this happens, the random walks on and on coincide so we conclude that
∎
Proposition 2.9.
Let be a finite connected subgraph of . Fix , and . Let and be integers such that
Consider the simple random walk on started from . Recall that denotes the hitting time of the level set at . This stopping time is a.s. finite since is finite and has a vertex at level . Let be the total time spent on the levels of before reaching level :
We have
Proof.
We construct a modified electrical network in the following way. First, we put unit conductances on each edge of . Then, we glue all vertices of at the final height together and call the resulting vertex . We also fix and create a new vertex connected by an edge of conductance to every vertices such that (c.f. figure 1 for an illustration). Consider now a unit current flowing from to . Let
be the total current flowing into the sink vertex . For , let also
denotes the total current flowing from level to . From a probabilistic point of view, is the probability that the walk on the electrical network started from hits before hitting . Similarly, is the probability that the walk hits before while exiting through one of the edges added at level . We now show that
(2.9) 
To do so, we will need the following lemma which provides a decomposition of a flow without cycle on arbitrary graphs.
Lemma 2.10.
Consider a finite connected graph A and fix three vertices . Let be a flow from to such that

For all , we have (source) and and (sinks).

the flow does not have any cycle.
Then, there exist a flow on from to such that

for any . Therefore, is a flow of total strength .

for all (nothing flows in ).

For any , (the flows and have the same direction).

For any , .

For any , .
(here and below, we use the convention that, for any flow, if and are not neighbors).
We postpone the proof of the lemma to finish that of the proposition. Fix and . Since the current fulfills assumptions 1. and 2. of the lemma, we can consider the flow as above and use it to create a new unit flow from to where some of the original current flowing into is diverted toward by going through the edges of conductance added at level . More precisely, we set, for ,
and
It is clear that satisfies the flow property. In words, the flow coincides with for levels below or equal to and coincides with for levels above . In order to maintain the flow node’s law, the missing flow going through the cutset of horizontal edges between levels and is rerouted through the edges at level that link to . Let (resp. ) denotes the energy dissipated by (resp. ). We estimate
(2.10)  
Here the factor corresponds to the resistance of the added edges. Thanks to properties (c) and (d) of the lemma, each term in the first sum is nonpositive so we can upper bound this sum by keeping only the terms corresponding to edges of the form where for some :
Now, since the flow has strength and there are at most edges in the cutset of edges linking level to , there must exist some with such that . Thus, we deduce that
(2.11) 
On the other hand, using the fact that and property (e), the second term in (2.10) can be upper bounded by:
(2.12)  
According to Thomson’s principle, the unit current has minimal energy among all unit flows hence for all . Combining (2.10), (2.11) and (2.12), we conclude that
which finally yields (2.9) by letting tend to .
The remaining of the proof is rather straightforward. First, we sum (2.9) for . Since is a unit current, we find that
and therefore, recalling the probabilistic interpretation of , we have proved that
(2.13) 
Let us now consider the natural coupling of the random walks (resp. ) starting from on the electrical networks (resp. ) such that both walks coincide until hits . More precisely, we construct both walks by first tossing a (biaised) coin at each step to decide whether exits by an edge of conductance when the coin gives a ”head” (and such an edge exist) and otherwise move the two walks together. Recall that is the total time spent over the vertices at levels belonging to before hitting level (i.e. hitting ). Furthermore, the vertices corresponding to levels in are, by construction, the vertices that share an edge of conductance with . Thus, we have
(2.14) 
Note that, each time is on a vertex that has an edge of conductance , there is a probability at most that the associated coin returns ”head” (because there is also at least one adjacent edge with unit conductance). Thus, we get
(2.15) 
Combining (2.13), (2.14) and (2.15) and choosing , we conclude that
which completes the proof of the proposition. ∎
Proof of Lemma 2.10.
First, let us notice that we can discard all the edges on which , keeping only the connected component of that contains . This is because will also be zero on edges where is zero. We now assume that is nonzero on all edges of . Then, the flow induces an oriented graph structure on so we can speak of “outgoing” and “incoming” edges from a vertex. We are going to construct starting from and going backward with respect to the graph orientation. At each step of the exploration process, we keep track of a partition of the vertices into inactive, active and completed vertices where

An active vertex has the flow defined on some outgoing edges but on no incoming edge.

An inactive vertex is such that the flow is not yet defined on any adjacent edge.

A completed vertex is such that the flow is already defined on all its adjacent edges.
To begin, we fix on all edges adjacent to and on all edges adjacent to . This is possible because there is no edge (with nonzero flow) between and thanks to assumption 1. Hence satisfies (a) and (b). Now, we set and while all other vertices are inactive. We show that, at each step, we can transform an active vertex into a complete one (possibly turning inactive vertices into active ones in the process) while constructing a flow which, restricted to the set of completed vertices, still satisfies all the required conditions of the lemma.
Indeed, suppose that we have performed some steps of our exploration process and have our sets of active, inactive and completed vertices. We claim that there must exist an active vertex such that is already defined on all of its outgoing edges. Indeed, if this was not the case, we could start from any active vertex and then recursively construct a path that follows the graph orientation and on which is not defined. But then, such a path is either infinite or contains cycles. As is finite, this path has cycles, which contradicts the initial assumption that has no cycle. So, let be such a vertex. The flow already defined on the subgraph spanned by the completed vertices has strength . In particular, the sum of on the outgoing edges of is at most . It is also smaller, by construction, than the sum of on the incoming edges of . Thus, it is now clear that we can fix on the incoming edges of in such way that (c), (d), (e) hold true. There are several ways to do it. For instance, we can set for any incoming edge where is the ratio of the total outgoing flow over the total incoming flow. Finally, we move to the set of completed vertices and activate all its adjacent currently inactive vertices. This complete the induction step and the proof of the lemma. ∎
Remark 2.11.
Under the hypotheses of Lemma 2.10, the function is also a flow which satisfies all the properties (a)  (e) when exchanging the roles of and . In particular, any flow that satisfies the hypotheses of the lemma can be written as the superposition of two flows and such that

.

is a flow from to and no flow enters nor exits .

is a flow from to and no flow enters nor exits .

The flows , and have the same sign on all edges.
As already noticed during the proof of the lemma, this decomposition is not, in general, unique. In particular, when is a current, the flows and need not be currents themselves.
Proposition 2.9 shows that the time spent on any distinct level sets is of order (at least) which is the correct diffusive scaling for the random walk on a subgraph of but it provides only a polynomially decaying upper bound. However, it is not difficult to bootstrap the previous result to get an exponential upper bound which is still homogeneous in .
Corollary 2.12.
Under the assumptions of Proposition 2.9, we have, for any ,
Proof.
Let and . We split the set in groups, each containing at least consecutive levels. Thus, in order for the local time to be smaller than , it has to be smaller than this value on each group. Making use of the Markov property at the time of first entrance in each group and applying repeatedly Proposition 2.9, we find that
∎
We can now complete the proof of Proposition 2.6.
Proof of Proposition 2.6.
Set which is the constant appearing in Corollary 2.8. Let and define by induction the sequence of stopping time by , and
with the usual convention that . Let be the event that the walk does not cross any edge of during the time interval :
We can couple the random walk on the electrical network (2.5) with the simple random walk on the subgraph up to time (the time when the walk on the electrical network leaves ). Thus, we deduce that, for any fixed ,
(2.16)  
On the one hand, before time , the simple random walk on cannot visit levels of more than times. Therefore, according to Corollary 2.12, we have
(2.17) 
On the other hand. Each time the walk on the electrical network (2.5) visits a site of , there is, at least one adjacent exit edge so it has probability at least to cross an edge of at the next step. Combining this fact with Corollary 2.8 and using the strong Markov property, we deduce that,
which implies,
(2.18) 
Thus, combining (2.16), (2.17), (2.18), we find that
(2.19) 
Finally, setting with and recalling the exact value of , we conclude that