Hitting times asymptotics for hard-core interactions on grids

Hitting times asymptotics for hard-core interactions on grids

Abstract

We consider the hard-core model with Metropolis transition probabilities on finite grid graphs and investigate the asymptotic behavior of the first hitting time between its two maximum-occupancy configurations in the low-temperature regime. In particular, we show how the order-of-magnitude of this first hitting time depends on the grid sizes and on the boundary conditions by means of a novel combinatorial method. Our analysis also proves the asymptotic exponentiality of the scaled hitting time and yields the mixing time of the process in the low-temperature limit as side-result. In order to derive these results, we extended the model-independent framework in [27] for first hitting times to allow for a more general initial state and target subset.

\DisableLigatures

[f]encoding = * \ListProperties(Progressive*=3ex)

Keywords: hard-core model; hitting times; Metropolis Markov chains; finite grid graphs; mixing times; low temperature.

1 Introduction

Hard-core lattice gas model. In this paper we consider a stochastic model where particles in a finite volume dynamically interact subject to hard-core constraints and study the first hitting times between admissible configurations of this model. This model was introduced in the chemistry and physics literature under the name “hard-core lattice gas model” to describe the behavior of a gas whose particles have non-negligible radii and cannot overlap [21, 36]. We describe the spatial structure in terms of a finite undirected graph of vertices, which represents all the possible sites where particles can reside. The hard-core constraints are represented by edges connecting the pairs of sites that cannot be occupied simultaneously. We say that a particle configuration on is admissible if it does not violate the hard-core constraints, i.e. if it corresponds to an independent set of the graph . The appearance and disappearance of particles on is modeled by means of a single-site update Markov chain with Metropolis transition probabilities, parametrized by the fugacity . At every step a site of is selected uniformly at random; if it is occupied, the particle is removed with probability ; if instead the selected site is vacant, then a particle is created with probability if and only if all the neighboring sites at edge-distance one from are also vacant. Denote by the collection of independent sets of . The Markov chain is ergodic and reversible with respect to the hard-core measure with fugacity on , which is defined as

(1)

where is the appropriate normalizing constant (also called partition function). The fugacity is related to the inverse temperature of the gas by the logarithmic relationship .

We focus on the study of the hard-core model in the low-temperature regime where (or equivalently ), so that the hard-core measure favors maximum-occupancy configurations. In particular, we are interested in how long it takes the Markov chain to “switch” between these maximum-occupancy configurations. Given a target subset of admissible configurations and an initial configuration , this work mainly focuses on the study of the first hitting time of the subset for the Markov chain with initial state at time .

Two more application areas. The hard-core lattice gas model is thus a canonical model of a gas whose particles have a non-negligible size, and the asymptotic hitting times studied in this paper provide insight into the rigid behavior at low temperatures. Apart from applications in statistical physics, our study of the hitting times is of interest for other areas as well. The hard-core model is also intensively studied in the area of operations research in the context of communication networks [23]. In that case, the graph represents a communication network where calls arrive at the vertices according to independent Poisson streams. The durations of the calls are assumed to be independent and exponentially distributed. If upon arrival of a call at a vertex , this vertex and all its neighbors are idle, the call is activated and vertex will be busy for the duration of the call. If instead upon arrival of the call, vertex or at least one of its neighbors is busy, the call is lost, hence rendering hard-core interaction. In recent years, extensions of this communication network model received widespread attention, because of the emergence of wireless networks. A pivotal algorithm termed CSMA [37] which is implemented for distributed resource sharing in wireless networks can be described in terms of a continuous-time version of the Markov chain studied in this paper. Wireless devices form a topology and the hard-core constraints represent the conflicts between simultaneous transmissions due to interference [37]. In this context is therefore called interference graph or conflict graph. The transmission of a data packet is attempted independently by every device after a random back-off time with exponential rate , and, if successful, lasts for an exponentially distributed time with mean . Hence, the regime describes the scenario where the competition for access to the medium becomes fiercer. The asymptotic behavior of the first hitting times between maximum-occupancy configurations provides fundamental insights into the average packet transmission delay and the temporal starvation which may affect some devices of the network, see [39].

A third area in which our results find application is discrete mathematics, and in particular for algorithms designed to find independent sets in graphs. The Markov chain can be regarded as a Monte Carlo algorithm to approximate the partition function or to sample efficiently according to the hard-core measure for large. A crucial quantity to study is then the mixing time of such Markov chains, which quantifies how long it takes the empirical distribution of the process to get close to the stationary distribution . Several papers have already investigated the mixing time of the hard-core model with Glauber dynamics on various graphs [3, 19, 20, 34]. By understanding the asymptotic behavior of the hitting times between maximum-occupancy configurations on as , we can derive results for the mixing time of the Metropolis hard-core dynamics on , which in general is smaller than for the usual Glauber dynamics, as illustrated in [25].

Results for general graphs. The Metropolis dynamics in which we are interested for the hard-core model can be put, after the identification , in the framework of reversible Freidlin-Wentzel Markov chains with Metropolis transition probabilities (see Section 2 for precise definitions). Hitting times for Freidlin-Wentzel Markov chains are central in the mathematical study of metastability. In the literature, several different approaches have been introduced to study the time it takes for a particle system to reach a stable state starting from a metastable configuration. Two approaches have been independently developed based on large deviations techniques: the pathwise approach, first introduced in [6] and then developed in [31, 32, 33], and the approach in [7, 8, 9, 10]. Other approaches to metastability are the potential theoretic approach [4, 5] and, more recently introduced, the martingale approach [1, 2], see [13] for a more detailed review.

In the present paper, we follow the pathwise approach, which has already been used to study many finite-volume models in a low-temperature regime, see [11, 12, 15, 16, 17, 24, 29, 30], where the state space is seen as an energy landscape and the paths which the Markov chain will most likely follow are those with a minimum energy barrier. In [31, 32, 33] the authors derive general results for first hitting times for the transition from metastable to stable states, the critical configurations (or bottlenecks) visited during this transition and the tube of typical paths. In [27] the results on hitting times are obtained with minimal model-dependent knowledge, i.e. find all the metastable states and the minimal energy barrier which separates them from the stable states. We extend the existing framework [27] in order to obtain asymptotic results for the hitting time for any starting state , not necessarily metastable, and any target subset , not necessarily the set of stable configurations. In particular, we identify the two crucial exponents and that appear in the upper and lower bounds in probability for in the low-temperature regime. These two exponents might be hard to derive for a given model and, in general, they are not equal. However, we derive a sufficient condition that guarantees that they coincide and also yields the order-of-magnitude of the first moment of on a logarithmic scale. Furthermore, we give another slightly stronger condition under which the hitting time normalized by its mean converges in distribution to an exponential random variable.

Results for rectangular grid graphs. We apply these model-independent results to the hard-core model on rectangular grid graphs to understand the asymptotic behavior of the hitting time , where and are the two configurations with maximum occupancy, where the particles are arranged in a checkerboard fashion on even and odd sites. Using a novel powerful combinatorial method, we identify the minimum energy barrier between and and prove absence of deep cycles for this model, which allows us to decouple the asymptotics for the hitting time and the study of the critical configurations. In this way, we then obtain sharp bounds in probability for , since the two exponents coincide, and find the order-of-magnitude of on a logarithmic scale, which depends both on the grid dimensions and on the chosen boundary conditions. In addition, our analysis of the energy landscape shows that the scaled hitting time is exponentially distributed in the low-temperature regime and yields the order-of-magnitude of the mixing time of the Markov chain .

By way of contrast, we also briefly look at the hard-core model on complete -partite graphs, which was already studied in continuous time in [38]. While less relevant from a physical standpoint, the corresponding energy landscape is simpler than that for grid graphs and allows for explicit calculations for the hitting times between any pair of configurations. In particular, we show that whenever our two conditions are not satisfied, and the scaled hitting time is not necessarily exponentially distributed.

2 Overview and main results

In this section we introduce the general framework of Metropolis Markov chains and show how the dynamical hard-core model fits in it. We then present our two main results for the hitting time for the hard-core model on grid graphs and outline our proof method.

2.1 Metropolis Markov chains

Let be a finite state space and let be the Hamiltonian, i.e. a non-constant energy function. We consider the family of Markov chains on with Metropolis transition probabilities indexed by a positive parameter

(2)

where is a matrix that does not depend on . The matrix is the connectivity function and we assume it to be

  • Stochastic, i.e.  for every ;

  • Symmetric, i.e.  for every ;

  • Irreducible, i.e. for any , , there exists a finite sequence of states such that , and , for . We will refer to such a sequence as a path from to and we will denote it by .

We call the triplet an energy landscape. The Markov chain is reversible with respect to the Gibbs measure

(3)

Furthermore, it is well-known (see for example [9, Proposition 1.1]) that the Markov chain is aperiodic and irreducible on . Hence is ergodic on with stationary distribution .

For a nonempty subset and a state , we denote by the first hitting time of the subset for the Markov chain with initial state at time , i.e.

Denote by the set of stable states of the energy landscape , that is the set of global minima of on , and by the set of metastable states, which are the local minima of in with maximum stability level (see Section 3 for definition). The first hitting time is often called tunneling time when is a stable state and the target set is some , or transition time from metastable to stable when and .

2.2 The hard-core model

The hard-core model on a finite undirected graph of vertices evolving according to the dynamics described in Section 1 can be put in the framework of Metropolis Markov chains. Indeed, we associate a variable with each site , indicating the absence () or the presence () of a particle in that site. Then the hard-core dynamics correspond to the Metropolis Markov chain determined by the energy landscape where

  • The state space is the set of admissible configurations on , i.e. the configurations such that for every pair of neighboring sites in ;

  • The energy of a configuration is ;

  • The connectivity function allows only for single-site updates (possibly void), i.e. for any ,

For the hard-core measure (1) on is precisely the Gibbs measure (3) associated with the energy landscape .

Our main focus in the present paper concerns the dynamics of the hard-core model on finite two-dimensional rectangular lattices, to which we will simply refer to as grid graphs. More precisely, given two integers , we will take to be a grid graph with three possible boundary conditions: Toroidal (periodic), cylindrical (semiperiodic) and open. We denote them respectively by , and . Figure 1 shows an example of the three possible types of boundary conditions.

(a) Open grid
(b) Cylindrical grid
(c) Toroidal grid
Figure 1: Examples of grid graphs with different boundary conditions

There are in total sites in . Every site is described by its coordinates , and since is finite, we assume without loss of generality that the leftmost (respectively bottommost) site of has the horizontal (respectively vertical) coordinate equal to zero. A site is called even (odd) if the sum of its two coordinates is even (odd, respectively) and we denote by and the collection of even sites and that of odd sites of , respectively.

The open grid is naturally a bipartite graph: All the neighbors in of an even site are odd sites and vice versa. In contrast, the cylindrical and toroidal grids may not be bipartite, so that we further assume that is an even integer for the cylindrical grid and that both and are even integers for the toroidal grid . Since the bipartite structure is crucial for our methodology, we will tacitly work under these assumptions for the cylindrical and toroidal grids in the rest of the paper. As a consequence, and are balanced bipartite graphs, i.e. . The open grid has even sites and odd sites, hence it is a balanced bipartite graphs if and only if the product is even. We denote by ( respectively) the configuration with a particle at each site in ( respectively). More precisely,

Note that and are admissible configurations for any choice of boundary conditions, and that and . In the special case where with , and, as we will show in Section 5, and . In all the other cases, we have and ; see Section 5 for details.

2.3 Main results and proof outline

Our first main result describes the asymptotic behavior of the tunneling time for any rectangular grid in the low-temperature regime . In particular, we prove the existence and find the value of an exponent that gives an asymptotic control in probability of on a logarithmic scale as and characterizes the asymptotic order-of-magnitude of the mean tunneling time . We further show that the tunneling time normalized by its mean converges in distribution to an exponential unit mean random variable.

Theorem 2.1 (Asymptotic behavior of the tunneling time ).

Consider the Metropolis Markov chain corresponding to hard-core dynamics on a grid as described in Subsection 2.2. There exists a constant such that

    [align=left]
  • For every

In the special case where with , (i), (ii), and (iii) hold also for the first hitting time , but replacing by .

Theorem 2.1 relies on the analysis of the hard-core energy landscape for grid graphs and novel results for hitting times in the general Metropolis Markov chains context. We first explain these new model-independent results and, afterwards, we give details about the properties we proved for the energy landscape of the hard-core model.

The framework [27] focuses on the most classical metastability problem, which is the characterization of the transition time between a metastable state and the set of stable states . However, the starting configuration for the hitting times we are interested in, is not always a metastable state and the target set is not always . In fact, the classical results can be applied for the hard-core model on grids for the hitting time only in the case of an grid with open boundary conditions and odd side lengths, i.e. . Many other interesting hitting times are not covered by the literature, including:

  • The hitting time when is a grid with open boundary conditions and odd side lengths, i.e. , which is a transition from the unique stable state to the metastable state ;

  • The hitting times when is an grid with for any boundary conditions, since the configurations and are both stable states;

  • The hitting time between any pair of local minima when is a complete -partite graph.

We therefore generalize the classical pathwise approach [27] to study the first hitting time for a Metropolis Markov chain for any pair of starting state and target subset . The interest of extending these results to the tunneling time between two stable states was already mentioned in [27, 33], but our framework is even more general and we could study for any pair , e.g. the transition between a stable state and a metastable one.

Our analysis relies on the classical notion of cycle, which is a maximal connected subset of states lying below a given energy level. The exit time from a cycle in the low-temperature regime is well-known in the literature [9, 10, 13, 31, 33] and is characterized by the depth of the cycle, which is the minimum energy barrier that separates the bottom of the cycle from its external boundary. The usual strategy presented in the literature to study the first hitting time from to is to look at the decomposition into maximal cycles of the relevant part of the energy landscape, i.e. . The first model-dependent property one has to prove is that the starting state is metastable, which guarantees that there are no cycles in deeper than the maximal cycle containing the starting state , denoted by . In this scenario, the time spent in maximal cycles different from , and hence the time it takes to reach from the boundary of , is comparable to or negligible with respect to the exit time from , making the exit time from and the first hitting time of the same order.

In contrast, for a general starting state and target subset all the maximal cycles of can potentially have a non-negligible impact on the transition from to in the low-temperature regime. By analyzing these maximal cycles and the possible cycle-paths, we can establish bounds in probability of the hitting time on a logarithmic scale, i.e. obtain a pair of exponents such that for every

The sharpness of the exponents and crucially depends on how precisely one can determine which maximal cycles are likely to be visited and which ones are not, see Section 3 for further details. Furthermore, we give a sufficient condition (see Assumption A in Section 3), which is the absence of deep typical cycles, which guarantees that , proving that the random variables converge in probability to as , and that . In many cases of interest, one could show that Assumption A holds for the pair without detailed knowledge of the typical paths from to . Indeed, by proving that the model exhibits absence of deep cycles (see Proposition 3.18), similarly to [27], also in our framework the study of the hitting time is decoupled from an exact control of the typical paths from to . More precisely, one can obtain asymptotic results for the hitting time in probability, in expectation and in distribution without the detailed knowledge of the critical configuration or of the tube of typical paths. Proving the absence of deep cycles when and corresponds precisely to identifying the set of metastable states , while, when and , it is enough to show that the energy barrier that separates any state from a state with lower energy is not bigger than the energy barrier separating any two stable states.

Moreover, we give another sufficient condition (see Assumption B in Section 3), called “worst initial state” assumption, to show that the hitting time normalized by its mean converges in distribution to an exponential unit mean random variable. However, checking Assumption B for a specific model can be very involved, and hence we provide a stronger condition (see Proposition 3.20), which includes the case of the tunneling time between stable states and the classical transition time from a metastable to a stable state. The hard-core model on complete -partite graphs is used as an example to illustrate scenarios where Assumption A or B is violated, and the asymptotic result for of the first moment and the asymptotic exponentiality of do not hold.

In the case of the hard-core model on a rectangular grid , we develop a powerful combinatorial approach which shows the absence of deep cycles (Assumption A) for this model, concluding the proof of Theorem 2.1. Furthermore, it yields the value of the energy barrier between and , which turns out to depend both on the grid size and on the chosen boundary conditions. This is illustrated by the next theorem, which is our second main result.

Theorem 2.2 (The exponent for rectangular grids).

Let a rectangular grid. Then the energy barrier between and appearing in Theorem 2.1 takes the values

The crucial idea behind the proof of Theorem 2.2 is that along the transition from to , there must be a critical configuration where for the first time an entire row or an entire column coincides with the target configuration . In such a critical configuration particles reside both in even and odd sites and, due to the hard-core constraints, an interface of empty sites should separate particles with different parities. By quantifying the “inefficiency” of this critical configuration we get the minimum energy barrier that has to be overcome for the transition from to to occur. The proof is then concluded by exhibiting a path that achieves this minimum energy and by exploiting the absence of other deep cycles in the energy landscape.

Lastly, we show that by understanding the global structure of an energy landscape and the maximum depths of its cycles, we can also derive results for the mixing time of the corresponding Metropolis Markov chains , as illustrated in Subsection 3.8. In particular, our results show that in the special case of an energy landscape with multiple stable states and without other deep cycles, the hitting time between any two stable states and the mixing time of the chain are of the same order-of-magnitude in the low-temperature regime. This is the case also for the Metropolis hard-core dynamics on grids, see Theorem 5.4 in Section 5.

The rest of the paper is structured as follows. Section 3 is devoted to the model-independent results valid for a general Metropolis Markov chain, which extend the classical framework [27]. The proofs of these results are rather technical and therefore deferred to Section 4. In Section 5 we develop our combinatorial approach to analyze the energy landscapes corresponding to the hard-core model on grids. We finally present in Section 6 our conclusions and indicate future research directions.

3 Asymptotic behavior of hitting times for Metropolis Markov chains

In this section we present model-independent results valid for any Markov chains with Metropolis transition probabilities (2) defined in Subsection 2.1. In Subsection 3.1 we introduce the classical notion of a cycle. If the considered model allows only for a very rough energy landscape analysis, well-known results for cycles are shown to readily yield upper and lower bounds in probability for the hitting time : indeed, one can use the depth of the initial cycle as (see Propositions 3.4) and the maximum depth of a cycle in the partition of as (see Proposition 3.7). If one has a good handle on the model-specific optimal paths from to , i.e. those paths along which the maximum energy is precisely the min-max energy barrier between and , sharper exponents can be obtained, as illustrated in Proposition 3.10, by focusing on the relevant cycle, where the process started in spends most of its time before hitting the subset . We even further sharpen these bounds in probability for the hitting time with Proposition 3.15 by studying the tube of typical paths from to or standard cascade, a task that in general requires a very detailed but local analysis of the energy landscape. To complete the study of the hitting time in the regime , we prove in Subsection 3.5 the convergence of the first moment of the hitting time on a logarithmic scale under suitable assumptions (see Theorem 3.17) and give in Subsection 3.6 sufficient conditions for the scaled hitting time to converge in distribution as to an exponential unit mean random variable, see Theorem 3.19. Furthermore, we illustrate in detail two special cases which fall within our framework, namely the classical transition from a metastable state to a stable state and the tunneling between two stable states, which is the relevant one for the model considered in this paper. In Subsection 3.7 we briefly present the hard-core model on a complete -partite graph, which is an example of a model where the asymptotic exponentiality of the scaled hitting times does not always hold. Lastly, in Subsection 3.8 we present some results for the mixing time and the spectral gap of Metropolis Markov chains and show how they are linked with the critical depths of the energy landscape.

In the rest of this section and in Section 4, will denote a general Metropolis Markov chain with energy landscape and inverse temperature , as defined in Subsection 2.1.

3.1 Cycles: Definitions and classical results

We recall here the definition of cycle and present some well-known properties.

Recall that a path has been defined in Subsection 2.1 as a finite sequence of states such that , and , for . Given a path in , we denote by its length and define its height or elevation by

(4)

A subset with at least two elements is connected if for all there exists a path , such that for every . Given a nonempty subset and , we define as the collection of all paths for some that do not visit before hitting , i.e.

(5)

We remark that only the endpoint of each path in belongs to . The communication energy between a pair is the minimum value that has to be reached by the energy in every path , i.e.

(6)

Given two nonempty disjoint subsets , we define the communication energy between and by

(7)

Given a nonempty set , we define its external boundary by

For a nonempty set we define its bottom as the set of all minima of the energy function on , i.e.

Let be the set of stable states, i.e. the set of states with minimum energy. Since is finite, the set is always nonempty. Define the stability level of a state by

(8)

where is the set of states with energy lower than . We set if is empty, i.e. when is a stable state. The set of metastable states is defined as

(9)

We call a nonempty subset a cycle if it is either a singleton or it is a connected set such that

(10)

A cycle for which condition (10) holds is called non-trivial cycle. If is a non-trivial cycle, we define its depth as

(11)

Any singleton for which condition (10) does not hold is called trivial cycle. We set the depth of a trivial cycle to be equal to zero, i.e. . Given a cycle , we will refer to the set of minima on its boundary as its principal boundary. Note that

In this way, we have the following alternative expression for the depth of a cycle , which has the advantage of being valid also for trivial cycles:

(12)

The next lemma gives an equivalent characterization of a cycle.

Lemma 3.1.

A nonempty subset is a cycle if and only if it is either a singleton or it is connected and satisfies

The proof easily follows from definitions (6), (7) and (10) and the fact that if is not a singleton and is connected, then

(13)

We remark that the equivalent characterization of a cycle given in Lemma 3.1 is the “correct definition” of a cycle in the case where the transition probabilities are not necessarily Metropolis but satisfy the more general Friedlin-Wentzell condition

(14)

where is an appropriate rate function . The Metropolis transition probabilities correspond to the case (see [14] for more details) where

The next theorem collects well-known results for the asymptotic behavior of the exit time from a cycle as becomes large, where the depth of the cycle plays a crucial role.

Theorem 3.2 (Properties of the exit time from a cycle).

Consider a non-trivial cycle .

  1. For any and for any , there exists such that for all sufficiently large

  2. For any and for any , there exists such that for all sufficiently large

  3. For any , there exists such that for all sufficiently large

  4. There exists such that for all sufficiently large

  5. For any , and , for all sufficiently large

  6. For any , any and all sufficiently large

The first three properties can be found in [33, Theorem 6.23], the fourth one is [33, Corollary 6.25] and the fifth one in [27, Theorem 2.17]. The sixth property is given in [31, Proposition 3.9] and implies that

(15)

The third property states that, given that is a cycle, for any starting state , the Markov chain visits any state before exiting from with a probability exponentially close to one. This is a crucial property of the cycles and in fact can be given as alternative definition, see for instance [9, 10]. The equivalence of the two definitions has been proved in [14] in greater generality for a Markov chain satisfying the Friedlin-Wentzell condition (14). Leveraging this fact, many properties and results from [9] will be used or cited.

We denote by the set of cycles of . The next lemma, see [33, Proposition 6.8], implies that the set has a tree structure with respect to the inclusion relation, where is the root and the singletons are the leafs.

Lemma 3.3 (Cycle tree structure).

Two cycles are either disjoint or comparable for the inclusion relation, i.e.  or .

Lemma 3.3 also implies that the set of cycles to which a state belongs is totally ordered by inclusion. Furthermore, we remark that if two cycles are such that , then ; this latter inequality is strict if and only if the inclusion is strict.

3.2 Classical bounds in probability for hitting time 

In this subsection we start investigating the first hitting time . Thus, we will tacitly assume that the target set is a nonempty subset of and the initial state belongs to . Moreover, without loss of generality, we will henceforth assume that

(16)

which means that we add to the original target subset all the states in that cannot be reached from without visiting the subset . Note that this assumption does not change the distribution of the first hitting time , since the states which we may have added in this way could not have been visited without hitting the original subset first.

Given a nonempty subset and , we define the initial cycle by

(17)

If , then and thus is a trivial cycle. If , the subset is either a trivial cycle (when ) or a non-trivial cycle containing , if . In any case, if , then . For every , we denote by the depth of the initial cycle , i.e.

Clearly if is trivial (and in particular when ), then . Note that by definition the quantity is always non-negative, and in general

with equality if and only if .

If , then the initial cycle is, by construction, the maximal cycle (in the sense of inclusion) that contains the state and has an empty intersection with . Therefore any path has at some point to exit from , by overcoming an energy barrier not smaller than its depth . The next proposition gives a probabilistic bound for the hitting time by looking precisely at this initial ascent up until the boundary of .

Proposition 3.4 (Initial-ascent bound).

Consider a nonempty subset and . For any there exists such that for sufficiently large

(18)

The proof is essentially adopted from [33] and follows easily from Theorem 3.2(i), since by definition of , we have that .

Before stating an upper bound for the tail probability of the hitting time , we need some further definitions. Given a nonempty subset , we denote by the collection of maximal cycles that partitions , i.e.

(19)

Lemma 3.3 implies that every nonempty subset has a partition into maximal cycles and hence guarantees that is well defined. Note that if is itself a cycle, then . The importance of the notion of initial cycle besides Proposition 3.4 is partially explained by the next lemma.

Lemma 3.5.

[27, Lemma 2.26] Given a nonempty subset , the collection of initial cycles is the partition into maximal cycles of , i.e.

We can extend the notion of depth to subsets which are not necessarily cycles by using the partition of into maximal cycles. More precisely, we define the maximum depth of a nonempty subset as the maximum depth of a cycle contained in , i.e.

(20)

Trivially if . The next lemma gives two equivalent characterizations of the maximum depth of a nonempty subset .

Lemma 3.6 (Equivalent characterizations of the maximum depth).

Given a nonempty subset ,

(21)

In view of Lemma 3.6, is the maximum initial energy barrier that the process started inside possibly has to overcome to exit from . As illustrated by the next proposition, one can get a (super-)exponentially small upper bound for the tail probability of the hitting time , by looking at the maximum depth of the complementary set , where the process resides before hitting the target subset .

Proposition 3.7 (Deepest-cycle bound).

[9, Proposition 4.19] Consider a nonempty subset and . For any there exists such that for sufficiently large

(22)

By definition we have , but in general and neither bound presented in this subsection is actually tight, so we will proceed to establish sharper but more involved bounds in the next subsection.

3.3 Optimal paths and refined bounds in probability for hitting time 

The quantity appearing in Proposition 3.4 only accounts for the energy barrier that has to be overcome starting from , but there is such an energy barrier for every state and it may well be that to reach it is inevitable to visit a state with . Similarly, also the exponent appearing in Proposition 3.7 may not be sharp in general. For instance, the maximum depth could be determined by a deep cycle in that cannot be visited before hitting or that is visited with a vanishing probability as . In this subsection, we refine the bounds given in Propositions 3.4 and 3.7 by using the notion of optimal path and identifying the subset of the state space in which these optimal paths lie.

Given a nonempty subset and , define the set of optimal paths as the collection of all paths along which the maximum energy is equal to the communication height between and , i.e.

(23)

Define the relevant cycle as the minimal cycle in such that , i.e.

(24)

The cycle is well defined, since the cycles in that contain are totally ordered by inclusion, as remarked after Lemma 3.3. By construction, and thus contains at least two states, so it has to be a non-trivial cycle. The minimality of with respect to the inclusion gives that

and then, by using Lemma 3.1, one obtains

(25)

The choice of the name relevant cycle for comes from the fact that all paths the Markov chain will follow to go from to will almost surely not exit from in the limit . Indeed, for the relevant cycle Theorem 3.2(iii) reads

(26)

The next lemma, which is proved in Section 4, states that an optimal path from to is precisely a path from to that does not exit from .

Lemma 3.8 (Optimal path characterization).

Consider a nonempty subset and . Then

Lemma 3.8 implies that the relevant cycle can be equivalently defined as

(27)

where is the minimum energy gap between an optimal and a non-optimal path from to , i.e.

In view of Lemma 3.8 and (26), the Markov chain started in follows in the limit almost surely an optimal path in to hit . It is then natural to define the following quantities for a nonempty subset and :

(28)

and

(29)

Definition (28) implies that every optimal path has to enter at some point a cycle in of depth at least , while definition (29) means that every cycle visited by any optimal path has depth less than or equal to .

An equivalent characterization for the energy barrier can be given, but we first need one further definition. Define as the subset of states which belong to at least one optimal path in , i.e.

(30)

Note that , since the endpoint of each path in belongs to , by definition (5). In view of Lemma 3.8, . We remark that this latter inclusion could be strict, since in general . Indeed, there could exist a state such that all paths that do not exit from always visit the target set before reaching , and thus they do not belong to (see definitions (5) and (23)), see Figure 2.

(a) The subset (in light gray)
(b) The partition into maximal cycles of , including the initial cycle (in dark gray)
Figure 2: Example of an energy landscape with highlighted the subset (in black), the relevant cycle and the subset (with diagonal mesh)

The next lemma characterizes the quantity as the maximum depth of the subset (see definition (20)).

Lemma 3.9 (Equivalent characterization of ).
(31)

Using the two quantities and , we can better control in probability the hitting time , as stated in the next proposition, which is proved in Section 4.

Proposition 3.10 (Optimal paths depth bounds).

Consider a nonempty subset and . For any there exists such that for sufficiently large

(32)

and