Time Averages of Markov Processes and Applications to Two-Timescale Problems

Time Averages of Markov Processes and Applications to Two-Timescale Problems

Abstract

We show a decomposition into the sum of a martingale and a deterministic quantity for time averages of the solutions to non-autonomous SDEs and for discrete-time Markov processes. In the SDE case the martingale has an explicit representation in terms of the gradient of the associated semigroup or transition operator. We show how the results can be used to obtain quenched Gaussian concentration inequalities for time averages and to provide deeper insights into Averaging principles for two-timescale processes.

\addbibresource

bib.bib \DTMsettimestyleiso

1 Introduction

For a Markov process with or let

in the continuous-time case or

in discrete time.

In the first part of this work, we will show a decomposition of the form

where is a martingale depending on and for which we will give an explicit representation in terms of the transition operator or semigroup associated to .

We then proceed to illustrate how the previous results can be used to obtain Gaussian concentration inequalities for when is the solution to an Itô SDE.

The last part of the work showcases a number of results on two-timescale processes that follow from our martingale representation.

2 Martingale Representation

Consider the following SDE with time-dependent coefficients on :

where is a standard Brownian motion on with filtration and are continuous in and locally Lipschitz continuous in . We assume that does not explode in finite time.

Denote the set of smooth compactly supported space-time functions on .

Let be the evolution operator associated to ,

For fixed consider the martingale

and observe that since is adapted and by the Markov property

with

By applying the Itô formula to we can identify the martingale . This is the content of the following short theorem.

Theorem 2.1.

For fixed, and

with

Proof.

From the Kolmogorov backward equation and since we have

By Itô’s formula

and we are done.

Remark 2.2 (Poisson Equation).

In the time-homogeneous case and when the limit below is finite then it is independent of and we have

This is the resolvent formula for the solution to the Poisson equation with .

By taking in Theorem 2.1 we can identify the martingale part in the martingale representation theorem for .

Corollary 2.3.

For fixed,

By applying the Itô formula to we obtain for fixed

(2.1)

and by integrating from to

This was observed at least as far back as \autociteelliott_integration_1989 and is commonly used in the derivation of probabilistic formulas for .

Combining the formula (2.1) with Theorem 2.1 we obtain the following expression for in terms of .

Corollary 2.4.

For , fixed and any

with

Proof.

Let . We have

where the last equality follows by integrating (2.1) from to (with ). Since and we get from Theorem 2.1 that

and the result follows with . ∎

Remark 2.5 (Carré du Champs and Mixing).

For differentiable functions let

Then we have the following expression for the quadratic variation of :

Furthermore, since

and setting we have

This shows how the expressions we obtain in terms of the gradient of the semigroup relate to mixing properties of .

Remark 2.6 (Pathwise estimates).

We would like to have a similar estimate for

Setting

we have

and

where the last equality follows from (for fixed)

2.1 Discrete time

Consider a discrete-time Markov process with transition operator

and generator

As in the continuous-time setting

is a martingale (by the definition of ) and by direct calculation

Let

and observe that

Note that

It follows that

with

Analogous to the continuous-time case, we define the carré du champs

and using the summation by parts formula

3 Concentration inequalities from exponential gradient bounds

In this section we focus on the case where we have uniform exponential decay of so that

(3.1)

for all and some class of functions .

We first show that exponential gradient decay implies a concentration inequality.

Proposition 3.1.

For fixed and all functions such that (3.1) holds we have

Proof.

By (3.1)

so that .

By Corollary 2.3 and since Novikov’s condition holds trivially due to being bounded by a deterministic function we get

By Chebyshev’s inequality

and the result follows by optimising over . ∎

The corresponding lower bound is obtained by replacing by .

For the rest of this section, suppose that and that we are in the time-homogeneous case so that . An important case where bounds of the form (3.1) hold is when there is exponential contractivity in the Kantorovich (Wasserstein) distance . If for any two probability measures on

(3.2)

then (3.1) holds for all Lipschitz functions with , .

Here the distance between two probability measures and on is defined by

where the infimum runs over all couplings of . We also have the Kantorovich-Rubinstein duality

(3.3)

and we use the notation

We can see that (3.2) implies (3.1) from

where the first inequality is due to the Kantorovich-Rubinstein duality (3.3) and the second is (3.1).

Bounds of the form (3.2) have been obtained using coupling methods in \autociteeberle_reflection_2016,eberle_quantitative_2016,wang_exponential_2016, under the condition that there exist positive constants such that

Similar techniques lead to the corresponding results for kinetic Langevin diffusions\autociteeberle_couplings_2017,.

Using a different approach, in [crisan_pointwise_2016] the authors directly show uniform exponential contractivity of the semigroup gradient for bounded continuous functions, focusing on situations beyond hypoellipticity.

Besides gradient bounds, exponential contractivity in also implies the existence of a stationary measure \autociteeberle_reflection_2016. Proposition 3.1 now leads to a simple proof of a deviation inequality that was obtained in a similar setting in \autocitejoulin_new_2009, via a tensorization argument.

Proposition 3.2.

If (3.2) holds then for all Lipschitz functions and all initial measures

Proof.

We start by applying Proposition 3.1 so that

By the Kantorovich-Rubinstein duality

from which the result follows immediately. ∎

4 Averaging: Two-timescale Ornstein-Uhlenbeck

Consider the following linear multiscale SDE on where the first component is accelerated by a factor :

with independent Brownian motions on . Denote and the associated semigroup and infinitesimal generator respectively.

Let and note that . We have by the regularity of and the Kolmogorov forward equation

so that

Repeating the same reasoning for and gives

From Corollary 2.3

with

This shows that for each fixed

is a Gaussian random variable with mean

and variance

5 Averaging: Exact gradients in the linear case

Consider

Denote the solution for and let . Then

The solution to the linear ODE for is

Since does not depend on we drop it from the notation. Now for any continuously differentiable function on and we obtain the following expression for the gradient of in the direction :

Since we can identify .

The eigenvalues of are with

By observing that

we see that asymptotically as

We can compute the following explicit expression for

with