Time Averages of Markov Processes and Applications to Two-Timescale Problems
We show a decomposition into the sum of a martingale and a deterministic quantity for time averages of the solutions to non-autonomous SDEs and for discrete-time Markov processes. In the SDE case the martingale has an explicit representation in terms of the gradient of the associated semigroup or transition operator. We show how the results can be used to obtain quenched Gaussian concentration inequalities for time averages and to provide deeper insights into Averaging principles for two-timescale processes.
For a Markov process with or let
in the continuous-time case or
in discrete time.
In the first part of this work, we will show a decomposition of the form
where is a martingale depending on and for which we will give an explicit representation in terms of the transition operator or semigroup associated to .
We then proceed to illustrate how the previous results can be used to obtain Gaussian concentration inequalities for when is the solution to an Itô SDE.
The last part of the work showcases a number of results on two-timescale processes that follow from our martingale representation.
2 Martingale Representation
Consider the following SDE with time-dependent coefficients on :
where is a standard Brownian motion on with filtration and are continuous in and locally Lipschitz continuous in . We assume that does not explode in finite time.
Denote the set of smooth compactly supported space-time functions on .
Let be the evolution operator associated to ,
For fixed consider the martingale
and observe that since is adapted and by the Markov property
By applying the Itô formula to we can identify the martingale . This is the content of the following short theorem.
For fixed, and
From the Kolmogorov backward equation and since we have
By Itô’s formula
and we are done.
Remark 2.2 (Poisson Equation).
In the time-homogeneous case and when the limit below is finite then it is independent of and we have
This is the resolvent formula for the solution to the Poisson equation with .
By taking in Theorem 2.1 we can identify the martingale part in the martingale representation theorem for .
By applying the Itô formula to we obtain for fixed
and by integrating from to
This was observed at least as far back as \autociteelliott_integration_1989 and is commonly used in the derivation of probabilistic formulas for .
For , fixed and any
Remark 2.5 (Carré du Champs and Mixing).
For differentiable functions let
Then we have the following expression for the quadratic variation of :
and setting we have
This shows how the expressions we obtain in terms of the gradient of the semigroup relate to mixing properties of .
Remark 2.6 (Pathwise estimates).
We would like to have a similar estimate for
where the last equality follows from (for fixed)
2.1 Discrete time
Consider a discrete-time Markov process with transition operator
As in the continuous-time setting
is a martingale (by the definition of ) and by direct calculation
and observe that
It follows that
Analogous to the continuous-time case, we define the carré du champs
and using the summation by parts formula
3 Concentration inequalities from exponential gradient bounds
In this section we focus on the case where we have uniform exponential decay of so that
for all and some class of functions .
We first show that exponential gradient decay implies a concentration inequality.
For fixed and all functions such that (3.1) holds we have
so that .
By Corollary 2.3 and since Novikov’s condition holds trivially due to being bounded by a deterministic function we get
By Chebyshev’s inequality
and the result follows by optimising over . ∎
The corresponding lower bound is obtained by replacing by .
For the rest of this section, suppose that and that we are in the time-homogeneous case so that . An important case where bounds of the form (3.1) hold is when there is exponential contractivity in the Kantorovich (Wasserstein) distance . If for any two probability measures on
then (3.1) holds for all Lipschitz functions with , .
Here the distance between two probability measures and on is defined by
where the infimum runs over all couplings of . We also have the Kantorovich-Rubinstein duality
and we use the notation
Bounds of the form (3.2) have been obtained using coupling methods in \autociteeberle_reflection_2016,eberle_quantitative_2016,wang_exponential_2016, under the condition that there exist positive constants such that
Similar techniques lead to the corresponding results for kinetic Langevin diffusions\autociteeberle_couplings_2017,.
Using a different approach, in [crisan_pointwise_2016] the authors directly show uniform exponential contractivity of the semigroup gradient for bounded continuous functions, focusing on situations beyond hypoellipticity.
Besides gradient bounds, exponential contractivity in also implies the existence of a stationary measure \autociteeberle_reflection_2016. Proposition 3.1 now leads to a simple proof of a deviation inequality that was obtained in a similar setting in \autocitejoulin_new_2009, via a tensorization argument.
If (3.2) holds then for all Lipschitz functions and all initial measures
We start by applying Proposition 3.1 so that
By the Kantorovich-Rubinstein duality
from which the result follows immediately. ∎
4 Averaging: Two-timescale Ornstein-Uhlenbeck
Consider the following linear multiscale SDE on where the first component is accelerated by a factor :
with independent Brownian motions on . Denote and the associated semigroup and infinitesimal generator respectively.
Let and note that . We have by the regularity of and the Kolmogorov forward equation
Repeating the same reasoning for and gives
From Corollary 2.3
This shows that for each fixed
is a Gaussian random variable with mean
5 Averaging: Exact gradients in the linear case
Denote the solution for and let . Then
The solution to the linear ODE for is
Since does not depend on we drop it from the notation. Now for any continuously differentiable function on and we obtain the following expression for the gradient of in the direction :
Since we can identify .
The eigenvalues of are with
By observing that
we see that asymptotically as
We can compute the following explicit expression for