Delay embedding of periodic orbits using a fixed observation function

# Delay embedding of periodic orbits using a fixed observation function

Raymundo Navarrete and Divakar Viswanath
###### Abstract

Delay coordinates are a widely used technique to pass from observations of a dynamical system to a representation of the dynamical system as an embedding in Euclidean space. Current proofs show that delay coordinates of a given dynamical system result in embeddings generically with respect to the observation function (Sauer, Yorke, Casdagli, J. Stat. Phys., vol. 65 (1991), p. 579-616). Motivated by applications of the embedding theory, we consider flow along a single periodic orbit where the observation function is fixed but the dynamics is perturbed. For an observation function that is fixed (as a nonzero linear combination of coordinates) and for the special case of periodic solutions, we prove that delay coordinates result in an embedding generically over the space of vector fields in the topology with .

Department of Mathematics, University of Michigan (raymundo/divakar@umich.edu).

## 1 Introduction

Suppose a physical system is described by the differential equation , where . Often the state vector is unobservable in its entirety, and that is especially true if is large. Thus, reconstructing the flow from observations is not straightforward. The technique of delay coordinates makes it possible to look at a single scalar observation and reconstruct the dynamics. We denote the scalar that is observed by . The observation function could be a projection to a single coordinate, for example, when the velocity of a fluid flow is recorded at a single point and in a single direction. It could be some other linear function of . More generally, the observation function could be nonlinear.

If is the time- flow map, the idea behind delay coordinates [18, 23, 25] is to use the delay vector

 ξ(x;τ,n)=(πx,πϕ−τ(x),…,πϕ−(n−1)τ(x)),

which is observable, as a surrogate for the point in phase space. For a suitable choice of delay and embedding dimension , delay coordinates yield a faithful representation of the phase space in a sense we will explain. Delay coordinates have been employed in many applications [2, 24]. Current theory for delay coordinates [23] applies perturbations to the observation function . We consider the situation where the observation function is fixed as a linear projection and only the dynamical system is perturbed.

Packard et al [18] demonstrated that coordinate vectors such as give good representations of strange attractors. They noted that delay coordinate vectors would be equivalent to coordinate vectors formed using derivatives of the observed quantity.

A mathematical analysis of delay coordinates was undertaken in a famous paper by Takens [25] and independently by Aeyels [1]. In particular, Takens considered when is an embedding. Suppose is a manifold of dimension , a submanifold of of dimension , and a continuous map from to the manifold . The restriction is an embedding of in if the tangent map has full rank at every point of , is injective, and maps open sets in to open sets in its range in the subspace topology [9, 13]. For the definition to make sense, the manifolds and must be at least . More generally, the manifolds and the map may be assumed to be with or with . Takens concluded that delay coordinates yield an embedding of compact manifolds without boundary if , for generic observation functions and generic vector fields . A property is generic in the topology if it holds for functions or belonging to a countable intersection of open and dense sets [19]. Because the spaces are Baire spaces [13], a countable intersection of open and dense sets is dense as well as uncountable.

The paper by Sauer et al [23] marked a major advance in the theory of delay coordinates. The approach to embedding theorems outlined by Takens relied on parametric transversality. Parametric transversality arguments typically have a local part and a global part, and the transition from local arguments to a global theorem is made using partitions of unity [13].

Sauer et al [23] sidestepped transversality theory almost entirely. Unlike in transversality theory, there is no explicitly local part in the arguments of Sauer et al [23]. The local part of the argument comes down to a verification of Lipshitz continuity. The set being embedded is only assumed to have finite box counting dimension. The arguments are mostly probabilistic and the globalization step relies only on the finiteness of the box counting dimension. The only real analogy to differential topology appears to be to the proof of Sard’s theorem [13], which too is proved using probabilistic arguments. Sauer et al prove prevalence [14], which goes beyond genericity. A property is prevalent with respect to the observation function , if the property holds when any given is replaced by , with being monomials indexed by the finite set , for almost every choice of the coefficients .

The embedding theorem of Sauer et al [23] fixes the dynamical system and allows only the observation function to be perturbed. The statements of genericity and prevalence are with regard to , not the original dynamical system. If consideration is restricted to subsets of box counting dimension , Sauer et al only require . Thus, we could even have .

As mentioned, we investigate embedding theorems in which the observation function is fixed. For example, could be fixed as a linear projection that extracts some component of the state vector. We allow perturbations of the dynamical system only.

The motivation for considering such embedding theorems is as follows. First, on purely aesthetic grounds, it appears desirable to have an embedding theory that depends upon the dynamics and not the observation function. Second, in many applications the observation function is fixed, whereas the dynamical system itself is parametrized [2, 4, 8, 20, 24]. If extracts a single component at a single point in the velocity field of a fluid, it is more pertinent to make the embedding theory depend upon the dynamics rather than upon the observation function.

Aeyels [1] stated that delay coordinates are injective for generic flows and a fixed observation function. In the context of applications, stronger theorems would be desirable as argued by Sauer et al [23]. First, an open and dense set can have arbitrarily small measure implying that prevalence, which is stronger than genericity, is a more appropriate concept. Second, the dynamics may be confined to an attractor of dimension much smaller than that of the state vector of the flow. In such a situation, we would like the embedding dimension to be determined by the dimension of the attractor and not the dimension of the state vector of the flow.

In this article, we consider the second of these two directions. Obtaining an embedding dimension that depends on the dimension of the attractor and not the flow introduces new difficulties when the observation is fixed and the flow is parametrized. Current proofs [23, 25] rely on perturbing the observation function to produce an embedding. When the observation function is fixed, the additional step of propagating perturbations to the flow to the observed delay coordinates will need to be handled. We need to understand how perturbing the flow perturbs the invariant set or attractor, which is assumed to persist, and how the perturbations to the invariant set or attractor propagate to delay coordinates. When the flow is fixed and the observation function is perturbed, the attractor to be embedded, which depends only upon the flow, is unchanged by the perturbations. In contrast, when the observation function is fixed and the flow is perturbed, the set to be embedded is altered by the perturbations.

To get a handle on such difficulties, we limit ourselves to hyperbolic periodic orbits and prove that they embed generically in . The techniques we use are those of transversality theory. Although periodic orbits are only a special case, they are an important special case and arise frequently in applications, for example [3, 7].

To conclude this introduction, we mention some other extensions of delay coordinate embedding theory. Embedding theory has been considered for endomorphisms [26] as well as delay differential equations [5], for continuous but not necessarily smooth observation functions [10, 11], and in concert with Kalman filtering [12]. The concept of determining modes and points in fluid mechanics and PDE is related to embedding theory [17, 20, 21]. Delay coordinates have been used for noise reduction [22, 27]. The embedding theory of Sauer et al [23] has been generalized to PDE by Robinson [20, 21]. The current embedding theory for PDE also relies on perturbing the observation function.

## 2 Embedding periodic signals in R3

In the next section, we consider periodic solutions of differential equations. In this section, we begin by considering periodic signals. A periodic signal is any function with a period . Figure 2.1 shows a periodic signal and its delay embedding in .

To make the definition of periodic signals more precise, let be the set of functions with period . Periodicity requires derivatives of to match at and . The domain of functions in , which we will write as for signals of period , is compact and homeomorphic to . More precisely, the domain is the identification space obtained by identifying and in . For convenience, we shall refer to it as , with the understanding that when we refer to an interval it can wrap around. The elements of will be referred to as periodic signals. Even if is constant, it must be equipped with a period , and if is chosen differently, we get a different element of .

For the periodic signal shown in Figure 2.1, the map for results in an embedding of the circle. Each point of the circle maps to a distinct point in so that the delay map is injective. The delay is also immersive because a small movement along the periodic signal maps to a small and nonzero movement in the embedding space . Because the delay map is both injective and immersive, it is an embedding.

Figure 2.2 shows a situation in which the delay map is not injective. This example is in fact the same as in Figure 2.1 but the period is taken to be double of what it is in Figure 2.1. As a result, points which are separated by the fundamental period map to the same point in . As shown in Figure 2.2, the signal may be modified so that the fundamental interval is not repeated and the delay map still fails to be injective. Later in this section, we will prove that signals whose delay maps embed the circle in are more typical.

### 2.1 Local argument for periodic signals

If and are two periodic signals, define

 dr(o,o′)=supk=0,…rsup0≤s<1|o(k)(sT)−o′(k)(sT′)|+|T−T′|. (2.1)

The topology on is defined by this metric. The norm of a periodic signal is By our definition, is not a vector space because signals with different periods cannot be added. However, signals of a fixed period are a vector space and is a norm over it.The topology is the union of topologies over as explained in [13]. For concepts and results of differentiable topology, such as critical points, regular values, and Sard’s theorem, our main reference is Hirsch [13]. The same topics are discussed from a dynamical point of view in [15, 19].

Figure 2.2 shows a signal which does not embed the circle in under delay mapping. However, it is clear from observation that points that are nearby such as and map to distinct points in . In fact, quite generally, if the number of critical points in is finite, nearby points in the signal will map to distinct points in , as we later prove. We begin by considering whether any periodic signal may be perturbed slightly so that it has only finitely many critical points.

###### Lemma 1.

Let , , be a periodic signal of period . If is a regular value of , then the periodic signal has finitely many critical points in .

###### Proof.

Suppose at infinitely many points on the compact circle . Let be an accumulation point of the set of zeros. Then and implying that is not a regular value of . ∎

The following lemma generates a periodic signal of period whose derivative is everywhere except over a given interval . Any function whose derivative is , , everywhere cannot be periodic. Therefore, the proof of the lemma comes down to modifying the derivative carefully in the interval .

###### Lemma 2.

Given and , for all sufficiently small there exists an infinitely differentiable periodic signal of period such that for and for . In addition, for , as .

###### Proof.

Let be an infinitely differentiable bump function with for , for , and for and . If then . The bump function is used to modify in the interval .

Define for and more generally

 do(t)dt=ϵ−kλ((t−α)/(β−α))

for . The idea behind the construction is shown in Figure 2.3: if the bump function is shifted to the interval and a suitable multiple is subtracted, may then be integrated to obtain a periodic function.

More precisely, it follows that The integral is zero if . For small, is small as well. We may obtain by integrating , with proportional to . ∎

The following lemma proves that any sufficiently smooth periodic signal can be perturbed to a nearby periodic signal with finitely many critical points.

###### Lemma 3.

If , , is a periodic signal, there exists another periodic signal of the same period with arbitrarily small and such that has only finitely many critical points (including local maxima and minima) and is a regular value of .

###### Proof.

If is constant we can perturb to for arbitrarily small and verify the theorem. We will assume that is not constant.

Consider as a map from the circle to . If is a regular value of this map, we are done by Lemma 1.

If not, there exists a regular value of arbitrarily close to by Sard’s theorem (here is needed). Suppose we look at . This function has a regular value at . However, the corresponding perturbation of is and is not periodic.

Because is not constant, there exists an interval in the circle over which is nonzero. Without loss of generality, we assume in the interval (consider for the case where the derivative is negative). Using Lemma 2, we may find a periodic signal such that for and for . Set to obtain a periodic signal with being a regular value of to complete the proof.∎

###### Remark.

Lemma 1 is evidently true if we only assume the second derivative of the periodic signal to exist and not necessarily continuous. In fact, Lemma 3 is also true under the same weaker assumption because, in one dimension, Sard’s theorem requires only the existence of the derivative (see Exercise 1 of Section 3.1 of [13]).

The proof of Lemma 3 may be illustrated using Figure 2.4. The figure shows a part of the graph of and its derivative . It is evident that the critical points of , where , accumulate at the origin. In fact, a small perturbation cannot eliminate the accumulation of critical points because does not have a second derivative at . However, if , a function whose second derivative looks like the derivative show in Figure 2.4, Sard’s theorem may be used to obtain a small perturbation such that is a regular value of the derivative of the perturbed function.

If is a periodic signal with finitely many critical points, then its circular domain may be decomposed into finitely many intervals with local minima and maxima at either end. Let denote the minimum width among such intervals. Because is monotonic in each interval, we refer to each such interval as the minimum interval of strict monotonicity. If the delay is , we denote the point by .

###### Lemma 4.

If , where is the minimum interval of strict monotonicity, and if the delay satisfies , then . If is a regular value of , we also have for all .

###### Proof.

Because , and lie in either the same interval of strict monotonicity of the periodic signal or in neighboring intervals. If they lie in the same interval, we must have either or proving the lemma.

If and lie in neighboring intervals, we may assume without loss of generality. If , there is nothing to prove. So we assume in addition. Again without loss of generality, we assume that first increases and then decreases as increases from to .

With these assumptions, and must lie in the same interval of monotonicity because , and therefore . Further and the unique minimum of for is attained when . Therefore , and we once again have .

For the claim about , we note that cannot equal zero at both and , because . ∎

With Lemma 4, the local argument for embedding periodic signals is partly complete. Globalizing the argument will involve additional perturbations, which we now define.

Let be a bump function with for , for , and for all . Let and . Define

 λj(t)=λ(t−jhh)

for and . We interpret modulo and regard as a periodic signal with the circular domain : a pulse of period and width centered at which is equal to for . We now consider the perturbation

 oϵ(t)=o(t)+ϵ0λ0(t)+ϵ1λ1(t)+⋯+ϵnλn(t), (2.2)

where . For any , there exists a bump function with such that and therefore if .

Before we turn to the global argument, we must prove that the local structure asserted by Lemma 4 is preserved when is perturbed to as in (2.2). The lemma below guarantees for . The bound ensures that can happen only when the intervals and do not overlap.

###### Lemma 5.

Let , , be a periodic signal defined over the domain and with minimum interval of strict monotonicity equal to . Assume that is a regular value of . There exists such that if , then for the perturbation defined by (2.2) and delay satisfying , we have for all with . In addition, remains a regular value of .

###### Proof.

By assumption the periodic signal has finitely many critical points. Let be the critical points in the circular interval ; at these points and only at these, we have . Since is a regular value of , we have for .

In the circle , choose compact intervals , , such that and for any . By continuity in the perturbing parameters , for sufficiently small the perturbed periodic signal (2.2) also has nonzero second derivative on .

Define the interval to be ( wraps around the circle). Each is an interval of strict monotonicity. By compactness, attains a minimum strictly greater than over . Again by continuity, any perturbation of the form (2.2) with sufficiently small also has nonzero derivative over .

Thus, for sufficiently small, remain intervals of strict monotonicity for the perturbed periodic signal, and each can contain at most one critical point of the perturbed periodic signal. The minimum interval of strict monotonicity is at least . We now apply Lemma 4 to infer that implies for . We limit to the interval to complete the proof. ∎

### 2.2 Global argument for periodic signals

The global argument relies on the parametric transversality theorem [13, 19].

###### Lemma 6.

Let , , be a periodic signal defined over the circle . There exists an arbitrarily small perturbation of the periodic signal to , with the same period, and a delay such that is an embedding, with a regular value of .

###### Proof.

By Lemma 3, we may make an initial perturbation to if necessary and assume that has finitely many critical points, that is a regular value of , and that is the minimum width of an interval of strict monotonicity.

Now consider perturbations of to of the form (2.2). By Lemma 5, we may assume for and for , provided is sufficiently small.

Consider the set

 T={(t1,t2)∣∣|t1−t2|>3τ,t1∈[0,T),t2∈[0,T)},

where is interpreted as the circle, as before. For the applicability of the parametric transversality theorem later in the proof, it is important to note that is a manifold of dimension without a boundary.

Consider as a function from the domain to . We will now verify that this function is transverse to the diagonal in . If there is nothing to prove. Suppose and consider the point in given by

 (oϵ(t1),oϵ(t1−τ),oϵ(t1−2τ),oϵ(t2),oϵ(t2−τ),oϵ(t2−2τ))

The intervals and are disjoint because . By construction, there exist such that are each equal to at exactly one of the six points and zero at the others. If the tangent direction in the domain is taken to perturb for , it maps to a perturbation of the -th coordinate in , more precisely the elementary vector . Therefore, the tangent map is surjective and transversality is verified.

By the parametric transversality theorem [Hirsch, Chapter 3, Theorem 2.7], we may choose arbitrarily small such that considered as a function from to is transverse to the diagonal of . Since is of dimension , that can only happen if for .

To complete the proof, we only need to check the smoothness/dimension condition in the parametric transversality theorem. The dimension of is and the codimension of the diagonal in is . Thus, it is sufficient if the map from to is which it is.∎

###### Lemma 7.

Let , , be a periodic signal such that is an embedding of the circle in for delay . There exists such that and (perturbation has same period) imply that is also an embedding of the circle .

###### Proof.

By the inverse function theorem (see [13, Appendix]), there exists such that for every there exists a neighborhood of over which is an injection if and . Using a Lebesgue- argument we may assume that for , making smaller if necessary.

Although arguments like the one above are common in differential topology, we state the version of the inverse function theorem invoked for clarity. The version used is as follows. Suppose is a map from , an open subset of to , an open subset of with . Suppose and that the tangent map is injective at . Then there exists a neighborhood of in the weak topology (), a neighborhood of , of , and of , such that for every there exists a diffeomorphism with restricted to coinciding with . This theorem is applied with and .

The rest of the proof is a standard compactness argument. Let

 min|t1−t2|≥ϵ0|o(t1;τ)−o(t2;τ)|=δ>0,

where the minimum exists because of compactness and is greater than because is an embedding. By continuity, the minimum must be positive for sufficiently close to . Similarly, immersivity of sufficiently close to is a direct consequence of compactness of the circle. Thus, is also an embedding.∎

###### Theorem 8.

The set of periodic signals , , for which there exists a delay such that is an embedding of the circle in is open and dense in .

###### Proof.

By Lemma 6, there exists an arbitrarily small perturbation to such that is an embedding for and with a regular value of . Thus the set of periodic signals with a delay embedding and with a regular value of is dense. We only have to prove that the set is open.

Given periodic signal with an embedding, Lemma 7 shows that remains an embedding for sufficiently small if . If , we may still apply Lemma 7, by defining which is a periodic signal of period . If ,then . Finally, is an embedding implies that is an embedding with . ∎

###### Theorem 9.

Suppose that , , and that is an embedding of the circle for some delay . Then remains an embedding if is close enough to .

###### Proof.

The arguments used in Lemma 7 and Theorem 8 apply with little change. ∎

## 3 Embedding periodic orbits in R3

Figure 3.1 shows a periodic orbit of the classical Lorenz system given by , , .111The periodic orbit of Figure 3.1 in [28] could not be computed using the techniques of [28]. It was computed some years later using an initial guess that was constructed from the periodic orbit . The signal extracted from that orbit is nearly flat for a significant duration when the origin is approached.

In this section, we will prove that “typical” periodic orbits (in a sense that will be made precise) yield signals that result in embeddings of the circle. The following proposition proves that an embedding using delay coordinates persists when the vector field is perturbed slightly. It is the easier half of the argument.

###### Proposition 10.

Let , where , , and an open subset of , be a dynamical system with a vector field, . Let be a hyperbolic periodic solution of period . Let and . Assume that be an embedding of the circle in . There exists an open neighborhood of in the topology such that for each in that neighborhood, there exists a -close hyperbolic periodic solution of period of and a close to such that is an embedding of the circle in .

###### Proof.

The fact that a hyperbolic periodic solution such as perturbs to a nearby hyperbolic solution in a small enough open neighborhood of is a standard result [20, Chapter 5]. If the signal is such that is an embedding of the circle, then is also an embedding for by Theorem 8. The proof of Theorem 8 uses the choice . ∎

Suppose that the delay map of a signal obtained by projecting the first component of a periodic orbit does not embed in . We will show that the differential equation , , can be perturbed ever so slightly such that a nearby periodic orbit of the perturbed equation results in an embedding of the circle. The proof relies on constructing a tube around the periodic orbit. A tube around a periodic orbit is illustrated in Figure 3.2.

To construct a tube around any periodic orbit in , we begin by defining in analogy to . Let be the set of periodic orbits that are times continuously differentiable. As before, we assume that is a parametrization of and for the period. As a part of the definition of , we require for . The set is endowed with a topology by defining the metric in analogy with (2.1):

The norm over is the -norm. The th derivative of is denoted by . For convenience, and are also denoted as and , respectively. The tangent vector at is defined as .

We denote the projection from to the first coordinate by . If is a solution of the dynamical system , we wish to show that either is such that is an embedding of the circle for some delay , or that there exists an arbitrarily close perturbed dynamical system with a nearby periodic orbit such that is an embedding of the circle, if .

To begin with, the signal may even be identically zero. In our proof, we use the results of the previous section to perturb it to such that is an embedding and then show how to perturb the flow to realize as .

The next lemma constructs a tube around the periodic orbit in (see Figure 3.2). That tube will be used to perturb to . Known results in differential geometry [6, 16] may be used to assert the existence of a tube. However, uniformity and smoothness guarantees that we need could not be found in the literature. Therefore, an elementary proof of the lemma is included. The proof will later be modified to deduce the existence of a tube whose radius is uniform in a neighborhood of . In the following lemma, may be thought of as the radius of a tube around .

###### Lemma 11.

Suppose , , and that its period is . Then there exists such that

• for

• if and , there exists a unique such that

###### Proof.

The proof is organized so as to be easy to uniformize in the next lemma.

1. Choice of and . Let and . The first part of the lemma would be satisfied if , or if .

2. Choice of and . First, we introduce the notation

 dpdt∣∣∣[t1,t2]

for a vector each of whose components is the corresponding component of evaluate at some . Crucially, each component may chose a different . This notation will facilitate application of the mean value theorem. The interval may wrap around , in which case the interval width must be taken to be and not . We ignore such wrap-arounds from this point onwards.
Suppose and . Then

The factor here arises in converting a componentwise bound using the -norm to a bound on the -norm. Evidently, if we choose and , we may assert that

 ∣∣∣∣∣∣˙p(tm)−dpdt∣∣∣[t1,t2]∣∣∣∣∣∣≤m (3.1)

for and .
If is the unit tangent vector to at , we have

 s(tm)⋅(p(t2)−p(t1) =s(tm)⋅(dpdt∣∣[t1,t2](t2−t1)) =s(tm)⋅˙p(tm)(t2−t1)+s(tm)⋅(dpdt∣∣[t1,t2]−˙p(tm))(t2−t1),

where the first equality is obtained by applying the mean value theorem to each component of . Now, by choice of . By (3.1), the second term in the display above is at most in magnitude. Therefore,

 |s(tm)⋅(p(t2)−p(t1)|≥m(t2−t1)

for and .

3. Choice of . Suppose is a vector orthogonal to and with as before. Then, we have , which implies

 |s(tm)⋅w1| ≤∣∣∣∣s(tm)−s(t1)∣∣∣∣∣∣∣∣w1∣∣∣∣

where the factor arises in converting a componentwise bound to a bound on the -norm. An explicit formula for the time derivative of the unit tangent, will be given in the next proof. If we choose , we may replicate the argument given using with and assert

4. Choice of . We define . Because a periodic orbit cannot self-intersect, we must have .

We will choose to be smaller than the least of

The first part of the lemma follows immediately. Now suppose and Suppose is equal to as well as for . By item 4 above, we must have , which we will now assume.

Because minimizes , we may differentiate and deduce . Equivalently . Thus, we may write with orthogonal to the tangent and . Likewise, we may write with orthogonal to the tangent and .

From , we obtain

Taking absolute values, applying item 2 above to the left hand side, and item 3 above to the right hand side, we get

 m(t2−t1)