Koopman Operator Spectrum and Data Analysis

Koopman Operator Spectrum and Data Analysis

Igor Mezić Mechanical Engineering, University of California Santa Barbara, Santa Barbara, CA 93106
July 5, 2019
Abstract

We examine spectral operator-theoretic properties of linear and nonlinear dynamical systems with equilibrium and quasi-periodic attractors and use such properties to characterize a class of datasets and introduce a new notion of the principal dimension of the data. Using the Kato Decomposition we develop a spectral expansion for general linear autonomous dynamical systems with analytic observables, and define the notion of generalized eigenfunctions of the associated Koopman operator. We interpret stable, unstable and center subspaces in terms of zero level sets of generalized eigenfunctions. We then utilize conjugacy properties of Koopman eigenfunctions and the new notion of open eigenfunctions - defined on subsets of state space - to extend these results to nonlinear dynamical systems with an equilibrium. We provide a characterization of (global) center manifolds, center-stable and center-unstable manifolds in terms of joint zero level sets of families of Koopman operator eigenfunctions associated with the nonlinear system. We also develop spectral expansions for a class of dynamical systems possessing globally stable limit cycles and limit tori, with observables that are square-integrable in on-attractor variables and analytic in off-attractor variables. We discuss definitions of stable, unstable and global center manifolds in such nonlinear systems with (quasi)-periodic attractors in terms of zero level sets of Koopman operator eigenfunctions. We define the notion of isostables for a general class of nonlinear systems. In contrast with the systems that have discrete Koopman operator spectrum, we provide a simple example of a measure-preserving system that is not chaotic but has continuous spectrum, and discuss experimental observations of spectrum on such systems. We also provide a characterization of data based on the obtained theoretical results and define the principal dimension for a class of datasets based on the lattice-type principal spectrum of the associated Koopman operator.

1 Introduction

Spectral theory of dynamical systems shifts the focus of investigation of dynamical systems behavior away from trajectories in the state space and towards spectral objects - eigenvalues, eigenfunctions and eigenmodes - of an associated linear operator. Specific examples are the Perron-Frobenius operator [1] and the composition operator - in measure-preserving setting called the Koopman operator [2, 3]. In this paper we study spectral properties of the composition operator for a class of dynamical systems and relate them to state space and data analyses.

In classical dynamical systems theory, the notion of conjugacy is an important one. For example, conjugacy is the setting in which linearization theorems, such as the Hartman-Grobman theorem, are proved. In the original investigations using the operator-theoretic approach to measure-preserving dynamical systems, the notion of conjugacy also played an important role [4]. One of the most important questions in that era was whether spectral equivalence of the Koopman operator spectra implies conjugacy of the associated dynamical systems. It was settled in the negative by von Neumann and Kolmogorov [5], where the examples given had complex - mixed or continuous - spectra. The transformation of spectral properties under conjugacy, pointed out in [6], was already used in the data-analysis context in [7]. Here we explore the relationship between the spectrum of the composition operator and conjugacy, for dissipative systems, and discuss the type of spectrum they exhibit for asymptotic behavior ranging from equilibria to quasi-periodicity. The approach, inspired by ideas in [8] extends the analysis in that paper to provide spectral expansions and treat the case of saddle point equilibria using the newly defined concept of open eigenfunctions of the Koopman operator on subsets of state space. While these systems have discrete spectrum, we also present a simple example of a measure-preserving (non-dissipative) system with non-chaotic dynamics with continuous spectrum.

In dissipative systems, the composition operator is typically non-normal, and can have generalized eigenfunctions. Gaspard and collaborators studied spectral expansions for dynamical systems containing equilibria and exhibiting pitchfork and Hopf bifurcations [9, 10]. The author presented the general on-attractor version of the expansion for evolution equations (possibly infinite-dimensional) possessing a finite-dimensional attractor in [11]. It is important to note that spectra of dynamical systems can have support on non-discrete sets in the complex plane, provided the space of observables is large enough, or the dynamics is complex enough [9, 11]. Here, we restrict our attention largely to observables that are in on-attractor variables and analytic in off-attractor variables, and find that the resulting spectra are - for quasi-periodic systems - supported on discrete sets in the complex plane. This observation by the author lead to development of the analytic framework for dissipative dynamical systems using Hardy-type spaces for dynamical systems, in which the composition operator is always spectral [12].

Eigenfunctions of the composition operator contain information about geometry of the state space. For example, invariant sets [13], isochrons [14, 15] and isostables (generalizaton of Fenichel fibers) [16] can all be defined as level sets of eigenfunctions of the operator. Here we extend this set of relations by showing that center-stable, center and center-unstable manifolds of an attractor can be defined as joint -level sets of a set of eigenfunctions. This can be viewed as shifting the point of view on such invariant manifolds from local - where the essential ingredient of their definition is tangency to a linear subspace [17] - to a global, level-set based definition. The connections between geometric theory and operator theory are explored further here: Floquet analysis in the case of a limit cycle, and generalized Floquet analysis [18] in the case of limit tori are used to obtain global (as opposed to local, as in geometric theory) results on spectral expansions. The usefulness of spectral expansions stems from the fact that most contributions to dynamics of autonomous dissipative systems are exponentially fast, and the dynamics is taken over by the slowest decaying modes and zero-real part eigenvalue modes. This has relationship to the theory of inertial manifolds.

On the data analysis side, the operator-theoretic analysis has recently gained popularity in conjunction with numerical methods such as variants of the Dynamic Mode Decomposition (DMD), Generalized Laplace Analysis, Prony and Hankel-DMD analysis [14, 11, 19, 6, 7, 20, 21, 22, 23], that can approximate part of the spectrum of an underlying linear operator under certain conditions on the data structure [23]. Since these methods operate directly on data (observables), they have been used to analyze a large variety of dynamical processes in many applications. We classify here the types of spectra associated with dynamical systems of different transient and asymptotic behavior. The spectrum is always found out to be of what we call the lattice type, and is defined as a linear combination over integers of principal eigenvalues, where is the dimension of the state space. This can help with understanding the dynamics underlying the spectra obtained from data. Namely, the principal dimension of the data can be determined by examining the lattice and finding the number of principal eigenvalues.

The paper is organized as follows: in section 2 we consider the case of linear systems, including those for which geometric and algebraic multiplicity is not equal. We obtain the spectral expansion using the Kato Decomposition. We also obtain explicit generalized eigenfunctions of the associated composition operator. Using the spectral expansion, the stable, unstable and center subspaces are defined as joint zero level sets of collections of eigenfunctions. An extension of these ideas to nonlinear systems with equilibria is given in section 4, utilizing the new concept of open eigenfunctions of the Koopman operator. The linearization theorem of Palmer is used to provide global definitions of center, center-stable and center-unstable manifolds using zero level sets of collections of composition operator eigenfunctions. Spectral expansion theorems for asymptotically limit cycling systems are given in section 5 for 2D systems and in section 6 for n-dimensional systems with a limit cycle. The reason for distinguishing between these two cases is that in the 2-dimensional case the eigenfunctions and eigenvalues can be derived explicitly in terms of averages over the limit cycle, while in the general case we use Floquet theory, due to the non-commutativity of linearization matrices along the limit cycle. In section 7 we derive the spectral expansion for systems globally stable to a limit torus, where attention has to be paid to the exact nature of the dynamics on the torus. Namely, Kolmogorov-Arnold-Moser type Diophantine conditions are needed for the asymptotic dynamics in order to derive the spectral expansion, providing another nice connection between the geometric theory and the operator theoretic approach to dynamical systems. We discuss the possibility of determining the principal dimension of the data using spectral expansion results in section 8. In section 9 we present a measure-preserving system that has a continuous Koopman operator spectrum, but integrable dynamics, and discuss the consequence for data analysis in such systems. We conclude in section 10.

2 Linear systems

2.1 Continuous-time Linear Systems with Simple Spectrum

In the case when the dynamical system is linear, and given by its matrix eigenvalues are eigenvalues of the associated Koopman operator. The associated Koopman eigenfunctions are given by [19]:

(1)

where are eigenvectors of the adjoint (that is, ), normalized so that , where is an eigenvector of , and denotes an inner product on the linear space in which the evolution is taking place. This is easily seen by observing

(2)

and thus Now, for any , as long as  has a full set of eigenvectors at distinct eigenvalues , we may write

Thus,

(3)

where is the vector function that associates Cartesian coordinates with the point (the initial condition) in state space. This is an expansion of the dynamics of observables - in this case the coordinate functions in terms of spectral quantities (eigenvalues, eigenfunctions and Koopman modes ) of the Koopman family . Considering (3), we note that the quantity we know as the eigenvector is not associated with the Koopman operator, but rather with the observable - if we changed the observable to, for example , being an matrix, then the expansion would read

(4)

and we would call the Koopman mode111Koopman modes are defined up to a constant, the same as eigenvectors. However, here we have defined projection with respect to a specific basis with an orthonormal dual. of observable . Assume now that the space of observables on we are considering is the space of complex linear combinations of . Then, is the projection of the observable onto the eigenspace of the Koopman family spanned by the eigenfunction .

Note that what changed between expansions (3) and (4) is the Koopman modes. On the other hand, the eigenvalues and eigenfunctions used in the expansion do not change. Thus, what changes with change in observables is their contribution to the overall evolution in the observable, encoded in . These properties persist in the fully nonlinear case, with the modification that the spectral expansion is typically infinite and can have a continuous spectrum part.

Note also that the evolution of coordinate functions can be written in terms of the evolution of Koopman eigenfunctions, by

(5)

2.2 Continuous-time Linear Systems: the General Case

In general, the matrix can have repeated eigenvalues and this can lead to a lack of eigenvectors. Recall that the algebraic multiplicity of an eigenvalue of is the exponent () of the polynomial factor of the characteristic polynomial . In other words, it is the number of repeat appearences of as a zero of the characteristic polynomial. An eigenvalue that repeats times does not necessarily have eigenvectors associated with it. Indeed - the algebraic multiplicity of is bigger than or equal to geometric multiplicity, which is the number of eigenvectors associated with . Such sonsiderations lead to the so-called Kato Decomposition. Kato Decomposition is an example of a spectral decomposition, where a linear operator is decomposed into a sum of terms consisting of scalar multiples of projection and nilpotent operators. For a finite-dimensional linear operator it reads [24]:

(6)

Each is a projection operator on the algebraic eigenspace that can be defined as the null space of , and is a nilpotent operator. We now use this spectral decomposition theorem for finite-dimensional linear operators to provide an easy, elegant proof of Hirsch-Smale theorem [25] on solutions of ordinary differential equations. Consider a linear ordinary differential equation on where is an matrix. It is well-known that the solution of this equation reads where is the initial condition. The exponentiation of the matrix reads

(7)

Now, from the Kato decomposition, and using the fact that

(8)

we obtain

(9)

where are eigenvalues of . We rewrite as

Note now that

(11)

We can rewrite the second sum in the last line of (LABEL:exp1) as

(12)

leading further to

(13)

Thus we get

(15)

Let us now connect this expansion to the formula we obtained previously, given by (3). In that case, we assumed that algebraic multiplicities of all eigenvalues are , and there is a full set of associated eigenvectors . Thus, the nilpotent part , and the projection of a vector on the eigenspace is

(16)

Using this with (15), we obtain (3).

More generally, let the dimension of each geometric eigenspace be equal to , let be the counter of distinct eigenvalues of and their multiplicities (or equivalently dimensions of algebraic eigenspaces corresponding to eigenvalues). Label the basis of the generalized eigenspace by , where are chosen so that . In other words, is a standard eigenvector of at and the generalized eigenvectors satisfy Now let where is the dual basis vector to and satisfies

(17)

Note that for .

(18)

We call the generalized eigenfunctions of the Koopman operator at eigenvalue .

Example 1.

To justify the name generalized eigenfunctions, consider the following simple example: let . Then , where is an eigenfunction of at satisfying Then

(19)

Thus, is in the nullspace of the differential operator .

Expanding from Example 1, for arbitrary, generalized eigenfunctions satisfy By integrating (18), the time evolution of the generalized eigenfunctions reads

(20)

(in fact by directly differentiating (20), one can easily find out that it satisfies (18)). Now writing

(21)

we get

leading further to

(23)

We connect the formula we just obtained with the expansion (15). Comparing the two, it is easy to see that

(24)

and

(25)

The above discussion also shows that, as long as we restrict the space of observables on to linear ones, , where is a vector in , then the generalized eigenfunctions and associated eigenvalues of the Koopman operator are obtainable in a straightforward fashion from the standard linear analysis of and its transpose.

It is easy to see that the most general case, in which dimension of geometric eigenspaces is not necessarily , is easily treated by considering geometric eigenspace of dimension say as two geometric eigenspaces of dimension . Keeping in mind that these correspond to - numerically - the same eigenvalue, we can define generalized eigenvectors corresponding to each eigenvector in - now separate - -dimensional geometric eigenspaces.

2.3 The Canonical Form of Linear Systems

The (generalized) Koopman eigenfunctions

can be thought of as “good” coordinates for linear systems. Let

From

(26)

we obtain

(27)

where

(28)

and

(29)

is the Jordan block corresponding to the eigenvalue . Note that Koopman eigenfunctions can be complex (conjugate) and thus this representation is in general complex.

The real form of the Jordan block corresponding to a complex eigenvalue whose geometric multiplicity is less than algebraic multiplicity is obtained using the variables (for ) and the polar coordinates

(30)

(for ). Thus,

and get transformed into to yield the -th Jordan block

(31)

where

(32)

and is the identity matrix.

We have the following corollary of the above considerations:

Corollary 2.1.

If a set of functions satisfy

(33)

where is the complex Jordan normal form of a matrix , then is a set of (generalized) eigenfunctions of

(34)

2.4 Stable, Unstable and Center Subspace

Let us recall the definition of stable, unstable and center subspaces of : the stable subspace of the fixed point is the location of all the points in that go to the fixed point at the origin as . The stable subspace is classically obtained as the span of (generalized) eigenvectors corresponding to eigenvalues of negative real part. In the same way, the unstable subspace of the fixed point is the location of all the points that go to the fixed point at the origin as , and is classically obtained as the span of (generalized) eigenvectors corresponding to eigenvalues of positive real part. The center subspace is usually not defined by its asymptotics (but could be, as we will see that it is the location of all the points in the state space that stay at the finite distance from the origin, or grow slowly (algebraically) as ), but rather as the span of (generalized) eigenvectors associated with eigenvalues of zero real part.

Looking at the equation (3), it is interesting to note that one can extract the geometrical location of stable, unstable and center subspaces from the eigenfunctions of the Koopman operator. We order eigenvalues from the largest to the smallest, where we do not pay attention to the possible repeat of eigenvalues. Let be the number of negative real part eigenvalues, and positive real part eigenvalues.

Proposition 2.1.

Let be positive real part eigenvalues, be real part eigenvalues, and be negative real part eigenvalues of a matrix of an LTI system. Let

(35)

be the (generalized) eigenfunctions of the associated Koopman operator. Then the joint level set of (generalized) eigenfunctions

(36)

is the stable subspace ,

(37)

is the center subspace , and

(38)

the unstable subspace .

Proof.

Note that setting leads to annulation of terms in (23) that are multiplied by , where . Thus, any initial condition belonging to has evolution governed by terms that asymptotically converge to and thus are parts of the stable subspace. Conversely, assume that does not belong to , but the trajectory starting at it asymptotically converges to . Since has non-zero projection on at least one of the (generalized) eigenvectors of , that are associated with eigenvalues of non-negative real part, we get a contradiction. The proof for the unstable subspace is analogous.

Since the center subspace is defined as the span of the (generalized) eigenvectors of having eigenvalues with zero real part, the initial condition in the center subspace can not have any projection on (generalized) eigenvectors associated with eigenvalues with positive or negative real part, and thus

This implies that is in . Conversely, if then does not have any projection on (generalized) eigenvectors associated with eigenvalues with positive or negative real part, and thus is in . ∎

This generalizes nicely to nonlinear systems (see below), in contrast to the fact that the standard definition, where e.g. the unstable space is the span of does not. Namely, even when the system is of the form

for bounded, , and small, using the span of eigenvectors we can only show existence of the unstable, stable and center manifolds that are tangent to the unstable, stable and center subspace , respectively.

So, the joint zero level sets of Koopman eigenfunctions define dynamically important geometric objects - invariant subspaces - of linear dynamical systems. This is not an isolated incident. Rather, in general the level sets of Koopman eigenfunctions reveal important information about the state space geometry of the underlying dynamical system.

3 Koopman Eigenfunctions Under Conjugacy

Spectral properties of the Koopman operator transform nicely under conjugacy. Here we use the notion of conjugacy defined more generally than in the classical context. In fact, we will define the notion of factor conjugacy - coming from the fact that we are combining notions of factors from measure theory [26], and the topological notion of conjugacy [17].

Let be the family of mappings and the Koopman operator associated with

with and a family of mappings and the Koopman operator associated with

Assume that is an eigenfunction of associated with eigenvalue . In addition, let be a mapping such that

(39)

i.e. the two dynamical systems are (factor) conjugate.222This is not the standard notion of conjugacy, since the dimensions of spaces that maps between is not necessarily the same, i.e. necessarily. Then we have

(40)

i.e. if is an eigenfunction at of , then the composition is an eigenfunction of at . As a consequence, if we can find a global conjugacy of a nonlinear system to a linear system, then the spectrum of the Koopman operator can typically be determined from the spectrum of the linearization at the fixed point. We discuss this, and some extensions, in the next section. The classical notion of topological conjugacy is obtained when and is a homeomorphism (a continuous invertible map whose inverse is also continuous). If is a diffeomorphism, then we have a diffeomorphic conjugacy. The notion of factor conjugacy is broader than those classical definitions, and includes the notion of semi-conjugacy, that is obtained when is continuous or smooth, but . 333The definition of factor conjugacy can be generalized to include dynamical systems on spaces and , where and (see [14], where such concept was defined for the case indicating conjugacy to a rotation).

Generalized eigenfuctions are preserved under conjugation, just like ordinary eigenfunctions: let be the geometric multiplicity of the eigenvalue . For we have (see (20)):

thus indicating that is a function that evolves in time according to the evolution equation (20) and thus is a generalized eigenfunction. Together with the fact that we already proved this for ordinary eigenfunctions in (40), we get

Proposition 3.1.

Let be the family of mappings and the Koopman operator associated with and the family of mappings and the Koopman operator associated with . In addition, let be a diffeomorphism such that , i.e. the two dynamical systems are diffeomorphically conjugate. If is a (generalized) eigenfunction at of , then the composition is a (generalized) eigenfunction of at .

4 Nonlinear Systems with Globally Stable Equilibria

Non-degenerate linear systems (i.e. those with ) have a single equilibrium at the origin as the distinguished solution. As the natural first extension to the nonlinear realm, it is interesting to consider a class of nonlinear systems that (at least locally) have an equilibrium as the only special solution, and consider what spectral theory of the Koopman operator for such systems can say.

For systems that are stable to an equilibrium from an open attracting set, we develop in this section a theory that strongly resembles that of linear systems - as could be expected once it is understood how Koopman eigenfunctions change under conjugacy. Geometric notions that were discussed in the previous LTI context, such as stable, unstable and center manifolds are developed in this section for nonlinear systems with globally stable equilibria. Since we use local conjugacy theorems, such as the Hartman-Grobman theorem, we start with the results that enables extension of an eigenfunction of the Koopman operator from an open set to a larger domain in state space.

4.1 Eigenfunctions of the Koopman Operator Defined on Subsets of the State-Space

The classical linearization theorems that we will utilize in our study are commonly defined on a neighborhood of a set of special dynamical significance, such as an equilibrium point, an invariant torus, or a strange attractor. The idea we pursue here is that extensions of such “local” eigenfunctions can be done using the flow, as long as the resulting set in state space does not begin to intersect itself. We first define the notion of an open eigenfunction and subdomain eigenfunction.

Definition 4.1.

Let , where is not an invariant set. Let , and a connected open interval such that . If

(41)

Then is called an open eigenfunction of the Koopman operator family , associated with an eigenvalue

If is a proper invariant subset of (in which case , for every ), we call a subdomain eigenfunction.

Clearly, if and , for every then is an ordinary eigenfunction. The following lemma enables an extension of eigenfunctions of the composition operator to a larger set:

Lemma 2.

Let be an open, connected set in the state space such that is a continuous function that satisfies

(42)

for some . For , let to be the time such that defined by

Let

Also, let

Let be the set of points for which is defined, i.e. . Let and be the times such that

(43)

and

(44)

Then is a connected open interval. For define

(45)

Assume there is a such that , or Then, is a continuous, open eigenfunction of on associated with the eigenvalue .

Proof.

All contain and are open and connected. Pick a in . For any such that we have

We obtain

(46)

and by assumption we know that is not invariant, so the function defined in (45) satisfies the requirement (41) for being an open eigenfunction of .

The continuity of is proved as follows: let , and choose an open interval such that . The set is a union of open sets, and its image under is . Thus, is a continuous function. ∎

Corollary 4.1.

If , and is a proper subset of , then is a subdomain eigenfunction of on associated with the eigenvalue .

Corollary 4.2.

If , and then is an eigenfunction of on associated with the eigenvalue .

Remark 3.

One can think of the function as the time to enter the closure of , either by a forward flow, or by backward flow from .

Remark 4.

It is easy to build open eigenfunctions around any point that is not an equilibrium. Namely, in an open set around any non-equilibrium point , the flow can be straightened out in coordinates such that . Now consider any function defined on the section , and define Then,

(47)

for any provided is real, and provided is complex, and the definition being valid in . Thus, singularities in state space, such as fixed points, and reccurrencies, such as those occuring in a flow around a limit cycle, serve to select ’s in the Koopman operator spectrum.

4.2 Poincaré Linearization and Eigenmode Expansion

We consider a continuously differentiable dynamical system defined in some open region of ,

(48)

where the origin is an equilibrium contained in . The matrix is the gradient of the vector field at , and is the “nonlinear” part of the vector field, . The system (48) induces a flow and the positively invariant basin of attraction of the fixed point is defined by

(49)

The Poincaré linearization, valid for analytic vector fields with no resonances amongst eigenvalues, in the neighborhood of a fixed point reads

Definition 4.2.

Let be the eigenvalues of . We say that is resonant if there are nonnegative integers and such that

(50)

and

(51)

If is not resonant, we say that it is nonresonant.

For analytic vector fields, according to the normal form theory, nonresonance, together with the condition that all eigenvalues are in the left half plane (stable case) or right half plane (unstable case), permits us to make changes of variables that remove nonlinear terms up to any specified order in the right-hand side of the differential equation [17]. Alternatively, the Siegel condition is required:

Definition 4.3.

We say that satisfy the Siegel condition if there are constants and such that

(52)

for all nonnegative integers satisfying

(53)

This leads to the Poincaré Linearization Theorem:

Theorem 4.1 (Poincaré Linearization Theorem).

Suppose that is analytic, , and that all the eigenvalues of are nonresonant and either all lie in the open left half-plane, all lie in the open right half-plane, or satisfy the Siegel condition. Then there is an analytic change of variables such that in a small neighborhood of the fixed point .

Poincaré linearization is used in normal form theory [27, 17], and the issue of resonances is the well-known reason that even analytic vector fields can not always be linearized using an analytic change of variables.

The Koopman group of operators associated with (48) evolves a (vector-valued) observable along the trajectories of the system, and is defined via the composition

Let be the eigenfunctions of the Koopman group associated with the Poincaré linearization matrix . Then are the (analytic) eigenfunctions of the Koopman operator associated with the nonlinear system. Clearly, . We will utilize

(54)

as a change of coordinates.

Like in the case of linear systems treated in section 2 we’d like to again get an expansion of observable into eigenfunctions of . If the observable is analytic, the Taylor expansion of around the origin444Note that the change of variables in the Poincaré-Siegel linearization is an analytic diffeomorphism [28], and thus is analytic. yields