Extreme learning machine for reduced order modeling of turbulent geophysical flows
We investigate the application of artificial neural networks to stabilize proper orthogonal decomposition based reduced order models for quasi-stationary geophysical turbulent flows. An extreme learning machine concept is introduced for computing an eddy-viscosity closure dynamically to take into account the effects of the truncated modes. We consider a four-gyre wind-driven ocean circulation problem as our prototype setting to assess the performance of the proposed data-driven approach. Our framework provides a significant reduction in computational time and effectively retains the dynamics of the full-order model during the forward simulation period beyond the training data set. Furthermore, we show that the method is robust for larger choices of time steps and can be used as an efficient and reliable tool for long time integration general circulation models.
The spatiotemporal complexity of many applications in computational sciences leads to very large-scale dynamical systems whose simulations make overwhelming and unmanageable demands on computational resources. Indeed, many problems remain intractable when multiple forward full-order numerical simulations are required. Since the computational cost of these high fidelity simulations is prohibitive, model order reduction approaches, also known as reduced order models (or ROMs), are commonly used to reduce this computational burden in many applications (e.g., see Brunton and Noack (2015) for a review of closed-loop control applications in fluid turbulence, and Daescu and Navon (2007); Cao et al. (2007); Daescu and Navon (2008); Fang et al. (2009) for a discussion of variational data assimilation applications in weather and climate modeling). A number of recent review articles have addressed the strengths of several modal analysis, reduced basis and model reduction techniques Antoulas et al. (2001); Smith et al. (2005); Quarteroni et al. (2011); Chinesta et al. (2011); Mezić (2013); Benner et al. (2015); Rowley and Dawson (2017); Taira et al. (2017). In their survey, dedicated primarily to the reduced-order modeling for fluid analysis and control, Rowley and Dawson Rowley and Dawson (2017) have discussed several techniques including proper orthogonal decomposition (POD), balanced truncation and balanced POD, eigensystem realization algorithms (ERA), dynamic mode decomposition (DMD) and Koopman operator theory with attention devoted to the similarities and analogies between these methods. An excellent overview and introduction to such techniques may also be found in Taira et al. (2017).
In this study, we consider the POD framework in combination with the Galerkin projection procedure Holmes et al. (1998), which is one of the prominent approaches for generating ROMs for nonlinear systems Ito and Ravindran (1998); Iollo et al. (2000); Noack et al. (2003); Rowley and Williams (2006); Bui-Thanh et al. (2008); Hay et al. (2009). POD extracts the most energetic modes (usually from high-fidelity experimental or numerical data), which are expected to contain the dominant statistical characteristics of these systems. It is therefore possible to obtain good approximations to the high-fidelity data with a few POD modes in which fine scale details are embedded. The resulting systems are low dimensional (due to truncation) but dense and provide robust surrogate models for forward simulations. It has been widely used in various disciplines under a variety of different names (e.g., see Narasimha (2011) for an excellent historical discussion).
Although the standard Galerkin projection provides an efficient way to generate ROMs, its applicability to complex systems is limited primarily due to errors associated with the truncation of POD modes. The limitation is more prominent in turbulent flow systems where an intense scale separation leads to insufficient embedding of dynamics within a feasibly small number of modes. To model the effects of the discarded modes, several closure models are devised (see for instance Couplet et al. (2003); Kalb and Deane (2007); Bergmann et al. (2009); Kalashnikova and Barone (2010); Wang et al. (2011); Carlberg et al. (2011); Wang et al. (2012); Amsallem and Farhat (2012); Balajewicz et al. (2013); Lassila et al. (2013); Baiges et al. (2015); Wells et al. (2017); Xie et al. (2017)) which serve a dual purpose: that of numerical stabilization as well as statistical fidelity preservation. Following the large eddy simulation (LES) ideas, it has been shown that the eddy viscosity concept provides an efficient framework to account the effect of the truncated modes Borggaard et al. (2011); Akhtar et al. (2012); San and Iliescu (2014, 2015). In this study, we put forth a robust dynamic procedure for computing the modal eddy viscosities in order to stabilize the ROMs. The novelty of our approach stems from the design of an artificial neural network (ANN) architecture to predict the magnitude of the mode dependent eddy viscosity dynamically, thus removing the need for an a-priori specification of an arbitrary value.
ANNs and other machine learning strategies have engendered a revolution in data-driven prediction applications and are seeing widespread investigation in the computational physics community. Previous studies into the feasibility of similar machine learning (ML) techniques for ROMs of various nonlinear systems may be found in Narayanan et al. (1999); Khibnik et al. (2000); Sahan et al. (1997); Moosavi et al. (2015, 2017); San and Maulik (2018). In particular, we have recently illustrated the ANN concept for model order reduction of the one-dimensional Burgers equation and the performence of several training algorithms has been documented San and Maulik (2018). In the present study, however, we put forth a modified ANN architecture since it is more appropriate to turbulent flows. The ML approaches have been also devised for use in feedback flow control where they generate a direct mapping of flow measurements to actuator control systems Gillies (1995, 1998, 2001); Faller and Schreck (1997); Hocevar et al. (2004); Efe et al. (2004, 2005); Lee et al. (1997). In our investigation, information from the high fidelity evolution of governing laws is leveraged to provide a supervised learning framework for a single layer ANN to stabilize ROMs of the mesoscale forced-dissipative geophysical turbulence system. In brief, an ANN estimates a nonlinear relationship between a desired set of inputs and targets provided viable benchmark data for their underlying statistical relationship is available. This subset of the ML field has seen wide application in function approximation, data classification, pattern recognition and dynamic systems control applications Widrow et al. (1994); Demuth et al. (2014) and is generating great interest for its utility in the reproduction of systems with pronounced nonlinear interactions Raissi et al. (2017a, b).
Before its deployment as a prediction or regression tool, an ANN is trained to accurately capture the nonlinear relationship between its inputs and outputs through some classical loss function (such as mean squared error). A regularized training ensures that the framework avoids overfitting any noise that may have been present in the training data. For our supervised learning framework, we utilize the extreme learning machine (ELM) Huang et al. (2006) training procedure, which stands out from other learning methods with extremely fast training, good generalization and universal approximation capabilities. For our investigation it is seen that a single hidden layer feed-forward ELM algorithm satisfies generalized training requirements with extremely reduced computational cost yet substantially accurate reproductions of training statistics.
For assessing our proposed framework, we utilize the governing laws given by the barotropic vorticity equation (BVE) model. It is a commonly used mathematical framework to study the forced-dissipative large scale ocean circulation problems, also known as the single-layer quasigeostrophic (QG) model Majda and Wang (2006). While POD, along with other optimal bases choices, has been used to derive computationally efficient ROMs of the BVE (see, e.g., Selten (1995); Galán del Sastre and Bermejo (2008); Crommelin and Majda (2004)), the present work represents the first attempt to model the unrepresented scales of the QG dynamics, mesoscale turbulence and their effect on mean circulation using an ANN based supervised machine learning framework.
Ii Full Order Modeling
Oceanic and atmospheric flows display an enormous range of spatial and temporal scales, from seconds to decades and from centimeters to thousands of kilometers. Thus, a model incorporating all the relevant physics of the ocean and atmosphere would be impractical for numerical simulations. During the last decades, significant advancements were made in developing simplified models for geophysical fluid dynamics Gill (1982); Pedlosky (1982); Vallis (2006); McWilliams (2006); Cushman-Roisin and Beckers (2011), which have been instrumental in providing relatively accurate numerical results at a reasonable computational price. Although these models have continued to produce increasingly accurate results and therefore improved weather forecasting, their use in long time integrations such as those required by climate modeling remains challenging Ghil et al. (2008); Lynch (2008). To illustrate our surrogote proposed framework, we consider the BVE model, which has been extensively used to study the forced-dissipative QG dynamics Majda and Wang (2006). The dimensionless BVE may be given by Greatbatch and Nadiga (2000); Holm and Nadiga (2003); San et al. (2011)
where is the kinematic vorticity and is the streamfunction. The nonlinear advection term is defined by the Jacobian
since we define the flow velocity components by
and the following kinematic relationship holds for satisfying the incompressibility constraint
where is the standard Laplacian operator. The dimensionless BVE given in Eq. (1), has two nondimensional parameters, the Reynolds and Rossby numbers, which are related to the characteristic length and velocity scales in the following way:
where is the horizontal eddy viscosity of the BVE model and is the gradient of the Coriolis parameter. We note that Eq. (1) uses the -plane approximation, valid for most oceanic basins, which accounts for the Earth’s rotational effects by approximating the Coriolis parameter. For the purpose of nondimensionalization, represents a characteristic horizontal length scale given by the basin dimension in the direction, and is a characteristic velocity scale (also known as the Sverdrup velocity) given by
where is the maximum amplitude of the double-gyre wind stress, is the mean fluid density, and is the mean depth of the ocean basin. Following Cummins (1992); Greatbatch and Nadiga (2000); Nadiga and Margolin (2001); Holm and Nadiga (2003); San et al. (2011), we consider a four-gyre circulation problem, a benchmark oceanic flow problem whose behavior is difficult to capture correctly in coarse grained models San et al. (2011). Indeed, as shown in San and Iliescu (2015), the standard model order reduction approaches without stabilization are incapable of resolving the correct physics. In our full order model (FOM) simulations we use a second-order accurate kinetic energy and enstrophy conserving Arakawa finite difference scheme Arakawa (1966). The derivatives in the linear terms are also approximated using the standard second-order finite differences. Our time advancement scheme is given by the classical total variation diminishing third-order accurate Runge-Kutta scheme Gottlieb and Shu (1998). Details of the Poisson solver, numerical schemes and boundary conditions used for this study may be found in San et al. (2011).
Iii Reduced order modeling
We build our reduced order modeling framework based on a standard projection methodology using the method of snapshots Sirovich (1987). Solving the FOM given by Eq. (1), the th record of the prognostic variable (vorticity field) is denoted for , where is the number of snapshots recorded for basis construction. Then we decompose the solution field into a time invariant averaged and a fluctuating component through Holmes et al. (1998); Noack et al. (2003),
where is the two-dimensional domain and the mean of the snapshot data is
In order to obtain the POD basis functions, a correlation matrix of the fluctuating part is constructed by
where the subscripts and refer to snapshot indexes. We must note that the data correlation matrix is a non-negative Hermitian matrix. We further define the inner product of two functions and as
where is the diagonal eigenvalue matrix and refers to right eigenvector matrix whose columns are eigenvectors of the correlation matrix . The eigenvalues are usually stored in descending order for practical purposes i.e., . Then the orthogonal POD basis functions of the vorticity field can be obtained as
where is the th eigenvalue, is the th component of the th eigenvector, and is the th POD mode. The kinematic relationship between streamfunction and vorticity given by Eq. (4) may be utilized to obtain the th basis function for the streamfunction, , by solving a Poisson equation
Now we can span our field variables into the POD modes as follows
where we have decomposed using time dependent modal coefficient and the POD modes . We note that the kinematic relationship given by Eq. (13) implies that the same accounts for the streamfunction as well. A ROM can be generated by a truncation of the total bases to only retained modes where . These largest energy containing modes correspond to the largest eigenvalues, , , …, . To obtain our standard ROM, an orthogonal Galerkin projection is performed by multiplying Eq. (1) with the POD basis functions and integrating over the domain . The resulting dynamical system for can be written as
The ROM given by Eq. (16) consists of coupled ordinary differential equations and can be solved by a standard time integrator (a third-order Runge-Kutta scheme is used in this study). We note that the resulting ROM is highly computationally efficient since all the POD basis functions and corresponding model coefficients given by Eq. (17) are precomputed from the data provided by snapshots. A complete specification of the dynamical system given by Eq. (16) may be obtained by the following projection of the initial condition:
where is the vorticity field specified at time .
The standard ROM given by Eq. (16) usually works well for a periodic or quasi-periodic system for which the largest modes adequately capture the system’s dynamics. However, one of the main sources of inaccuracy in a truncated ROM framework is the potential for instability due to neglecting the contributions of the higher POD modes. Therefore, many stabilization schemes are utilized in order to improve the performance of the ROMs Couplet et al. (2003); Kalb and Deane (2007); Bergmann et al. (2009); Kalashnikova and Barone (2010); Wang et al. (2011); Carlberg et al. (2011); Wang et al. (2012); Amsallem and Farhat (2012); Balajewicz et al. (2013); Lassila et al. (2013); Baiges et al. (2015); Wells et al. (2017); Xie et al. (2017)). Using an eddy viscosity approach, the stabilization of the ROM can be achieved by San and Iliescu (2014, 2015)
where, using the Smagorinsky model and the analogy between ROM and LES, additional two terms can be written as
where is the modal eddy viscosity parameter. This free stabilization parameter may be simply considered as a global constant for all the modes Aubry et al. (1988); Wang et al. (2012). The global constant eddy viscosity idea may improved by supposing that the amount of dissipation is not identical for all the POD modes Rempfer (1991); Cazemier (1997). It has been shown that finding an optimal value for this parameter significantly improves the predictive performance of ROMs San and Iliescu (2014, 2015). Therefore, the chief novelty of the present study is the utilization of a novel ML framework to estimate these modal eddy viscosity coefficients to stabilize and overcome errors due to the finite truncation in ROMs. We determine dynamically from our ML framework during the evolution of each temporal mode at each time step.
Iv Artificial Neural Network Architecture
In this section, we introduce a single hidden layer feedforward ANN architecture for predicting modal eddy viscosity coefficients for stabilization of ROMs. Figure 1 illustrates our ANN architecture which consists of an input layer, a hidden layer and an output layer. Each layer possesses a predefined number of nodes called neurons. Except for the input neurons, each neuron has an associated bias and activation function. The main goal in any supervised learning framework is to find a mapping between input nodes and output nodes. Mathematically, we are looking for a mapping to establish a relationship between input nodes and output nodes as follows:
where is the number of input neurons and is the number of output neurons. If refers to the number of hidden layer neurons, the th output node can be computed as
where are the connection weights between the neurons in input and hidden layers, and are the weights between the neurons in hidden and output layers. Here, and are neurons’ activation functions; and and are called biases operating as thresholds for hidden and output layers, respectively. In this study, we have utilized the tan-sigmoid activation function for the hidden layer neurons, which can be expressed as
and a linear activation function for the output layer neurons given by
While it has been reported that sigmoidal activation functions saturate across a large portion of their domain Goodfellow et al. (2016), our reasoning behind the use of the classical tan-sigmoid activation was to leverage the benefit of the saturation behavior to obtain improved aggregate behavior.
iv.1 Extreme learning machine
Introducing sample training data examples (i.e., input-output pairs), the weights and biases can be computed in a supervised learning framework using either well established iterative back propagation methods Carrillo et al. (2016) or pseudoinverse approaches Cancelliere et al. (2017). In this study, the ANN architecture is trained by utilizing an extreme learning machine (ELM) approach proposed in Huang et al. (2006) for extremely fast training of a single hidden layer feedforward network. The ELM approach requires no biases in the output layer (i.e., ). In the ELM method, the weights and biases are initialized randomly from a uniform distribution (i.e., between -1 and 1 in our study) and no longer modified. Therefore the only unknowns to be determined are weights. Using the linear activation function for the output layer, Eq. (22) can be written for sample examples
where and refer to the training input-output data pairs. Using a more convenient matrix notation (i.e., , , , , and ), our learning problem can be written as
where is given by
and the solution for the weights can be computed by
where is the pseudoinverse of . In order to compute the pseudoinverse, we apply the following singular value decomposition (SVD) to the matrix since its number of rows is greater than its number of columns in typical ML applications (i.e., )
where and are column-orthogonal and orthogonal matrices, and is a diagonal matrix whose elements (i.e., ) are non-negative and called singular values. Using the SVD, the pseudoinverse of becomes
where can be computed from by taking the reciprocal of each non-zero element (i.e., ). However, the presence of tiny singular values can cause numerical instability. Therefore, a well-known Tikhonov type regularization is often introduced by
where controls the trade off between the least-squares error and the penalty term for regularization (e.g., see Cancelliere et al. (2017)). In the present study we set . Finally, using Eq. (29), unknown weights can be computed by
iv.2 Training data
Our architecture is devised to take inputs accessible to us during the time integration of the ROM and estimate the modal eddy viscosity coefficient. Our high fidelity snapshot data (from which POD bases are constructed) are also used to train our architecture. First, we denote the right hand side of Eq. (16) as
and then apply the Galerkin projection to FOM given by Eq. (1), which yields the true solution
The ideal stabilization would thus conform to the differences between these quantities i.e.,
We know from Eq. (III) that
where we redefine
as the eddy viscosity stabilization for each mode within the training data set. With this calculation of the ideal stabilization, we hypothesize that a mode dependent nonlinear (but unknown) relationship exists between the resolved modes in the ROM that estimates optimally. To conclude, our ANN framework is trained between inputs given by the modal index , , and (i.e., they are all available during the ROM time stepping) and to predict an approximation for . We thus have 3 inputs to our network with hidden layer neurons to obtain 1 output (which is the modal eddy viscosity coefficients). The architecture of our ANN is shown in Figure 1.
To validate our proposed ANN framework, we consider the four-gyre barotropic circulation problem Cummins (1992); Greatbatch and Nadiga (2000); Nadiga and Margolin (2001); Holm and Nadiga (2003); San et al. (2011)). This test problem yields four gyres circulation patterns in the time mean in a shallow ocean basin , and represents an ideal test for the viability of the proposed ROM. Indeed it was shown that ROMs without stabilization are incapable of resolving the mean dynamics San and Iliescu (2015).
The dimensionless form of the BVE describing the QG problem is evolved from to using a fixed time step on a Munk layer resolving computational grid resolution. The dimensionless parameters of the BVE system are chosen as and . We must note that to is our data collection window (for the purpose of POD basis generation as well as ANN training) due to a statistically steady state reached after the initial transient period. 900 snapshots are collected during this period which are equally distributed in time. The ideal eddy viscosity is also computed at these snapshots for the use of training our machine learning framework. We note here that our ROM (whether purely truncated or stabilized by ANN) is utilized for predictions upto utilizing the POD modes obtained from our previously mentioned data collection window. This may be considered to be a challenging validation of our dual data-driven methodology for the QG problem. For both our standard ROM and stabilized ROM-ANN computations, the same time step with is used for time integration of the dynamical system. Sensitivity studies for varying time steps will be also presented later.
Figure 2 shows the accumulation of energies in the form of eigenvalue magnitudes where it can be seen that a large majority (close to 75%) of the energies are accumulated in the first 30 modes of the transformed space. Figure 3 shows the gradual convergence of the ROM (i.e., without stabilization) to the four-gyre circulation pattern with increasing . Indeed, nonphysical two-gyre pattern is observed for the case of and .
Figure 4 shows the performance of the proposed framework (i.e., ROM-ANN) against the standard Galerkin projection based ROM with . Full order model (FOM) projections to reduced space are also shown for the purpose of comparison. It can easily be seen that the ELM stabilization reproduces the four-gyre pattern accurately as against the standard implementation of the ROM which fails to capture the pattern. This is observed for both streamfunction and vorticity contours. Figure 5 shows a qualitative comparison of the effect of the number of neurons where similar performance improvements are obtained for our choice of neurons. Table I shows a quantitative comparison of the improvement obtained by the proposed stabilization (for different neurons as well) against the standard ROM implementations with different number of modes. It is easily observed that the stabilization acts adequately in reproducing excellent agreement with full-order statistics at very low number of retained modes. Note that these plots and tabulated statistics are all for the statistically steady state behavior of the QG problem in our assessment window (i.e., to ), which is beyond the training data window.
Figure 6 shows a comparison for the evolution of through nondimensional time for both ROM and ROM-ANN implementations in comparison to the FOM projection. The ROM-ANN has a default neurons in this example. It can clearly be seen that the use of the stabilization prevents the explosion of numerical instability in the coarse truncated ROMs with and . At however, the first modal evolution shows a stable statistical steady state for the ROM.
Another benefit of the ROM-ANN mechanism over the standard ROM implementation is the possibility of using large time steps in the ordinary differential equation integrator. In the present study, a time step of was chosen for the FOM simulation to ensure a CFL criterion of less than 1.0 was always respected (as observed in the time series plot in Figure 7) due to the numerical stability of the numerical schemes. Figure 8 shows the vorticity and streamfunction contours when our stabilized method (i.e., the ROM-ANN with and ) is used with different time steps. It can be seen that a much larger time step of can be effectively used to obtain statistically accurate results without any divergence. Thus our proposed ANN based eddy viscosity stabilization is ideally suited to a fast prediction of the underlying dynamics. Figure 9 shows the evolution of the first temporal coefficient for the aforementioned ROM-ANN framework where it is seen that very high values of the time step do not affect the statistical viability of the stabilized ROM and leads to an excellent reduction in computational expense (the largest time step provides excellent results at a CPU time of 1.61 seconds in comparison to approximately 700 seconds for the default time step which is required for the FOM). We also note that the FOM required 195.4 hours CPU time to complete the forward simulation between and .
Vi Summary and Conclusions
In this paper, we have studied the feasibility of using a machine learning framework to stabilize projection based ROMs for solving a forced-dissipative general circulation problem. We construct a single hidden layer feedforward ANN to predict modal eddy viscosity coefficients dynamically. Our approach can be considered as a semi non-intrusive (without the need for an online access to the FOM for the ROM prediction), since the ANN architecture only requires reduced order space quantities to predict the stabilization term. A regularized ELM approach is used for training where we use the same data snapshots as we used for generating the POD basis functions. In that sense, there are two data-driven components to this research: high fidelity snapshots of data from DNS are utilized not just for POD basis synthesis but also for training our machine learning framework utilized for a-posteriori stabilization of the ROM-ANN. Results indicate that the utilization of the proposed framework lets the user deploy an extremely truncated system without losing any statistical fidelity. Also, time steps much larger than those necessary for FOM forward simulations may be utilized in the ROM-ANN thus leading to exceptional computational performance. We conclude that the method presented in this paper is robust enough to stabilize ROMs dynamically and satisfies the dual demands of statistical accuracy as well as low computational expense in surrogate forward predictions in long-term evolution of geophysical turbulent flows.
- Brunton and Noack (2015) S. L. Brunton and B. R. Noack, “Closed-loop turbulence control: Progress and challenges,” Applied Mechanics Reviews 67, 050801 (2015).
- Daescu and Navon (2007) D. N. Daescu and I. M. Navon, “Efficiency of a POD-based reduced second-order adjoint model in 4D-Var data assimilation,” International Journal for Numerical Methods in Fluids 53, 985 (2007).
- Cao et al. (2007) Y. Cao, J. Zhu, I. M. Navon, and Z. Luo, “A reduced-order approach to four-dimensional variational data assimilation using proper orthogonal decomposition,” International Journal for Numerical Methods in Fluids 53, 1571 (2007).
- Daescu and Navon (2008) D. Daescu and I. Navon, “A dual-weighted approach to order reduction in 4DVAR data assimilation,” Monthly Weather Review 136, 1026 (2008).
- Fang et al. (2009) F. Fang, C. Pain, I. Navon, G. Gorman, M. Piggott, P. Allison, P. Farrell, and A. Goddard, “A POD reduced order unstructured mesh ocean modelling method for moderate Reynolds number flows,” Ocean modelling 28, 127 (2009).
- Antoulas et al. (2001) A. C. Antoulas, D. C. Sorensen, and S. Gugercin, “A survey of model reduction methods for large-scale systems,” Contemporary Mathematics 280, 193 (2001).
- Smith et al. (2005) T. R. Smith, J. Moehlis, and P. Holmes, “Low-dimensional modelling of turbulence using the proper orthogonal decomposition: a tutorial,” Nonlinear Dynamics 41, 275 (2005).
- Quarteroni et al. (2011) A. Quarteroni, G. Rozza, and A. Manzoni, “Certified reduced basis approximation for parametrized partial differential equations and applications,” Journal of Mathematics in Industry 1, 3 (2011).
- Chinesta et al. (2011) F. Chinesta, P. Ladeveze, and E. Cueto, “A short review on model order reduction based on proper generalized decomposition,” Archives of Computational Methods in Engineering 18, 395 (2011).
- Mezić (2013) I. Mezić, “Analysis of fluid flows via spectral properties of the Koopman operator,” Annual Review of Fluid Mechanics 45, 357 (2013).
- Benner et al. (2015) P. Benner, S. Gugercin, and K. Willcox, “A survey of projection-based model reduction methods for parametric dynamical systems,” SIAM Review 57, 483 (2015).
- Rowley and Dawson (2017) C. W. Rowley and S. T. Dawson, “Model reduction for flow analysis and control,” Annual Review of Fluid Mechanics 49, 387 (2017).
- Taira et al. (2017) K. Taira, S. L. Brunton, S. Dawson, C. W. Rowley, T. Colonius, B. J. McKeon, O. T. Schmidt, S. Gordeyev, V. Theofilis, and L. S. Ukeiley, “Modal analysis of fluid flows: An overview,” AIAA Journal 55, 4013 (2017).
- Holmes et al. (1998) P. Holmes, J. L. Lumley, and G. Berkooz, Turbulence, coherent structures, dynamical systems and symmetry (Cambridge University Press, 1998).
- Ito and Ravindran (1998) K. Ito and S. Ravindran, “A reduced-order method for simulation and control of fluid flows,” Journal of Computational Physics 143, 403 (1998).
- Iollo et al. (2000) A. Iollo, S. Lanteri, and J.-A. Désidéri, “Stability properties of POD–Galerkin approximations for the compressible Navier–Stokes equations,” Theoretical and Computational Fluid Dynamics 13, 377 (2000).
- Noack et al. (2003) B. R. Noack, K. Afanasiev, M. Morzynski, G. Tadmor, and F. Thiele, “A hierarchy of low-dimensional models for the transient and post-transient cylinder wake,” Journal of Fluid Mechanics 497, 335 (2003).
- Rowley and Williams (2006) C. W. Rowley and D. R. Williams, “Dynamics and control of high-Reynolds-number flow over open cavities,” Annual Review of Fluid Mechanics 38, 251 (2006).
- Bui-Thanh et al. (2008) T. Bui-Thanh, K. Willcox, and O. Ghattas, “Model reduction for large-scale systems with high-dimensional parametric input space,” SIAM Journal on Scientific Computing 30, 3270 (2008).
- Hay et al. (2009) A. Hay, J. T. Borggaard, and D. Pelletier, “Local improvements to reduced-order models using sensitivity analysis of the proper orthogonal decomposition,” Journal of Fluid Mechanics 629, 41 (2009).
- Narasimha (2011) R. Narasimha, “Kosambi and proper orthogonal decomposition,” Resonance 16, 574 (2011).
- Couplet et al. (2003) M. Couplet, P. Sagaut, and C. Basdevant, “Intermodal energy transfers in a proper orthogonal decomposition-Galerkin representation of a turbulent separated flow,” Journal of Fluid Mechanics 491, 275 (2003).
- Kalb and Deane (2007) V. L. Kalb and A. E. Deane, “An intrinsic stabilization scheme for proper orthogonal decomposition based low-dimensional models,” Physics of fluids 19, 054106 (2007).
- Bergmann et al. (2009) M. Bergmann, C.-H. Bruneau, and A. Iollo, “Enablers for robust POD models,” Journal of Computational Physics 228, 516 (2009).
- Kalashnikova and Barone (2010) I. Kalashnikova and M. F. Barone, “On the stability and convergence of a Galerkin reduced order model (ROM) of compressible flow with solid wall and far-field boundary treatment,” International Journal for Numerical Methods in Engineering 83, 1345 (2010).
- Wang et al. (2011) Z. Wang, I. Akhtar, J. Borggaard, and T. Iliescu, “Two-level discretizations of nonlinear closure models for proper orthogonal decomposition,” Journal of Computational Physics 230, 126 (2011).
- Carlberg et al. (2011) K. Carlberg, C. Bou-Mosleh, and C. Farhat, “Efficient non-linear model reduction via a least-squares Petrov–Galerkin projection and compressive tensor approximations,” International Journal for Numerical Methods in Engineering 86, 155 (2011).
- Wang et al. (2012) Z. Wang, I. Akhtar, J. Borggaard, and T. Iliescu, “Proper orthogonal decomposition closure models for turbulent flows: a numerical comparison,” Computer Methods in Applied Mechanics and Engineering 237–240, 10 (2012).
- Amsallem and Farhat (2012) D. Amsallem and C. Farhat, “Stabilization of projection-based reduced-order models,” International Journal for Numerical Methods in Engineering 91, 358 (2012).
- Balajewicz et al. (2013) M. J. Balajewicz, E. H. Dowell, and B. R. Noack, “Low-dimensional modelling of high-Reynolds-number shear flows incorporating constraints from the Navier–Stokes equation,” Journal of Fluid Mechanics 729, 285 (2013).
- Lassila et al. (2013) T. Lassila, A. Manzoni, A. Quarteroni, and G. Rozza, in Reduced Order Methods for Modeling and Computational Reduction, edited by A. Quarteroni and G. Rozza (Springer, Milano, 2013).
- Baiges et al. (2015) J. Baiges, R. Codina, and S. Idelsohn, “Reduced-order subscales for POD models,” Computer Methods in Applied Mechanics and Engineering 291, 173 (2015).
- Wells et al. (2017) D. Wells, Z. Wang, X. Xie, and T. Iliescu, “An evolve-then-filter regularized reduced order model for convection-dominated flows,” International Journal for Numerical Methods in Fluids 84, 598 (2017).
- Xie et al. (2017) X. Xie, D. Wells, Z. Wang, and T. Iliescu, “Approximate deconvolution reduced order modeling,” Computer Methods in Applied Mechanics and Engineering 313, 512 (2017).
- Borggaard et al. (2011) J. Borggaard, T. Iliescu, and Z. Wang, “Artificial viscosity proper orthogonal decomposition,” Mathematical and Computer Modelling 53, 269 (2011).
- Akhtar et al. (2012) I. Akhtar, Z. Wang, J. Borggaard, and T. Iliescu, “A new closure strategy for proper orthogonal decomposition reduced-order models,” Journal of Computational and Nonlinear Dynamics 7, 034503 (2012).
- San and Iliescu (2014) O. San and T. Iliescu, “Proper orthogonal decomposition closure models for fluid flows: Burgers equation,” International Journal of Numerical Analysis and Modeling, Series B 5, 217 (2014).
- San and Iliescu (2015) O. San and T. Iliescu, “A stabilized proper orthogonal decomposition reduced-order model for large scale quasigeostrophic ocean circulation,” Advances in Computational Mathematics 41, 1289 (2015).
- Narayanan et al. (1999) S. Narayanan, A. Khibnik, C. Jacobson, Y. Kevrekedis, R. Rico-Martinez, and K. Lust, in Control Applications, 1999. Proceedings of the 1999 IEEE International Conference on (IEEE, 1999), vol. 2, pp. 1151–1156.
- Khibnik et al. (2000) A. Khibnik, S. Narayanan, C. Jacobson, and K. Lust, in Continuation Methods in Fluid Dynamics (Vieweg, 2000), vol. 74, pp. 167–178.
- Sahan et al. (1997) R. Sahan, N. Koc-Sahan, D. Albin, and A. Liakopoulos, in Proceedings of the 1997 IEEE International Conference on Control Applications, (IEEE, 1997), pp. 359–364.
- Moosavi et al. (2015) A. Moosavi, R. Stefanescu, and A. Sandu, “Efficient Construction of Local Parametric Reduced Order Models Using Machine Learning Techniques,” arXiv preprint arXiv:1511.02909 (2015).
- Moosavi et al. (2017) A. Moosavi, R. Stefanescu, and A. Sandu, “Multivariate predictions of local reduced-order-model errors and dimensions,” arXiv preprint arXiv:1701.03720 (2017).
- San and Maulik (2018) O. San and R. Maulik, “Neural network closures for nonlinear model order reduction,” Advances in Computational Mathematics, DOI:10.1007/s10444-018-9590-z (2018).
- Gillies (1995) E. Gillies, Ph.D. thesis, PhD. Dissertation, University of Glasgow, Scotland (1995).
- Gillies (1998) E. Gillies, “Low-dimensional control of the circular cylinder wake,” Journal of Fluid Mechanics 371, 157 (1998).
- Gillies (2001) E. Gillies, “Multiple sensor control of vortex shedding,” AIAA Journal 39, 748 (2001).
- Faller and Schreck (1997) W. E. Faller and S. J. Schreck, “Unsteady fluid mechanics applications of neural networks,” Journal of Aircraft 34, 48 (1997).
- Hocevar et al. (2004) M. Hocevar, B. SÌirok, and I. Grabec, “Experimental turbulent field modeling by visualization and neural networks,” Journal of Fluids Engineering 126, 316 (2004).
- Efe et al. (2004) M. O. Efe, M. Debiasi, H. Ozbay, and M. Samimy, in Proceedings of the International Conference on Mechatronics, 2004. (IEEE, 2004), pp. 560–565.
- Efe et al. (2005) M. Efe, M. Debiasi, P. Yan, H. Ozbay, and M. Samimy, in 43rd AIAA Aerospace Sciences Meeting and Exhibit (2005), p. 294.
- Lee et al. (1997) C. Lee, J. Kim, D. Babcock, and R. Goodman, “Application of neural networks to turbulence control for drag reduction,” Phys. Fluids 9, 1740 (1997).
- Widrow et al. (1994) B. Widrow, D. E. Rumelhart, and M. A. Lehr, “Neural networks: applications in industry, business and science,” Communications of the ACM 37, 93 (1994).
- Demuth et al. (2014) H. B. Demuth, M. H. Beale, O. De Jess, and M. T. Hagan, Neural Network Design (Martin Hagan, 2014).
- Raissi et al. (2017a) M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations,” arXiv preprint arXiv:1711.10561 (2017a).
- Raissi et al. (2017b) M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations,” arXiv preprint arXiv:1711.10566 (2017b).
- Huang et al. (2006) G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine: theory and applications,” Neurocomputing 70, 489 (2006).
- Majda and Wang (2006) A. Majda and X. Wang, Nonlinear dynamics and statistical theories for basic geophysical flows (Cambridge University Press, 2006).
- Selten (1995) F. M. Selten, “An efficient description of the dynamics of barotropic flow,” Journal of the Atmospheric Sciences 52, 915 (1995).
- Galán del Sastre and Bermejo (2008) P. Galán del Sastre and R. Bermejo, “Error estimates of proper orthogonal decomposition eigenvectors and Galerkin projection for a general dynamical system arising in fluid models,” Numerische Mathematik 110, 49 (2008).
- Crommelin and Majda (2004) D. T. Crommelin and A. J. Majda, “Strategies for model reduction: comparing different optimal bases,” Journal of the Atmospheric Sciences 61, 2206 (2004).
- Gill (1982) A. E. Gill, Atmosphere-ocean dynamics (Academic press, 1982).
- Pedlosky (1982) J. Pedlosky, Geophysical fluid dynamics (New York and Berlin, Springer-Verlag, 1982).
- Vallis (2006) G. K. Vallis, Atmospheric and oceanic fluid dynamics: fundamentals and large-scale circulation (Cambridge University Press, 2006).
- McWilliams (2006) J. C. McWilliams, Fundamentals of geophysical fluid dynamics (Cambridge University Press, 2006).
- Cushman-Roisin and Beckers (2011) B. Cushman-Roisin and J.-M. Beckers, Introduction to geophysical fluid dynamics: physical and numerical aspects (Academic Press, 2011).
- Ghil et al. (2008) M. Ghil, M. D. Chekroun, and E. Simonnet, “Climate dynamics and fluid mechanics: Natural variability and related uncertainties,” Physica D: Nonlinear Phenomena 237, 2111 (2008).
- Lynch (2008) P. Lynch, “The origins of computer weather prediction and climate modeling,” Journal of Computational Physics 227, 3431 (2008).
- Greatbatch and Nadiga (2000) R. J. Greatbatch and B. Nadiga, “Four-gyre circulation in a barotropic model with double-gyre wind forcing,” Journal of Physical Oceanography 30, 1461 (2000).
- Holm and Nadiga (2003) D. D. Holm and B. T. Nadiga, “Modeling mesoscale turbulence in the barotropic double-gyre circulation,” Journal of Physical Oceanography 33, 2355 (2003).
- San et al. (2011) O. San, A. E. Staples, Z. Wang, and T. Iliescu, “Approximate deconvolution large eddy simulation of a barotropic ocean circulation model,” Ocean Modelling 40, 120 (2011).
- Cummins (1992) P. F. Cummins, “Inertial gyres in decaying and forced geostrophic turbulence,” Journal of Marine Research 50, 545 (1992).
- Nadiga and Margolin (2001) B. T. Nadiga and L. G. Margolin, “Dispersive-dissipative eddy parameterization in a barotropic model,” Journal of Physical Oceanography 31, 2525 (2001).
- Arakawa (1966) A. Arakawa, “Computational design for long-term numerical integration of the equations of fluid motion: Two-dimensional incompressible flow. Part I,” Journal of Computational Physics 1, 119 (1966).
- Gottlieb and Shu (1998) S. Gottlieb and C.-W. Shu, “Total variation diminishing Runge-Kutta schemes,” Mathematics of Computation 67, 73 (1998).
- Sirovich (1987) L. Sirovich, “Turbulence and the dynamics of coherent structures. I. Coherent structures,” Quarterly of Applied Mathematics 45, 561 (1987).
- Ravindran (2000) S. Ravindran, “A reduced-order approach for optimal control of fluids using proper orthogonal decomposition,” International Journal for Numerical Methods in Fluids 34, 425 (2000).
- Aubry et al. (1988) N. Aubry, P. Holmes, J. L. Lumley, and E. Stone, “The dynamics of coherent structures in the wall region of a turbulent boundary layer,” Journal of Fluid Mechanics 192, 115 (1988).
- Rempfer (1991) D. Rempfer, Ph.D. thesis, University of Stuttgart (1991).
- Cazemier (1997) W. Cazemier, Ph.D. thesis, Rijksuniversiteit Groningen (1997).
- Goodfellow et al. (2016) I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning, vol. 1 (MIT press Cambridge, 2016).
- Carrillo et al. (2016) M. Carrillo, U. Que, and J. A. González, “Estimation of Reynolds number for flows around cylinders with lattice Boltzmann methods and artificial neural networks,” Physical Review E 94, 063304 (2016).
- Cancelliere et al. (2017) R. Cancelliere, R. Deluca, M. Gai, P. Gallinari, and L. Rubini, “An analysis of numerical issues in neural training by pseudoinversion,” Computational and Applied Mathematics 36, 599 (2017).