Surrogate-assisted reliability-based design optimization: a survey and a new general framework
Reliability-based design optimization (RBDO) is an active field of research with an ever increasing number of contributions. Numerous methods have been proposed for the solution of RBDO, a complex problem that combines optimization and reliability analysis. Classical approaches are based on approximation methods and have been classified in review papers. In this paper, we first review classical approaches based on approximation methods such as FORM, and also more recent methods that rely upon surrogate modelling and Monte Carlo simulation. We then propose a general framework for the solution of RBDO problems that includes three independent blocks, namely adaptive surrogate modelling, reliability analysis and optimization. These blocks are non-intrusive with respect to each other and can be plugged independently in the framework. After a discussion on numerical considerations that require attention for the framework to yield robust solutions to various types of problems, the latter is applied to three examples (using two analytical functions and a finite element model). Kriging and support vector machines together with their own active learning schemes are considered in the surrogate model block. In terms of reliability analysis, the proposed framework is illustrated using both crude Monte Carlo and subset simulation. Finally, the covariance-matrix adaptation - evolution scheme (CMA-ES), a global search algorithm, or sequential quadratic programming (SQP), a local gradient-based method, are used in the optimization block. The comparison of the results to benchmark studies show the effectiveness and efficiency of the proposed framework.
Keywords: Reliability-based design optimization – RBDO – Surrogate modelling – Simulation methods – Active learning
Realistic design of structures requires taking into account uncertainties which are ubiquitous to real world applications, e.g.in manufacturing tolerances, loads or environmental and operational conditions. Casting the problem as a standard deterministic design does not allow one to safeguard the structure against unforeseen failures. Various methods have been introduced in the literature and applied by field engineers for the design of structures under uncertainties. Partial safety factors (Ditlevsen and Madsen, 1996), for instance, allow the analyst to implicitly account for uncertainties through the use of conservative characteristics design parameters. However, such an approach often results in unnecessarily conservative and therefore costly design solutions. More convenient approaches have been developed under the framework of reliability-based design optimization, where in most cases, the design constraints are assessed in terms of failure probabilities with respect to some predefined performance function. This implies modelling probabilistically the sources of uncertainty and then properly propagating those uncertainties to the quantities of interest. The complexity of this problem stems from the coupling of optimization and structural reliability analysis. A natural approach to its solution consists in nesting the two levels i.e.exploring the design space using an appropriate optimization algorithm and computing the failure probability of each explored design using a structural reliability method. This is known in the literature as the two-level approach (Chateauneuf, 2008) and includes popular methods such as the reliability index approach (Nikolaidis and Burdisso, 1988) or the performance measure approach (Tu et al., 1999). In two-level approaches, the inner reliability analysis is carried out through an approximation method, typically the first-order reliability method (FORM). The cost of this naive approach can be somewhat high as it may require an overall large number of performance functions evaluations. Simplifying formulations such as the so-called decoupled or mono-level approaches have also been proposed in the literature (See Aoues and Chateauneuf (2010); Valdebenito and Schuëller (2010)). Despite some gain in efficiency, they remain expensive for the solution of real world problems, i.e.they require thousands of computer model runs. Additionally, the use of approximation methods to evaluate the structural reliability constraints can hinder their application scope, due to their lack of convergence for highly non-linear performance functions. Some authors have proposed methods that rely on direct Monte Carlo simulation to evaluate the failure probability. The interest in such approaches has dramatically grown with the introduction of surrogate modelling in the RBDO framework. In a nutshell, the idea of surrogate modelling is to replace the original expensive-to-evaluate model by a cheap approximation that can be used for subsequent analyses. In this paper, a classification of such surrogate-assisted RBDO approaches is proposed based on the different ways the surrogate model is introduced in the RBDO framework. Upon reviewing various methods in the literature, a unified modular and non-intrusive framework is proposed. The modularity here means that the framework consists of three independent blocks, namely design optimization, reliability analysis and surrogate modelling. By non-intrusiveness, it is meant here that the analyst can plug-in any method of his choice for each block without the need of changing anything else in the remaining blocks. Furthermore, the surrogate models are built adaptively so as to enhance the efficiency of the proposed framework. As an illustration, different configurations of each block are considered and applied to three problems of increasing complexity, which involve two analytical functions and a finite element model. More specifically, we consider:
support vector machines and Kriging as surrogate models;
crude Monte Carlo and subset simulation as reliability methods and
constrained covariance matrix adaptation - evolution scheme (CMA-ES) and sequential quadratic programming (SQP) as optimizers.
2 Formulation of the RBDO problem
There are various ways of formulating a reliability-based design optimization problem. A common feature shared by all these approaches is that they attempt to trade the cost of the system with its reliability. Frangopol (1985) and Chateauneuf (2008) make a short review from the historical viewpoint of the various RBDO formulations. The early approaches proposed to minimize a function which consists of an initial cost (mainly the addition of design and construction costs) and an expected cost of failure (See for instance Enevoldsen and Sørensen (1994)). Due to the difficulty of properly assigning monetary cost to structural failure, this formulation was not of practical interest for engineering applications. It is now mostly investigated under the framework of risk-based or life-cycle cost optimization (Beck and Gomes, 2012; Frangopol and Maute, 2003). Alternative formulations for RBDO have henceforth emerged. One, for instance, consists in maximizing the reliability, or equivalently minimizing the failure probability, under given cost constraints (See for instance Kuschel and Rackwitz (1997); Royset et al. (2001); Taflanidis and Beck (2008)).
The approach we consider in this paper, which is also the prevalent in current research, consists in minimizing an initial cost under some probabilistic constraints (Hilton and Feigen, 1960). Let be an dimensional vector defining some design parameters of the structure. In the presence of uncertainties, the variability of these parameters can be described by introducing random variables . These variables are called design parameters with uncertainty, or by slight abuse, random design variables in the sequel. In some cases, design parameters are purely deterministic. To simplify the notation, they are equally considered as at this stage, where the related random variables would simply have a zero variance. Other random parameters that may affect the structural response without being considered as decision parameters may also be of interest for the analysis. They are known as environmental variables and are denoted here by . A typical RBDO formulation may then read (Dubourg et al., 2011):
where is the cost function to be minimized under constraints that are classified into two categories. The first, , are a set of deterministic analytical functions that define the feasible domain, e.g.bounds between the design parameters. The second category, which consists of probabilistic constraints, set an upper threshold on the failure probability for each identified failure mode of the structure (herein ), assuming a series system behaviour. The latter is defined through a so-called limit-state function which partitions the random input space into safe and failure domains. By introducing a vector that gathers the random design and environmental variables, the failure probability for a given design can be expressed as (Ditlevsen and Madsen, 1996):
where represents the failure domain and is the joint distribution of all variables given a particular value of .
Equation (2) constitutes a multidimensional integration problem over an implicitly defined domain whose solution is difficult. This difficulty to some extent explains the delay between the early RBDO formulations and the implementation of solution procedures which flourished in the 80s. In general, the reliability techniques used to solve this integration problem mainly rely either on first-order approximations or on simulation methods, as detailed in the sequel.
3 A short review of RBDO solution techniques
3.1 Classical RBDO methods
Reliability-based design optimization is a rather rich and active field of research experiencing a continuous flow of publications. All the proposed methods seek to reduce the computational cost of the RBDO problem, mostly by introducing approximations in the reliability analysis or by reformulating the optimization problem. We start our survey using the classification suggested in Chateauneuf (2008); Aoues and Chateauneuf (2010), namely two-level, mono-level and decoupled approaches. Some techniques do not entirely fit this classification though, as will be shown in the sequel. We then proceed with other, more recent approaches that used advanced techniques. Older reviews of RBDO can also be found in Valdebenito and Schuëller (2010); Aoues and Chateauneuf (2010).
3.1.1 Two-level approaches
A two-level approach is the most direct way to solve a RBDO problem. It consists of two nested loops: the outer loop explores the design space via a suitable optimization scheme while the inner loop performs a reliability analysis. As the values of design parameters vary across iterations, it is necessary to repeatedly run a full reliability analysis, which may be computationally expensive. The overall cost of this approach is in most cases prohibitive. To alleviate this burden, the two main strategies in this category resort to approximation techniques, more specifically the first-order reliability method (FORM). The reliability index approach (RIA) consists in simply using FORM in the inner loop while replacing the probabilistic constraint by an equivalent reliability index (Nikolaidis and Burdisso, 1988; Lee et al., 2002). An alternative formulation, the so-called performance measure approach (PMA), relies on an inverse FORM analysis in the inner loop that transforms the probabilistic constraints into equivalent target performance measures (Tu et al., 1999; Youn et al., 2005). The main advantages of these approaches lie on the attractivity of FORM which, besides being relatively cheap, allows for an efficient computation of the gradient of the reliability index with respect to the design variables. However, FORM is also the weakness of these approaches as it shows limitations when it comes to problems involving either highly non-linear limit-state functions or multiple failure regions.
Using the same framework for two-level approaches, some contributions have suggested the use of simulation methods in the inner loop (See for instance Royset (2004)). Even when using advanced simulation methods whose cost is relatively small compared to crude Monte Carlo sampling, the overall cost of such approaches is prohibitively high. In general, they are coupled with surrogate models. This aspect is discussed in details later.
3.1.2 Mono-level approaches
Mono-level approaches attempt to solve the RBDO problem by avoiding the reliability analysis and enforcing optimality conditions as initially proposed in Madsen and Hansen (1992). This, in theory, allows one to reduce the computational cost since the failure probability is no more explicitly computed. Kuschel and Rackwitz (1997) introduced the Karush-Kuhn-Tucker (KKT) optimality conditions of the first-order reliability method for the solution of two differently formulated RBDO problems, namely the minimization of cost under reliability constraints and the maximization of reliability under cost constraints. A large number of techniques based on KKT conditions have been proposed since then (See for instance Kharmanda et al. (2002); Agarwal et al. (2007); Kaymaz (2007)). The most popular mono-level techniques are the single loop single vector and single loop approach, which are based on approximating the minimum performance target point using the limit-state function sensitivities (Chen et al., 1997; Liang et al., 2004, 2007).
Despite the gain in efficiency with respect to two-level approaches, mono-level formulations also suffer from some limitations. In the numerical benchmark carried out by Aoues and Chateauneuf (2010), it is shown for instance that SLA and the KKT-based approaches often fail to converge when the starting point of the optimization problem is far from the optimal solution. Lack of robustness of these methods is also observed when target reliability indexes are large or when the design variables are the mean of the random parameters.
3.1.3 Decoupled approaches
As an alternative to mono-level approaches, so-called decoupled approaches have been introduced. They consist in solving the RBDO problem through a sequence of deterministic optimization and reliability analysis. In fact an approximate deterministic optimization problem is solved using information from a previous reliability analysis.
Early contributions in this category suggested to approximate the reliability constraints by introducing Taylor series expansions. Li and Yang (1994) formulated a linear programming problem where at each iteration, the first-order Taylor series expansion of the reliability index evaluated at the previous cycle is used. Royset et al. (2001) introduced an approach which relies on an semiinfinite optimization algorithm. The most popular decoupled approach is the sequential optimization and reliability assessment (SORA) where the probabilistic constraints are translated for deterministic optimization through inverse FORM (Du and Chen, 2004). The associated deterministic optimization problem is obtained by shifting the random variables using the most probable target point found at the previous reliability cycle. Numerous papers have been developed to improve the efficiency of SORA. For instance, Cho and Lee (2011) combine convex linearization to SORA. Cheng et al. (2006) introduced the sequential approximate programming (SAP) which relies on KKT. In SAP, a sub-optimization problem with an approximate objective and constraint functions is solved at each iteration. To avoid running the reliability analysis necessary for obtaining the reliability index and its sensitivities at each cycle, the latter quantities are approximated using a recurrence formula. Zou and Mahadevan (2006) also suggested a similar approach where the Taylor series are used to expand on the failure probability rather than the reliability index. In all these approaches, sensitivities of the failure probability or of the reliability index are necessary. They may be obtained in some cases using analytical derivations. When such analytical derivations are not available, the authors suggest the use of finite differences, though at a loss of efficiency.
In general, decoupled approaches suffer the same drawback as mono-level approaches. Given that the reliability and optimization problems are expected to converge simultaneously, approximation errors in the early cycles may lead the algorithm in the wrong search direction. This is exacerbated when the initial design is far from the optimal solution and when the limit-state function is highly non-linear.
3.2 Use of advanced techniques and simulation in RBDO
The three groups of methods introduced in the previous section rely on approximation methods for computing failure probabilities. The numerous contributions all aimed at reducing the computational burden resulting from a direct solution of the problem. Various applications have shown that their respective efficiency is tightly linked to the problem at hand. To take advantage on the best of each method, some researchers have proposed hybrid formulations which combine methods from the different groups. For instance, Youn (2007); Lim and Lee (2016); Li et al. (2015); Jiang et al. (2017) proposed various approaches that can switch from either single-loop to decoupled or double-loop according to some predefined criteria. Even though such approaches may increase the rate of convergence of these approximation-based methods, they remain flawed because they inherit the well-known limitations of FORM, which can cause inaccuracies or even divergence of the optimization when the limit-state function is highly non-linear or in presence of multiple failure regions. As a consequence, other techniques have been developed in an attempt to overcome these limitations. To deal with high curvatures of the limit-state surfaces, one approach has been the introduction of the second-order reliability method (SORM). This was already discussed in Shetty et al. (1998), while more recently Strömberg (2017) proposed coupling SORM and sequential quadratic programming (SQP). The MPP-based dimension reduction method (Rahman and Xu, 2004) has also been used to improve the accuracy of failure probability estimates w.r.t. FORM (Rahman and Wei, 2008; Lee et al., 2008).
Strategies based on higher-order approximations may attenuate issues due to high-curvature of the limit-state function, however they do not directly tackle the challenges related to the search of the most probable failure or target points (respectively MPP and MPTP). The use of simulation methods in the estimation is in this respect an appropriate solution. Numerous contributions have considered simulation methods in the inner loop of the two-level framework. Due to their high cost, classical simulation techniques are often associated to surrogate models, as discussed in the next section. However, some methods that solely depend on advanced simulation techniques, have been developed. For instance, Rashki et al. (2014) use a weighted simulation method to solve RBDO problems which are however limited to cases with only random design variables (no deterministic and no environmental variables). Beaurepaire et al. (2013) use bridge importance sampling where the idea is to recycle information from previous iterations during the optimization process. Direct integration of the simulation methods to optimization is also possible. This has been done for instance in Royset (2004) where sensitivities of failure probability obtained using Monte Carlo simulation or importance sampling are developed. Finally, Taflanidis and Beck (2008) introduced the so-called stochastic subset optimization (SSO) for RBDO problems where the failure probability appears in the objective function. Upon formulating an augmented reliability problem where the design variables are artificially considered as uncertain, SSO iteratively identifies subsets of the original design space with high likelihood of containing the optimal design. Identifying sub-regions of the design space is a challenging task which involves a non-smooth optimization problem. To bypass this issue, Jia and Taflanidis (2013) proposed to use kernel density estimation to directly approximate the objective function instead of working with subsets. Remaining in the context of augmented reliability problem, Liu and Cheung (2017) recently proposed a similar method where the failure probability, seen as a function of the design parameters, is approximated.
Another reformulation of the RBDO which is based on quantiles has been introduced by Moustapha et al. (2016):
and . Such a formulation has been used in the literature under the name of percentile approach (as used in PMA and SORA) whose solution relies on the so-called minimum target performance point (MPTP), located using inverse FORM. Eq. (3) solves the same problem but in a more direct manner where Monte Carlo simulation is considered for the computation of the quantile. The latter can be seen as an equivalent of the limit-state function evaluated at the MPTP.
All these approaches based on advanced simulation techniques have brought a substantial gain in model evaluation savings, i.e.going from for a direct two-level approach with crude Monte Carlo simulation to calls to the performance function. However, they still remain expensive when considering time-consuming performance function evaluations. For practical applications, efficient methodologies associate simulation methods with surrogate modelling.
3.3 Use of surrogate models in RBDO
The basic idea in surrogate modelling is to replace an expensive-to-evaluate function with a cheap approximation constructed using an experimental design, a.k.a training set, consisting of a limited number of evaluations of the original model. Various surrogate models have been employed in RBDO problems using different solution techniques, i.e.based on simulation or approximation methods. However, the schemes in which they are integrated differ from one contribution to the other. This is summarized in the flowchart of Figure 1.
The first difference that exists between the various approaches in the literature relates to the output function that is actually surrogated. One particular approach consists in directly approximating the relationship between a given design and the corresponding failure probability (resp. reliability index) (branch I. in Figure 1). In other words, the experimental design consists of pairs (resp. ). The optimization can then proceed using any suitable algorithm and replacing the true failure probability with the one estimated by the surrogate model. This approach has been used in various recent contributions. For instance, Foschi et al. (2002) use response surface modelling to approximate the relationship between different design choices and the corresponding reliability indexes computed by importance sampling. More recently, Lehký et al. (2017) developed an approach where the failure probability computed through FORM is approximated using artificial neural networks (a type of surrogate model in itself). In these approaches, one sample of the training set requires a full reliability analysis based on the true model. This becomes problematic when the size of the experimental design is large and/or when the performance functions are expensive-to-evaluate.
A less computationally expensive alternative consists in directly creating an approximation of the performance function and to use it for reliability analysis (branch II. in Figure 1). Again, building on this principle, various schemes have been proposed recently. One obvious approach is to build distinct surrogate models locally used for each reliability analysis in the inner loop of a two-level approach (branch II.1 in Figure 1). Agarwal and Renaud (2004) carry out a double-loop approach where at each iteration, a second-order response surface model is built around the MPP of the previous iteration in order to perform an approximate reliability analysis. Similarly, Papadrakakis et al. (2005) locally trains a neural network and use it to compute a Monte Carlo estimate of the failure probability at each iteration of a double-loop optimization process. This approach is also not optimal as it requires building from scratch a surrogate model at each iteration of the optimization process.
Another option that has been explored is building a single surrogate model that can be used to assess the failure probabilities considering multiple design choices (branch II.2 in Figure 1). The challenge with such an approach is to build an accurate surrogate model over a large area. An important observation though is that there is no need for a surrogate model that is highly accurate globally, as only sub-regions will be explored during optimization. As a consequence, most of the methods developed in this category resort to adaptive sampling in order to ensure accuracy only in regions of interest. Various methods have been developed depending on the space chosen to build the surrogate model. In problems where only design variables are considered to be possibly uncertain, the space over which the surrogate model is built may coincide with the design space (branch II.2.a in Figure 1). In Chen et al. (2015), a Kriging model is built over the whole design space and SORA is used as the RBDO method. Li et al. (2016) solve a RBDO problem using a Kriging model defined on the design space together with importance sampling. In the two contributions, the surrogate models are built adaptively using respectively the importance boundary sampling and a local sampling method based on the MPP.
Other authors have considered to rather build a surrogate model in a sub-region of the space rather than a global one (branch II.2.b in Figure 1). This sub-region, often referred to as local window or trust region, consists of a hypercube in a hybrid space encompassing design and random variables (Kharmanda et al., 2002) whose size and center is updated as the optimization progresses. The difference between the local approximation (branch II.1) and the global approximation with trust region (branch II.2.b) is that in the former, the built surrogate model can only be used with the current value of the design parameters whereas in the latter, the input space allows for the computation of the failure probability for different designs using the same surrogate model. As an example of methods based on trust-region, Lee (1997) builds dynamic Kriging models on local windows and associates this with MCS for the solution of RBDO problems. Other similar contributions include Taflanidis and Medina (2014); Zhang et al. (2017); Gao and Li (2017); Gaspar et al. (2017). Lee et al. (2011); Song (2013) argue that the accuracy of the surrogate model can be increased, specially for high-dimensional problems, if the local window is hyperspherical. They then propose methods respectively based on virtual support vector machines or dynamic Kriging.
In most of the contributions listed above, it is not clear how the design space and the reliability space are explored simultaneously by the surrogate model. A rigorous framework has been established based on the idea of augmented reliability space (Au, 2005). The associated scheme can be classified in the group labeled II.2.a in Figure 1. The original principle is to artificially consider design variables as uncertain and then use conditional sampling and Bayes theorem to compute the failure probability given a realization of the design variables. Adopting this philosophy, Taflanidis (2007); Taflanidis and Beck (2008) formulate a problem where the failure probability lies in the objective function and then construct the stochastic subset optimization algorithm. Taflanidis and Medina (2014); Zhang et al. (2017) solve a similar problem, now using adaptive Kriging and trust regions. Dubourg et al. (2011) also developed the so-called meta-IS approach by combining augmented space, adaptive Kriging and importance sampling. Finally, Moustapha et al. (2016); Moustapha and Sudret (2017) formulate a quantile-based RBDO problem and solve it using Monte Carlo sampling based respectively on Kriging and polynomial chaos expansions models built over an augmented space.
The use of an augmented space allows one to rigorously solve problems where all combination of deterministic/random and design/environmental variables can be considered. Starting from this premise, an original generalized framework for RBDO combining adaptive surrogate modelling and simulation is now proposed.
4 Proposed framework
As shown in the previous review section, there is a tremendous amount of methods developed for the solution of RBDO problems. Early methods were based on local approximations whose aim was to reduce the associated computational burden. Even though these approaches can solve a fair amount of problems, they are limited by their lack of robustness due to the very use of FORM. Recent papers attempted to face these limitations but their computational cost remain relatively large, thus limiting their range of applications to simple toy problems. As a consequence, methods based on simulation methods have been developed to bypass the FORM-related issues. The improvement of accuracy and robustness brought by these methods however comes with of a dramatic increase of the computational cost. They have therefore been associated to surrogate modelling to solve real-world problems where the performance function relies on expensive-to-evaluate models.
Various methodologies that combine surrogate modelling and advanced reliability techniques to solve RBDO problems have been proposed. However as indicated in the previous section, these methods are either able to solve only a limited class of problems or are problem-specific i.e., their implementations require advanced knowledge and specialized derivations. In this paper, we propose a generalized framework based on simulation and adaptive surrogate modelling for the solution of RBDO problems. The interest of the proposed framework can be delineated in the following three points.
First, the framework is based on a formulation where all possible combinations of random/deterministic on the one hand and design/environmental variables on the other hand, can be accounted for at once. Second, this modular framework is non-intrusive, i.e.it consists of three distinct and independent blocks, namely surrogate modelling, structural reliability and optimization. There is no coupling between the different blocks in the basic form of the proposed framework. For instance, the fact that we do not consider the use of analytical gradients for optimization allows us to avoid specific implementations related to the sensitivity of failure probability (or reliability index), which would depend on the reliability technique used. Finally, there is no specific requirement as to which method goes to each block. This will be illustrated in the next section where we will consider applications with different techniques for each of the three blocks namely i.e.Kriging vs. support vector regression, Monte Carlo simulation vs. subset sampling and gradient-based vs. gradient-free optimization algorithms.
4.2 Detailed implementation of the approach
As shown in Figure 2, beside initialization, the proposed framework consists of three different blocks which are now detailed.
4.2.1 Surrogate modelling
Surrogate modelling is in the core of the proposed framework. The basic idea is to replace any expensive-to-evaluate black-box function by an easy-to-evaluate analytical function. This is made possible by assuming that the original model has some accommodating properties such as regularity/continuity and follows a general functional shape. Based on different assumptions, many different surrogate modelling techniques are nowadays available, among which Gaussian process modelling (a.k.a Kriging), polynomial chaos expansions, support vector machines, artificial neural networks, etc. Each of these surrogate models possesses some structure and parameters that need to be calibrated through a learning algorithm on a limited set of data also known as the experimental design or training set: , where is an input of the computational model and is the associated response.
As explained in the previous section, there are various ways of building a surrogate model for RBDO. In this work, we consider building a single surrogate model in an augmented space which spans both the space of random or deterministic design parameters and environmental variables. In most cases, the corresponding input space is a bounded hypercube. The augmented space considered here aims at defining a confidence region over which the surrogate model will be evaluated so as to minimize extrapolation, i.e.evaluation of the surrogate model far from existing training points. More precisely, it consists of a tensor product of confidence regions defined for the design parameters and environmental variables. For the former, we proceed by simply extending the deterministic bounds on the design parameters in each dimension to reduce to an acceptable level the probability of sampling outside this area for designs close to bounds. Let us denote these probabilities by and respectively for the lower and upper bounds in each dimension . The bounds of the associated augmented space in the -th dimension therefore read:
where and are respectively the inverse cumulative distribution functions (CDF) of the random variables associated to lower and upper bounds of , denoted by and . The design confidence region is eventually obtained by the following tensorization:
For an environmental variable, the marginal confidence region is obtained by defining a symmetric bounded area (with respect to the probability content) around its mean value. By defining the above probability by , the marginal confidence region can be obtained as follows:
where is the inverse CDF associated to the environmental variable . The associated confidence region therefore reads:
Note that for the environmental variables, the construction of is not limited to this hypercube. In fact, since the environmental variables do not evolve during the optimization, the original support of these random variables can be directly used as marginal confidence region . Also, transformed variables in the usual standard normal space could be considered. In this paper, we however consider the construction in (8), which to the authors experience leads to more accurate surrogate models.
Finally, the augmented space follows as a tensor product of the defined marginal confidence regions, i.e.:
An illustration of an augmented space for a three-dimensional problem is shown in Figure 3. This problem consists of two design- and one environmental variables. Design parameter is the mean value of random variable , while design parameter is deterministic. This is directly accounted for in the construction of by extending the bounds in the first dimension only, i.e. and . The flexibility of this construction allows one to consider all possible combinations of deterministic or random variables and the presence or not of random environmental variables.
Once the augmented space is defined, the next step is to sample points that will be evaluated using the original model in order to generate a training set. A common approach is to sample the space uniformly. This can be achieved using stratified sampling approaches such as the well-known Latin hypercube sampling (McKay et al., 1979) or low-discrepancy sequences such as the Sobol’ sequence (Sobol’, 1967). To reach the level of surrogate model accuracy necessary for a valid RBDO solution, the space needs to be densely sampled. Unfortunately, in some cases the resulting training set would be so large that it would defy the purpose of using surrogates in the first place. However, it can be observed that the accuracy of the surrogate model does not necessarily need to be large over the entire space but only in some limited regions of interest. This observation gave rise to approaches known as adaptive design or active learning, where the idea is to start with a small training set and then gradually increase its size by strategically adding points to improve the accuracy of the surrogate model following the requirements of the analysis at hand (reliability analysis or optimization).
When a limit-state function is approximated, the region of interest lies in the vicinity of the limit-state surface. Various enrichment methods exist in this perspective. By various means, they all aim to explore the space i.e.adding points in areas which are scarce in data and exploit the space by focusing on the vicinity of the approximated limit-state surface.
Active learning is used in this framework to reduce the number of calls to the original model. This allows one to use a single global surrogate model for the reliability analyses in each iteration of the optimization process.
4.3 Structural reliability analysis
The structural reliability analysis block is considered as a black-box module in the proposed framework. This means that any method to estimate failure probabilities that is judged most suitable for the problem at hand can be used. In this work, focus is given to simulation methods rather than approximation techniques due to the fact that the limit-state surface is an inexpensive-to-evaluate surrogate model. The former are known to yield more accurate results, at the expense of a dramatically increased computational cost though. Naturally, crude Monte Carlo simulation is the most robust approach when it comes to estimating failure probabilities. However the variance of the estimates decays very slowly with respect to the number of samples. This becomes even more problematic in the presence of a rare event (i.e.with an extremely low failure probability). Advanced simulation methods such as subset simulation (Au and Beck, 2001; Papaioannou et al., 2015), cross-entropy importance sampling (Kurtz and Song, 2013; Geyer et al., 2019) or line sampling (Pradlwarter et al., 2007; de Angelis et al., 2015) that try to achieve a faster decrease of the variance of with diminishing values of the target failure probability have been developed and shall be used whenever justified.
4.4 Design optimization
Global vs. local
As stated earlier, any optimization algorithm can be plugged non-intrusively in the proposed framework. In general, one must distinguish so-called global from local search methods. In the former, the entire design space is explored by sampling points according to a mechanism proper to the algorithm. This sampling procedure is expected to converge towards the region with the highest likelihood of containing the optimal solution. In contrast, the latter starts with an initial guess and generates a sequence of improved designs using some local information on the cost function. The sequence is expected to converge to a local minimizer which cannot be guaranteed to be the global one, except if some conditions are met, e.g. the function is convex. In contrast, global algorithms possess internal mechanisms that increase the likelihood of reaching the global optimizer, at usually a typical higher cost w.r.t. local optimizers.
A peculiar aspect of the RBDO problem is that the constraints are stochastic in nature. In such a context, the computation of the sensitivities required by gradient-based approaches is not trivial. For some reliability methods, it is possible to derive the sensitivities of the failure probability with respect to the design variable, for instance using the so-called score functions together with crude Monte Carlo simulation. Sensitivities of the cost with respect to the design variables through analytical functions can improve the efficiency of the optimization algorithm. However, such information may not be available and in any case it is not general, i.e.it depends on the reliability method or distribution functions of the random variables. This defies the purpose of providing a generalized non-intrusive framework.
In this work, we rather suggest the numerical estimation of the sensitivities. One widespread approach relies on finite-differences where the gradient can be estimated at the cost of (forward or backward) or (central finite difference) evaluations of the original model. One of the issues with this approach is the stochasticity in the failure probability. In fact, the noise inherent to each estimation of or in the finite difference procedure (due to the finite sets of Monte Carlo sample) could make it difficult to properly estimate the gradient, as one cannot anymore differentiate the information given by the actual perturbation of the design from that due to random noise. To avoid this issue, it is possible to resort to the so-called common random numbers approach (Spall, 2003a). This consists in introducing a consistent error between the estimates of the failure probabilities for two different designs. In practice, this may be achieved by using the same stream of random numbers to generate the samples needed to estimate . Exterior sampling approach (ESA) is similar to CRN as the same stream of random numbers is used throughout the entire optimization process (Taflanidis and Beck, 2008). This actually transforms the stochastic optimization problem into a deterministic one that can be solved using any general-purpose optimization algorithm. However, the solution may be biased due to the introduction of the consistent error. This can be accounted for by either increasing the sample size or repeating the optimization procedure multiple times.
Stochastic approximation methods
The use of common random numbers or ESA however, does not solve all challenges related to RBDO using finite-difference-based approaches. For instance, the finite difference step size should be consistent with the variance of the estimated failure probability. Besides, for some advanced simulation methods such as subset simulation or importance sampling, the use of the ESA does not necessarily translate into a deterministic optimization problem. In fact the topology in the reliability analysis can change drastically between two infinitesimally close designs. An alternative approach to approximate gradients is based on simultaneous perturbation (Spall, 1998b, a) where, instead of varying one component of the design at a time (as in finite-difference), one can simultaneously and randomly perturb all components of the design. The simultaneous perturbation stochastic approach (SPSA) (Spall, 2003b) is based on this principle and has the advantage of requiring only two function evaluations for each gradient estimate, regardless of the problem dimension. Taflanidis and Beck (2008) explore its use in RBDO where the formulation consists solely in minimizing the failure probability. This does not directly apply to the formulation considered here as the stochastic part is in the constraints. One may however consider constrained formulations of SPSA as proposed for instance in Wang (1998, 2003).
In our framework, we focus more on global stochastic algorithms. The reasons are two-fold. First, as discussed earlier, such methods have a greater likelihood to find the global optimizer. The associated increase in computational effort can be leveraged by the fact that the cost function is a simple analytical function and that the failure probability estimate is based on a surrogate model. Second, the fact that only first-order information is needed allows one to avoid issues related to computing sensitivities of the failure probability which can be, at best, inefficient.
5 Case studies
In this section, we will consider three different configurations of the proposed framework to solve a set of benchmark problems. For each configuration case, we will consider different methods in each block as detailed in Table 1. The reference solution corresponds to actually using the original model associated to a large Monte Carlo set for the reliability analysis. The optimization algorithm is hybrid covariance matrix adaptation - evolution scheme (CMA-ES), meaning that we start with the CMA-ES (Hansen and Ostermeier, 2001; Arnold and Hansen, 2012) to locate the global minimum and refine the found solution using a gradient-based algorithm. The entire analysis is carried out in UQLab, a Matlab-based framework for uncertainty quantification (Marelli and Sudret, 2014). The classical algorithms presented in the review are available in the RBDO module of UQLab together with the proposed algorithm (Moustapha et al., 2019).
|Framework||Surrogate model||Reliability method||Optimization algorithm|
|Case||SVR||Monte Carlo simulation||SQP|
|Case||Kriging||Quantile Monte Carlo||CMA-ES with enrichment|
|Reference||Original model||Monte Carlo simulation||Hybrid CMA-ES|
In the first case, we consider adaptive support vector regression as a surrogate model, crude Monte Carlo simulation for the reliability analysis and the sequential quadratic programming (SQP) as provided by Matlab’s fmincon as the gradient-based optimization algorithm. As the latter two methods are standard, we focus only on support vector regression and its adaptive enrichment in the sequel.
Basics of SVR
Support vector regression (Vapnik, 1995; Smola and Schölkopf, 2004; Moustapha et al., 2018) is a machine learning technique often used for surrogate modelling in structural reliability. Given a training set as defined in Section 4.2.1, the SVR prediction for a new point reads:
where is a set of unknown coefficients, is an unknown bias term and is a predefined kernel function. The unknown parameters are found by minimizing a regularized loss function. In this paper, we consider the quadratic -insensitive loss function defined by if and otherwise. In practice, the coefficients correspond to Lagrange multipliers that can be found by minimizing the following functional (Bourinet, 2018):
where with and being respectively the Gram and identity matrices of size .
Eq. (11) is a convex quadratic optimization programming problem which can be solved using any specialized solver. The remaining bias term in Eq. (10) can be obtained as by-product of the solution of this problem.
In general, one needs to calibrate the model i.e.find the optimal hyper-parameters, namely the penalty term , the -insensitive tube width and the parameter(s) of the chosen kernel function. This is done here by minimizing an approximation of the leave-one-out (LOO) cross-validation error, more specifically the so-called span estimate (Chapelle et al., 2002). The LOO error is aimed at approximating the surrogate model generalization error. The overall optimization problem is carried out here using the covariance matrix adaptation-evolution scheme (CMA-ES) (Hansen and Ostermeier (2001), see also Bourinet (2018); Moustapha et al. (2018)).
Adaptive construction of the training set
Various methods have been proposed for the adaptive construction of support vector machines in the context of structural reliability. We consider a modified version of the method proposed in Basudhar and Missoum (2008) where the authors suggest to solve the following maximin problem:
In other words, this problem consists in finding the point from the currently approximated limit-state surface (exploitation) that is the furthest away from the existing training set (exploration).
Finding the next point to add however consists in solving a continuous maximin problem which may be expensive. In this paper, we propose to discretize this formulation. In this respect, we first sample a set of candidates for enrichment . Then a subset of this set consisting of the closest points to the limit-state surface is selected. Finally the next best point is chosen by adapting Eq. (12), i.e.:
where with being an -quantile of . In this paper, we set .
Two convergence criteria are considered here. The first first one is related to the stability of the failure domain within iterations of enrichment and reads:
where is the number of failed samples in as computed at the -th enrichment iteration.
The second criterion relates to the stability of failed vs. safe predictions: the surrogate model is assumed to be good enough when the number of points that are predicted in the current safe domain while it belonged to the failure domain in the previous iteration (and vice-versa) is low. This can be written as:
where denotes the surrogate model at the th iteration.
Convergence is assumed in the following applications when these two criteria are satisfied in two iterations in a row. The thresholds are set to and in the case studies presented in Section 6.
Basics of Kriging
Kriging (Santner et al., 2003; Rasmussen and Williams, 2006; Lataniotis et al., 2018) is a surrogate modelling technique which considers the model as a realization of a stochastic Gaussian process that can be cast as:
where the so-called trend consists of a linear combination of preselected basis functions and is a zero-mean stationary Gaussian process with auto-covariance . In the last equation, is a constant variance of the process and is a preselected auto-correlation function with parameters .
The calibration of this model consists in estimating the regression coefficients
and the parameters of the selected auto-correlation function. The weight coefficients can be estimated through least-square regression while the auto-correlation parameters can be estimated through cross-validation and maximum likelihood estimation (Bachoc, 2013). Once the calibration is done, the prediction for a new point is assumed to follow a Gaussian distribution i.e. , with
where is the generalized least-square estimate of the weight coefficients , is a vector of cross-correlations between the point and each point of the training set with components . The following notations have been introduced for the sake of clarity: and .
Adaptive construction of the training set
In this contribution, the learning function introduced by Bichon et al. (2008) is considered. This consists in choosing the point which maximizes the so-called expected feasibility function as the next point to add in the experimental design:
In the case , we consider the quantile-based formulation introduced in Eq. (3). Furthermore, a slight modification in the enrichment scheme is introduced in order to increase the performance compared to case . In fact, the enrichment procedure introduced above seeks to produce a surrogate model that is accurate in the vicinity of the limit-state surface in the entire augmented space. However optimization is the ultimate goal of the proposed methodology: in this context, only a subset of the space is of interest, i.e.where the cost function is minimal. To take advantage of this, we consider a two-stage enrichment scheme as originally proposed in Moustapha et al. (2016). In the first stage, enrichment is done as described in Case but with a looser convergence criterion. Herein, the thresholds and in Eqs. (14) and (15) are both increased to . The idea behind is to locate the limit-state surface without necessarily focusing on extreme accuracy. The remaining epistemic uncertainty due to surrogate modelling is taken care of in the second stage where optimization is coupled with enrichment. The procedure consists in ensuring accuracy of the estimated failure probabilities, or herein quantiles, at each iteration of the optimization process. For this, we consider a metric introduced in Dubourg et al. (2011) and adapted for quantile estimation in Moustapha et al. (2016) where an upper and lower bounds of the quantiles estimates (resp. and ) are introduced using the models defined respectively by and . It can be shown that
Even though this bound does not indicate where the true quantile lies, it is a measure of how accurate the estimated quantile is with respect to the Kriging epistemic uncertainty. Therefore, at each iteration of the optimization process, the following criterion is checked:
where is a threshold to be defined. If this condition is not respected, then enrichment is made locally using the procedure introduced in Section 5.2 where the candidates for enrichment are simply the set of points that are locally used to compute the failure probability.
For this special application using CMA-ES, the criterion is computed only for sampled points that are feasible and improve the current best point. This is to avoid enrichment in the exploratory part of CMA-ES. Other tricks can be used to make the procedure more robust and efficient (e.g.a threshold that decreases as the optimization progresses) but this is not in the scope of this paper. With this slight modification, Case does not exactly illustrate the proposed framework and is qualified here as gray-box because of the coupling between optimization and surrogate modelling. Its main interest however is that it greatly enhances the efficiency of the solution.
6 Application examples
In this section, the three configurations of the proposed framework are illustrated using three examples of increasing complexity. Furthermore, the versatility of the surrogate framework is shown by considering the different classes of probabilistic inputs which consist in a) design variables with uncertainty only, b) design and environmental variables and c) deterministic design parameters and random environmental variables.
6.1 Two-dimensional highly non-linear limit-state function
In this first example, we consider a two-dimensional highly non-linear function for which the RBDO problem reads (Lee and Jung, 2008):
where . The reference solution for this problem is corresponding to an optimal design . This solution is found using a direct double-loop approach on the original functions with a large size Monte Carlo simulation in the inner loop (see also Dubourg (2011)).
To solve this problem, an initial experimental design of size is drawn using Latin hypercube sampling (LHS). The solution is repeated times to account for the inherent statistical variability of the different solution schemes. Figure 4 shows boxplots of the found solutions (left panel) together with the total number of model evaluations (right panel) for each case. The results are also summarized in Table 2 together with some benchmark solutions for comparison purposes. It can be observed that the three framework configurations lead to an accurate solution at a reasonable cost. In fact the number of calls to the original model is smaller than approximation approaches such as PMA as reported by Lee and Jung (2008) or the Kriging-based RBDO approach proposed in Dubourg et al. (2011). The larger number reported in Dubourg et al. (2011) can be explained by the fact that they also approximate the first constraint by a surrogate model.
|Performance measure approach (PMA)|
As calculated in Lee and Jung (2008)
As calculated in Dubourg (2011)
Median values found from replications
Let us now look at the convergence of each framework configuration. Figure 5 shows the final surrogate models and the experimental design points for the median solution of each solving strategy. The entire panel window shows the augmented space while the design space is limited by the inner black rectangle. As explained earlier, the augmented space in this case is simply obtained by extending the bounds of the design space in each direction. The limit-state surface as computed by the original model is drawn with the thick blue line and the gray shaded area represents the unfeasible space due to the soft constraint. Figure (a)a show additionally contours of the cost function and the reference solution represented by the magenta diamond. In the remaining panels, the initial experimental design is shown by the blue squares. The red circles correspond to the enrichment points added prior to starting the optimization process. The approximated limit-state surface is represented by the dashed red line. The cases and show approximately the same enrichment scheme. The case where SVR together with the maximin enrichment scheme are used (Figure (b)b) produces the most uniform sampling with almost all the new points located near to the final limit-state surface. The framework configuration (Figure (c)c) produces a slightly less accurate Kriging approximation. The case (Figure (d)d) corresponds to the coupling of enrichment with optimization. In fact, only two points are added at the first stage of enrichment. The green diamonds show the points that are added during optimization. Almost all the points added in this stage are located around the solution. The final Kriging model is not accurate everywhere in the space but only in areas of the limit-state surface around the optimum. Overall, this allows the approach to converge to the solution with fewer number of model evaluations. Particularly, very few points are added where the cost function is large or in unfeasible regions due to the soft constraints.
6.2 Short column under oblique bending
This benchmark deals with a short column structure of rectangular cross-section which is subject to an axial load and biaxial bending moments and . The limit-state function characterizes the performance of the structure with respect to its yield stress and reads:
The probabilistic model is defined in Table 3. Note that the uncertainty in the design parameters has been artificially introduced by Dubourg (2011) to accommodate his method. Even though in the proposed framework such trick is not necessary, we keep the same problem formulation so as to be able to compare the results.
The optimization is started with an initial experimental design of size . For this six-dimensional example, the convergence criteria do not lead to robust solutions with respect to the replications. As shown in Table 4, the median number of calls to the limit-state surface are respectively and for the cases and . The corresponding solutions however show some scatter around the reference value. The case allows us to reduce both the solution scatter and the average number of model evaluations.
As calculated in Lee and Jung (2008)
As calculated in Dubourg (2011)
Median values found from the replications
The constraints are not saturated in the other references because their RBDO formulation is slightly different:
6.3 Dome structure
This final example is related to the optimization of the dome structure introduced by Kaveh and Talatahari (2009), albeit in a deterministic context. This problem is extended here to account for uncertainties in the loading as proposed by Torre et al. (2018). The dome, illustrated in Figure 7, consists of bars divided into seven groups as numbered in panel A. The dome surface is divided into seven sectors to which a Gumbel-distributed load is assigned with the following mean values: kN/m in the top and north-east sectors A, B and F, kN/m in the north-west and south-east sectors C, E, G and I and finally kN/m in the south-west sectors D and H. The coefficient of variation is in all cases. These loads are supposed to model snow falling from north-east direction (Torre et al., 2018). In addition, self-weight and deterministic service loads are applied to each node of the structure as follows: kN to node , kN on nodes to and kN on nodes to (Kaveh and Talatahari, 2009). The node groups and sectors are illustrated using different color schemes (respectively shades of blue and red).
The optimization problem consists in minimizing the weight of the structure with design variables being the bars cross-sectional areas cm while ensuring that the vertical displacement of node remains below a given threshold cm. Accounting for uncertainties, the target failure probability is set to .
The problem is solved using only the framework configuration which has proven to be the most efficient in the other applications. An initial experimental design of points is drawn using optimal Latin hypercube sampling. The first stage of enrichment is then carried out and convergence is achieved after two iterations only. The final solution is achieved with a total of model evaluations. The optimal weight is found to be tons. Figure 8 shows the relative areas with respect to lower and upper bounds for each design dimension. The largest section is attributed to the group of bars labeled which is close to the maximum allowable size. The two groups of bars which are not in the radial direction and not connected to node do not really influence the deflection and are hence set close to their lower bounds by the optimizer.
This result given above assumes that the surrogate model is accurate enough. In order to validate the surrogate model accuracy in the vicinity of the limit-state surface, a Monte Carlo set of points is drawn using the final design, i.e.the design components are those of the optimal design while the environmental variables follow their own distributions. Since the constraints are saturated, the estimated quantiles should be equal to cm. Figure 9 shows the boxplot of the quantiles with bootstrap replications when using the final Kriging model and the original model. Bootstrap is used here to account for the small size of the validation Monte Carlo set. Accounting for the bootstrap standard deviation, the targeted threshold belongs to a small confidence interval around the estimated quantiles. Thus the final Kriging model is deemed accurate enough considering this validation set even though it seems to slightly underestimate the true quantile. Note that a better fit could be obtained by calibrating the enrichment convergence criterion to allow for more enrichment points.
Finally it is worth emphasizing that only finite element runs were used to solve a -dimensional RBDO problem.
This paper aims at introducing a global and unified framework for the solution of reliability-based design optimization (RBDO) problems. A review of classical approximation-based approaches is first carried out following the widely accepted classification into double-loop, single-loop and decoupled methods similar to Chateauneuf (2008). Upon stressing the limitations of such methods, more recent approaches that combine simulation techniques for the reliability analysis and surrogate-modelling are further reviewed. These methods have been developed mainly since 2010. They are classified here according to the way the surrogate model is integrated into the RBDO framework.
Then, a global framework that combines three independent blocks, namely adaptive surrogate modelling, reliability analysis and optimization, is proposed together with some numerical considerations, e.g. on the gradients estimation. It is argued that the proposed framework is modular and non-intrusive, meaning that each block can be set independently from the others. Hence the analyst can freely choose his favorite surrogate modelling, reliability analysis and optimization method.
Finally, an illustration of the proposed framework is made using different combinations for each block. Each of the three examples treats an RBDO formulation with the three classes of probabilistic inputs: only random design parameters which appears as parameters of random variables; deterministic design- and random environmental variables and eventually the combination of all types of variables. All these cases can be treated in the same way through the use of an augmented space. It is worth emphasizing that most of the current literature focus on one formulation and require adaptations to treat other scenarios. The results for the three examples are compared to literature references. Efficiency of the framework is demonstrated by the number of calls to the performance functions, which is in the same range as other surrogate-based methods in the literature. As expected, the efficiency and robustness is much larger than approaches that rely on approximation methods such as FORM for the reliability analysis. However results are dependent on the convergence of the enrichment scheme which require a proper calibration. The proposed general framework is currently integrated as a new module (Moustapha et al., 2019) of the uncertainty quantification platform UQLab (Marelli and Sudret, 2014), which will ensure further dissemination of surrogate-based RBDO methods.
- Agarwal et al. (2007) Agarwal, H., C. K. Mozumder, J. E. Renaud, and L. T. Watson (2007). An inverse-measure-based unilevel architecture for reliability-based design optimization. Struct. Multidisc. Optim. 33(3), 217–227.
- Agarwal and Renaud (2004) Agarwal, H. and J. Renaud (2004). Reliability-based design optimization using response surfaces in application to multidisciplinary systems. Eng. Opt. 36(3), 291–311.
- Aoues and Chateauneuf (2010) Aoues, Y. and A. Chateauneuf (2010). Benchmark study of numerical methods for reliability-based design optimization. Struct. Multidisc. Optim. 41(2), 277–294.
- Arnold and Hansen (2012) Arnold, D. V. and N. Hansen (2012). A (1+1)-CMA-ES for constrained optimisation. In T. Soule and J. H. Moore (Eds.), Genetic and evolutionary computation conference, pp. 297–304.
- Au (2005) Au, S.-K. (2005). Reliability-based design sensitivity by efficient simulation. Comput. Struct. 83(14), 1048–1061.
- Au and Beck (2001) Au, S. K. and J. L. Beck (2001). Estimation of small failure probabilities in high dimensions by subset simulation. Prob. Eng. Mech. 16(4), 263–277.
- Bachoc (2013) Bachoc, F. (2013). Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model misspecifications. Comput. Stat. Data Anal. 66, 55–69.
- Basudhar and Missoum (2008) Basudhar, A. and S. Missoum (2008). An improved adaptive sampling scheme for the construction of explicit boundaries. Struct. Multidisc. Optim. 42(4), 517–529.
- Beaurepaire et al. (2013) Beaurepaire, P., H. A. Jensen, G. I. Schuëller, and M. A. Valdebenito (2013). Reliability-based optimization using bridge importance sampling. Prob. Eng. Mech. 34, 48–57.
- Beck and Gomes (2012) Beck, A. T. and W. J. S. Gomes (2012). A comparison of deterministic, reliability-based and risk-based structural optimization under uncertainty. Prob. Eng. Mech. 28, 18–29.
- Bichon et al. (2008) Bichon, B. J., M. S. Eldred, L. Swiler, S. Mahadevan, and J. McFarland (2008). Efficient global reliability analysis for nonlinear implicit performance functions. AIAA Journal 46(10), 2459–2468.
- Bourinet (2018) Bourinet, J.-M. (2018). Reliability analysis and optimal design under uncertainty - Focus on adaptive surrogate-based approaches. Université Blaise Pascal, Clermont-Ferrand, France. Habilitation à diriger des recherches, 243 pages.
- Chapelle et al. (2002) Chapelle, O., V. Vapnik, and Y. Bengio (2002). Model selection for small sample regression. Machine Learning 48(1), 9–23.
- Chateauneuf (2008) Chateauneuf, A. (2008). Structural design optimization considering uncertainties, Chapter 1, pp. 3–30. Taylor & Francis.
- Chen et al. (1997) Chen, X., K. Hasselman, T., and D. J. Neil (1997). Reliability-based structural design optimization for practical applications. In 38th Structures, Structural Dynamics, and Materials Conference, pp. 2724–2732.
- Chen et al. (2015) Chen, Z., S. Peng, X. Li, H. Qiu, H. Xiong, L. Gao, and P. Li (2015). An important boundary sampling method for reliability-based design optimization using Kriging model. Struct. Multidisc. Optim. 52(1), 55–70.
- Cheng et al. (2006) Cheng, G., L. Xu, and L. Jiang (2006). A sequential approximate programming strategy for reliability-based structural optimization. Comput. Struct. 84(21), 1353–1367.
- Cho and Lee (2011) Cho, T. M. and B. C. Lee (2011). Reliability-based design optimization using convex linearization and sequential optimization and reliability assessment method. Structural Safety 33(1), 42–50.
- de Angelis et al. (2015) de Angelis, M., E. Patelli, and M. Beer (2015). Advanced line sampling for efficient robust reliability analysis. Structural Safety 52(B), 170–182.
- Ditlevsen and Madsen (1996) Ditlevsen, O. and H. Madsen (1996). Structural reliability methods. J. Wiley and Sons, Chichester.
- Du and Chen (2004) Du, X. and W. Chen (2004). Sequential optimization and reliability assessment method for efficient probabilistic design. J. Mech. Design 126(2), 225–233.
- Dubourg (2011) Dubourg, V. (2011). Adaptive surrogate models for reliability analysis and reliability-based design optimization. Ph. D. thesis, Université Blaise Pascal, Clermont-Ferrand, France.
- Dubourg et al. (2011) Dubourg, V., B. Sudret, and J.-M. Bourinet (2011). Reliability-based design optimization using Kriging and subset simulation. Struct. Multidisc. Optim. 44(5), 673–690.
- Enevoldsen and Sørensen (1994) Enevoldsen, I. and J. D. Sørensen (1994). Reliability-based optimization in structural engineering. Structural Safety 15(3), 169–196.
- Foschi et al. (2002) Foschi, R. O., H. Li, and J. Zhang (2002). Reliability and performance-based design: a computational approach and applications. Structural Safety 24(2–4), 205–218.
- Frangopol (1985) Frangopol, D. M. (1985). Structural optimization using reliability concepts. J. Struct. Eng. 111(11), 2288–2301.
- Frangopol and Maute (2003) Frangopol, D. M. and K. Maute (2003). Life-cycle reliability-based optimization of civil and aerospace structures. Comput. Struct. 81, 397–410.
- Gao and Li (2017) Gao, T. and J. Li (2017). A derivative-free trust-region algorithm for reliability-based optimization. Struct. Multidisc. Optim. 55(4), 1535–1539.
- Gaspar et al. (2017) Gaspar, B., A. P. Teixeira, and C. Guedes Soares (2017). Adaptive surrogate model with active refinement combining Kriging and a trust region method. Reliab. Eng. Syst. Saf. 165, 277–291.
- Geyer et al. (2019) Geyer, S., I. Papaiannou, and D. Straub (2019). Cross-entropy-based importance sampling using gaussian densities revisited. Structural Safety 76, 15–27.
- Hansen and Ostermeier (2001) Hansen, N. and A. Ostermeier (2001). Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195.
- Hilton and Feigen (1960) Hilton, H. H. and M. Feigen (1960). Minimum weight analysis based on structural reliability. J. aerospace sci. 27(9), 641–652.
- Jia and Taflanidis (2013) Jia, G. and A. A. Taflanidis (2013). Non-parametric stochastic subset optimization for optimal-reliability design problems. Comput. Struct. 126, 86–99.
- Jiang et al. (2017) Jiang, C., H. Qiu, L. Gao, X. Cai, and P. Li (2017). An adaptive hybrid single-loop method for reliability-based design optimization using iterative control strategy. Struct. Multidisc. Optim., 1–16.
- Kaveh and Talatahari (2009) Kaveh, A. and S. Talatahari (2009). Particle swarm optimizer, and colony strategy and harmony search scheme hybridized for optimization of truss structures. Comput. Struct. 87, 1267–283.
- Kaymaz (2007) Kaymaz, I. (2007). Approximation methods for reliability-based design optimization problems. GAMM-Mitt 30(2), 225–268.
- Kharmanda et al. (2002) Kharmanda, G., A. Mohamed, and M. Lemaire (2002). Efficient reliability-based design optimization using a hybrid space with application to finite element analysis. Struct. Multidisc. Optim. 24(3).
- Kurtz and Song (2013) Kurtz, N. and J. Song (2013). Cross-entropy-based adaptive importance sampling using gaussian mixture. Structural Safety 42, 35–44.
- Kuschel and Rackwitz (1997) Kuschel, N. and R. Rackwitz (1997). Two basic problems in reliability-based structural optimization. Math. Method. Oper. Res. 46(3), 309–333.
- Lataniotis et al. (2018) Lataniotis, C., S. Marelli, and B. Sudret (2018). The gaussian process modeling module in UQLab. Soft Computing in Civil Engineering 2(3), 91–116.
- Lee et al. (2008) Lee, I., K. K. Choi, L. Du, and D. Gorsich (2008). Inverse analysis method using MPP-based dimension reduction for reliability-based design optimization of nonlinear and multi-dimensional systems. Comput. Methods Appl. Mech. Engrg. 198, 14–27.
- Lee et al. (2011) Lee, I., K. K. Choi, and L. Zhao (2011). Sampling-based RBDO using the stochastic sensitivity analysis and dynamic Kriging method. Struct. Multidisc. Optim. 44(3), 299–317.
- Lee et al. (2002) Lee, J.-O., Y.-S. Yang, and W.-S. Ruy (2002). A comparative study of reliability-index and target-performance-based probabilistic structural design optimization. Comput. Struct. 80, 257–269.
- Lee (1997) Lee, P. M. (1997). Bayesian statistics An introduction, second edition. Arnold.
- Lee and Jung (2008) Lee, T. and J. Jung (2008). A sampling technique enhancing accuracy and efficiency of metamodel-based RBDO: Constraint boundary sampling. Comput. Struct. 86(13-14), 1463–1476.
- Lehký et al. (2017) Lehký, D., O. Slowik, and D. Novák (2017). Reliability-based design: Artificial neural networks and double-loop reliability-based optimization approaches. Adv. Eng. Soft., 1–13.
- Li et al. (2015) Li, G., Z. Meng, and H. Hu (2015). An adaptive hybrid approach for reliability-based design optimization. Struct. Multidisc. Optim. 51(5), 1051–1065.
- Li and Yang (1994) Li, W. and L. Yang (1994). An effective optimization procedure based on structural reliability. Comput. Struct. 52(5), 1061–1067.
- Li et al. (2016) Li, X., H. Qiu, Z. Chen, L. Gao, and X. Shao (2016). A local Kriging approximation method using MPP for reliability-based design optimization. Comput. Struct. 162, 102–115.
- Liang et al. (2004) Liang, J., Z. Mourelatos, and J. Tu (2004). A single-loop method for reliability-based design optimization. In Proc. DETC’04 ASME 2004 Design engineering technical conferences and computers and information in engineering conference, Sept.28 - Oct. 2, 2004, Salt Lake City, Utah, USA.
- Liang et al. (2007) Liang, J., Z. P. Mourelatos, and E. Nikolaidis (2007). A single-loop approach for system reliability-based design optimization. J. Mech. Des. 129(12), 1215 – 1224.
- Lim and Lee (2016) Lim, J. and B. Lee (2016). A semi-single-loop method using approximatino of most probable point for reliability-based design optimization. Struct. Multidisc. Optim. 53(4), 745–757.
- Liu and Cheung (2017) Liu, W.-S. and S. H. Cheung (2017). Reliability based design optimization with approximate failure probability function in partitioned design space. Reliab. Eng. Syst. Saf. 167, 602–611.
- Madsen and Hansen (1992) Madsen, H. O. and P. F. Hansen (1992). A comparison of some algorithms for reliability based structural optimization and sensitivity analysis. In R. Rackwitz and P. Thoft-Christensen (Eds.), Reliability and Optimization of Structural Systems’91. Lectures Notes in Engineering, Volume 76, pp. 443–451. Springer, Berlin, Heidelberg.
- Marelli and Sudret (2014) Marelli, S. and B. Sudret (2014). UQLab: A framework for uncertainty quantification in Matlab. In Vulnerability, Uncertainty, and Risk (Proc. 2nd Int. Conf. on Vulnerability, Risk Analysis and Management (ICVRAM2014), Liverpool, United Kingdom), pp. 2554–2563.
- McKay et al. (1979) McKay, M. D., R. J. Beckman, and W. J. Conover (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 2, 239–245.
- Moustapha et al. (2018) Moustapha, M., C. Lataniotis, S. Marelli, and B. Sudret (2018). UQLab user manual – Support vector machines for regression. Technical report, Chair of Risk, Safety & Uncertainty Quantification, ETH Zurich. Report # UQLab-V1.1-111.
- Moustapha et al. (2019) Moustapha, M., S. Marelli, and B. Sudret (2019). UQLab user manual – Reliability-based design optimization. Technical report, Chair of Risk, Safety & Uncertainty Quantification, ETH Zurich. Report # UQLab-V1.2-114.
- Moustapha and Sudret (2017) Moustapha, M. and B. Sudret (2017). Quantile-based optimization under uncertainties using bootstrap polynomial chaos expansions. In Proc. 12th Internatinoal Conference on Structural Safety and Reliability (ICOSSAR), August 6-10, 2017, Vienna, Austria.
- Moustapha et al. (2016) Moustapha, M., B. Sudret, J.-M. Bourinet, and B. Guillaume (2016). Quantile-based optimization under uncertainties using adaptive Kriging surrogate models. Struct. Multidisc. Optim. 54(6), 1403–1421.
- Moustapha et al. (2018) Moustapha, M., B. Sudret, J.-M. Bourinet, and B. Guillaume (2018). Comparative study of Kriging and support vector regression for structural engineering applications. ASCE-ASME J. Risk Uncertainty Eng. Syst., Part A: Civ. Eng. 4(2).
- Nikolaidis and Burdisso (1988) Nikolaidis, E. and R. Burdisso (1988). Reliability based optimization: A safety index approach. Comput. Struct. 28(6), 781–788.
- Papadrakakis et al. (2005) Papadrakakis, M., N. D. Lagaros, and V. Plevris (2005). Design optimization of steel structures considering uncertainties. Eng. Struct. 27(9), 1408–1418.
- Papaioannou et al. (2015) Papaioannou, I., W. Betz, K. Zwirglmaier, and D. Straub (2015). MCMC algorithms for subset simulation. Prob. Eng. Mech. 41, 89 – 103.
- Pradlwarter et al. (2007) Pradlwarter, H. J., G. I. Schuëller, P. S. Koutsourelakis, and D. C. Charmpis (2007). Application of line sampling simulation method to reliability benchmark problems. Structural Safety 29(3), 208–221.
- Rahman and Wei (2008) Rahman, S. and D. Wei (2008). Design sensitivity and reliability-based structural optimization by univariate decomposition. Struct. Multidisc. Optim. 35(3), 245–261.
- Rahman and Xu (2004) Rahman, S. and H. Xu (2004). A univariate dimension-reduction method for multi-dimensional integration in stochastic mechanics. Prob. Eng. Mech. 19, 393–408.
- Rashki et al. (2014) Rashki, M., M. Miri, and M. A. Moghaddam (2014). A simulation-based method for reliability-based design optimization problems with highly nonlinear constraints. Automation in Construction 47, 24–36.
- Rasmussen and Williams (2006) Rasmussen, C. E. and C. K. I. Williams (2006). Gaussian processes for machine learning (Internet ed.). Adaptive computation and machine learning. Cambridge, Massachusetts: MIT Press.
- Royset (2004) Royset, J. O. (2004). Reliability-based optimal design using sample average approximations. Prob. Eng. Mech. 19, 331–343.
- Royset et al. (2001) Royset, J. O., Der Kiureghian, A., and E. Polak (2001). Reliability-based optimal structural design by the decoupling approach. Reliab. Eng. Sys. Safety 73(3), 213– 221.
- Santner et al. (2003) Santner, T. J., B. J. Williams, and W. I. Notz (2003). The Design and Analysis of Computer Experiments. Springer, New York.
- Shetty et al. (1998) Shetty, N. K., C. Guedes-Soares, P. Thoft-Christensen, and F. M. Jensen (1998). Fire safety assessment and optimal design of passive fire protection for offshore structures. Reliab. Eng. Syst. Saf. 61(1-2), 139–149.
- Smola and Schölkopf (2004) Smola, A. J. and B. Schölkopf (2004). A tutorial on support vector regression. Stat. Comput. 14, 199–222.
- Sobol’ (1967) Sobol’, I. M. (1967). Distribution of points in a cube and approximate evaluation of integrals. U.S.S.R Comput. Maths. Math. Phys. 7, 86–112.
- Song (2013) Song, H. (2013). Efficient sampling-based RBDO by using virtual support vector machine and improving the accuracy of the Kriging method. Ph. D. thesis, University of Iowa, USA.
- Spall (1998a) Spall, J. C. (1998a). Implementation of the simultaneous perturbation algorithm for stochastic optimization. IEEE Trans. Aerospace Electronic Systems 34(3), 817–823.
- Spall (1998b) Spall, J. C. (1998b). An overview of the simultaneous perturbation method for efficient optimization. Johns Hopkins Apl. Technical Digest 19(4), 482–492.
- Spall (2003a) Spall, J. C. (2003a). Introduction to stochastic search and optimization: Estimation, simulation and control, Chapter 14: Simulation-based optimization I: regression, common random numbers, and selection methods. John Wiley & Sons.
- Spall (2003b) Spall, J. C. (2003b). Introduction to stochastic search and optimization: Estimation, simulation and control. John Wiley & Sons.
- Strömberg (2017) Strömberg, N. (2017). Reliability-based design optimization using SORM and SQP. Struct. Multidisc. Optim. 56(3), 631–645.
- Taflanidis (2007) Taflanidis, A. A. (2007). Stochastic system design and applications to stochastic robust structural control. Ph. D. thesis, California Institute of Technology, Pasadena, California, USA.
- Taflanidis and Beck (2008) Taflanidis, A. A. and J. L. Beck (2008). Stochastic subset optimization for optimal reliability problems. Prob. Eng. Mech 23, 324–338.
- Taflanidis and Medina (2014) Taflanidis, A. J. and A. C. Medina (2014). Adaptive Kriging for simulation-based design under uncertainty: Development of metamodels in augmented input space and adaptive tuning of their characteristics. In Proc. 4th International Conference On Simulation And Modeling Methodologies, Technologies And Applications, August 28-30, 2014, Vienna, Austria.
- Torre et al. (2018) Torre, E., S. Marelli, P. Embrechts, and B. Sudret (2018). A general framework for data-driven uncertainty quantification under complex input dependencies using vine copulas. Prob. Eng. Mech.. (in press).
- Tu et al. (1999) Tu, J., K. K. Choi, and Y. H. Park (1999). A new study on reliability-based design optimization. J. Mech. Des. 121, 557 – 564.
- Valdebenito and Schuëller (2010) Valdebenito, A. M. and G. I. Schuëller (2010). A survey on approaches for reliability-based optimization. Struct. Multidisc. Optim. 42, 645–663.
- Vapnik (1995) Vapnik, V. N. (1995). The Nature of Statistical Learning Theory. Springer-Verlag, New York.
- Wang (2003) Wang, G. G. (2003). Adaptive response surface method using inherited Latin Hypercube design points. J. Mech. Design 125, 210–220.
- Wang (1998) Wang, I.-J. ad Spall, J. C. (1998). A constrained simulation perturbation stochastic approximation algorithm based on penalty functions. In Proceedings of the 1998 IEEE ISIC/CIRA/ISAS Joint Conference, Sept. 14–17, 1998, Gaithersburg, MD, USA.
- Youn (2007) Youn, B. D. (2007). Adaptive-loop method for non-deterministic design optimization. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability 221(2), 107–116.
- Youn et al. (2005) Youn, B. D., K. K. Choi, and L. Du (2005). Enriched performance measure approach for reliability-based design optimization. AIAA Journal 43(4), 874–884.
- Zhang et al. (2017) Zhang, J., A. A. Taflanidis, and J. C. Medina (2017). Sequential approximate optimization for design under uncertainty problems utilizing Kriging metamodeling in augmented input space. Comput. Methods Appl. Mech. Engrg. 315, 369–395.
- Zou and Mahadevan (2006) Zou, T. and S. Mahadevan (2006). A direct decoupling approach for efficient reliability-based design optimization. Struct. Multidisc. Optim. 31, 190–200.