Deep Learning for Accelerated Reliability Analysis of Infrastructure Networks
Abstract
Natural disasters can have catastrophic impacts on the functionality of infrastructure systems and cause severe physical and socioeconomic losses. Given budget constraints, it is crucial to optimize decisions regarding mitigation, preparedness, response, and recovery practices for these systems. This requires accurate and efficient means to evaluate the infrastructure system reliability. While numerous research efforts have addressed and quantified the impact of natural disasters on infrastructure systems, typically using the Monte Carlo approach, they still suffer from high computational cost and, thus, are of limited applicability to large systems. This paper presents a deep learning framework for accelerating infrastructure system reliability analysis. In particular, two distinct deep neural network surrogates are constructed and studied: (1) A classifier surrogate which speeds up the connectivity determination of networks, and (2) An endtoend surrogate that replaces a number of components such as roadway status realization, connectivity determination, and connectivity averaging. The proposed approach is applied to a simulationbased study of the twoterminal connectivity of a California transportation network subject to extreme probabilistic earthquake events. Numerical results highlight the effectiveness of the proposed approach in accelerating the transportation system twoterminal reliability analysis with extremely high prediction accuracy.
keywords:
Reliability Analysis, Infrastructure Systems, Surrogates, Neural Networks, Deep Learning, Uncertainty Quantification, Natural Disasters.sort&compress ”@ M[1]¿\arraybackslashm#1
1 Introduction
The hazard reliability for an infrastructure system is defined to be the degree of assurance that the system will continue to successfully operate at a desired level of performance during a certain period of time and in a specified environment in the aftermath of a hazard zacks2012introduction (). Assessment of the impact of natural disasters on infrastructure systems is of importance toward four main objectives: (1) Planning for actions that eliminate or reduce the longterm risk to human life and infrastructure systems (e.g.godschalk1999natural ()); (2) Disaster preparation or adjustment, which aims to reduce the risk of damages and injuries while enabling the capability to cope with the temporary disruption of the infrastructure systems (e.g.paton2003disaster ()); (3) Development of effective emergency response strategies (e.g.perry2007natural ()); and (4) Postdisaster recovery planning (e.g.adie2001holistic ()). These four are, respectively, known as the mitigation, preparedness, response, and recovery practices.
A variety of analytical wang2008integrated (), simulation stern2017accelerated (); bocchini2011stochastic (); chang2010transportations (); bocchini2011generalized (); nabian2017uncertainty (), and optimization liu2009two () approaches are proposed in the literature for hazard reliability analysis of infrastructure systems. A comprehensive literature review on transportation infrastructure system performance in disasters is provided in faturechi2014measuring (). Simulationbased reliability assessment of large infrastructure systems are often computationally intractable or expensive due to the large number of network components, complex network topology, statistical dependence between component failures, and uncertainties in the hazard models. This will impose limitations on design optimization or sensitivity analysis of these systems. Alternatively, a more efficient response assessment for large infrastructure systems can be made possible by using approximate surrogates koziel2013surrogate ().
Surrogates are fast models that approximately describe the relationship between the system inputs and outputs and serve as a substitute for more expensive simulation tools. If the response evaluated by the reference expensive model is denoted by , a surrgate seeks to provide a global approximate function . This is typically done by using a set of inputs , and the corresponding ‘exact’ system outputs . There are several types of surrogate techniques to choose from. Among the most popular ones are polynomial functions (e.g.queipo2005surrogate ()), radial basis functions (e.g.wild2008orbit ()), Kriging (e.g.kleijnen2009kriging ()), support vector machines (SVMs) (e.g.stern2017accelerated (); tabatabaee2012two ()), and neural network (e.g.ziyadi2016efficient ()).
Universal approximation theorem in the mathematical theory of artificial neural networks rigorously proves that the standard multilayer, feedforward, neural networks consist of one or more hidden layers with sufficiently many hidden units and, with arbitrary nonconstant activation functions, can approximate any Borelmeasurable function in a finitedimensional space up to any arbitrary degree of accuracy hornik1989multilayer (); hornik1991approximation (). This signifies that any failure in function approximation with sufficient accuracy by a multilayer network must be due to insufficient number of hidden units, inadequate learning, or lack of a deterministic inputoutput map hornik1989multilayer (). Although this theorem states that singlehiddenlayer neural networks are already universal approximations, implementation of multiple layers will improve the performance of the neural network lecun2015deep (). With the cuttingedge neural network architectures and advanced training algorithms, deep learning has recently been successfully used to solve elusive problems lecun2015deep () and have won several machine learning contests schmidhuber2015deep (). Deep learning consists of the development of computational models using multiple processing layers in order to learn data representations with multiple abstraction levels lecun2015deep (); goodfellow2016deep (); demuth2014neural ().
The goal of this paper is to propose a general framework to accelerate reliability analysis of infrastructure systems. In this paper, we demonstrate how one can achieve this goal using deep neural network surrogates in the context of twoterminal reliability assessment of transportation networks subject to extreme earthquake events. Two distinct deep neural network surrogates are constructed and studied: a classifier surrogate, which speeds up the twoterminal connectivity evaluation for a given network topology, and an endtoend surrogate that replaces the entire Monte Carlo simulation and can be used to immediately calculate the average (expected) twoterminal connectivity given the failure probability of network components. Although the idea of using artificial neural networks in reliability analysis of structures and infrastructure systems has been previously studied (e.g. cheng2008new (); papadrakakis2002reliability (); gomes2004comparison (); hurtado2001neural (); srivaree2002estimation (); elhewy2006reliability (); cardoso2008structural (); zhang2004performance ()), the major contributions of this work are as follows: (1) Neural network surrogates with multiple hidden layers were used to enhance the performance of surrogatebased twoterminal reliability analysis; (2) An endtoend surrogate was proposed, which bypasses the samplebased calculations module that typically requires prohibitively large number of Monte Carlo simulations; and (3) In training the endtoend surrogate, instead of using exact training data, we propose to use the predictions of the classifier surrogate to drastically reduce the computational time. We will numerically show that the proposed endtoend surrogate is capable of accelerating the twoterminal reliability analysis of transportation networks by more than four orders of magnitude, and how such acceleration can substantially facilitate sensitivity analysis and potentially other planning procedures for large networks.
The remainder of this paper is organized as follows. A general simulationbased framework for twoterminal reliability analysis of transportation networks subject to earthquake events is described in Section 2. Next, Section 3 presents the proposed surrogatebased analysis of twoterminal reliability using deep neural networks. Finally, the accuracy and efficiency of the proposed surrogatebased analysis is demonstrated through a case study for the San JoseMountain view transportation network in Section 4.
2 TwoTerminal Reliability Analysis
This section explains a general framework for twoterminal reliability analysis. First, the twoterminal connectivity of a network is introduced. Next, ground motion prediction equations are introduced, which enable the prediction of ground motion intensity measures at the location of network components. Given these predictions, it is then illustrated how one can evaluate the vulnerability of network components by the use of fragility analysis. Finally, a Monte Carlo simulation procedure is described for the analysis of systemlevel response.
2.1 TwoTerminal Connectivity
Consider a transportation network represented by a graph , where is the set of nodes and is the set of links (i.e. roadways). In the aftermath of an earthquake, the links connecting pairs of nodes , may stop functioning primarily due to bridge failures. The twoterminal connectivity is defined as follows. Given a source node and a terminal node , the twoterminal connectivity is the condition where at least a connection exists between the source and terminal nodes. A pair of adjacent nodes are disconnected if there is at least one failed bridge on the link . In this work, it is assumed that bridges are the only components of the transportation network that are vulnerable to and get impacted by seismic hazards. This assumption is very common in the literature (e.g. bocchini2011stochastic ()). The twoterminal connectivity problem is relevant when, for instance, the accessibility from a major attraction point to a major hospital, or from a feedstock to demand zones, is to be maintained during an emergency kang2008matrix ().
2.2 Ground Motion Prediction
For engineering applications, the evaluation of earthquake ground motions is generally performed using empirical Ground Motion Prediction Equations (GMPE) stewart2015selection (); bommer2010selection (). GMPEs are statistical models that provide a means to predict the ground motion intensity measures, such as peak ground motions or response spectra, as a function of earthquake magnitude, sourcetosite distance, fault mechanism, local site conditions, etc. GMPEs are generally constructed based on empirical data and are empirical regression models of recorded data. A summary of all the empirical GMPEs for estimation of earthquake Peak Ground Acceleration (PGA) and elastic response spectral ordinates published between 1964 and 2016 is provided in douglas2017ground ().
In this work, to determine the ground motion (specifically its spectral acceleration ) at a bridge site, the GraizerKalkan 2015 (GK15) GMPE graizer2016summary (); graizer2015update () is adopted. GK15 consists of predictive equations for spectral acceleration and PGA that are derived based on physical simulations and empirical data, which are applicable to earthquakes of moment magnitude between 5.0 and 8.0, at closest distances to fault rupture plane ranging from 0 to 200 km, at sites having in the range of 200 to 1,300 m/s, and for spectral periods of 0.015 s. In GK15, the PGA, herein denoted by , is calculated as a multiplication of a series of functions, and in natural logarithmic scale, is given by
(1) 
where represents a scaling function for magnitude and style faulting, is a model for ground motion attenuation, is a model for adjustment to the attenuation rate in order to take into account the regional anelastic attenuation, represents the site amplification model, and represents a model for basin scaling. is the residual variability, which accounts for unexplained variability in the ground motion data used for the calibration of GMPE. In seismic hazard analysis, reducing this residual variability is of a high priority, since at large values of , the probabilities of exceedance go up rapidly with . As will be shown in the numerical examples, the twoterminal connectivity of San JoseMountain View transportation network is significantly affected by this residual variability graizer2016summary ().
The form of GK15 for the 5% damped response ordinates is
(2) 
where the spectral shape is parameterized by , , , and basin depth under the site . For the analysis of bridge fragility, as described in the next subsection, spectral accelerations at 0.3 s and 1.0 s are used.
2.3 Bridge Fragility Analysis
There are several wellestablished ways for the analysis of structural response to natural hazards. In this study, the HAZUSMH fragility model fema2008hazus (), developed by the Federal Emergency Management Agency (FEMA), is implemented for the calculation of transportation network bridge response to earthquake ground shaking. HAZUSMH is a standardized methodology for the estimation of potential physical, economic, and social losses from earthquakes, hurricanes, and floods. For a given level of ground motion, fragility curves or damage functions for bridges are modeled as functions with lognormal distributions that yield the probability of reaching or exceeding different damage states. Individual fragility curves are parametrized by a median value of ground motion or ground failure, and an associated standard deviation.
The required inputs needed to estimate the damages to a bridge in HAZUSMH fragility model are geographical location of the bridge (latitude and longitude), spectral accelerations at s and s at the bridge location, peak ground acceleration, soil type, and bridge classification. Bridges are classified into 28 primary types based on several structural characteristics, such as seismic design, structure type, number of spans, and pier type. Five damage states are considered for bridges, which are none, slight, moderate, extensive, and complete damage states. Extensive damage for bridges is defined by shear failure, degradation of columns with no collapse, differential settlement at connections, large residual movement at connections, and shear key failure at abutments. In this study, it is assumed that the bridges will stop functioning at the onset of extensive damage state immediately after an earthquake event.
For each of the bridge classes, a total of four different fragility curves are constructed from the combination of two lognormal distributions for ground shaking and ground failure. Afterward, specific fragility curves for individual bridges are constructed by updating the generic curves based on the bridge characteristics. The output of fragility analysis for each bridge is four different curves that represent the probability of that bridge exceeding a damage state for a given level of ground motion. These fragility curves are then used, as illustrated in the next subsection, in order to calculate the systemlevel response of transportation network to an earthquake via a simulationbased study of twoterminal connectivity.
2.4 TwoTerminal Reliability Analysis
In order to estimate the systemlevel network response to an earthquake affecting it components, e.g. roadways, Monte Carlo Simulation (MCS) may be used. MCS is a straightforward, easy to implement, approach ideally suited to parallel computing. For calculation of the twoterminal connectivity in this study, network realizations are drawn by randomly removing roadways according to their survival probabilities, given by Equation 4. Specifically, the damage state for each roadway is modeled as a Bernoulli random variable with the following distribution
(3) 
where denotes the survived and failed states, respectively, with a survival probability of . A roadway with at least one failed bridge will be removed. Therefore, the survival probability of roadway with bridges of IDs is calculated in logarithmic scale as
(4) 
where is the survival probability of bridge .
Let and denote, respectively, the source and terminal nodes predetermined by the stakeholder. For a network realization using the MC sample, the twoterminal connectivity is assessed by evaluating whether there is any connection between the source and terminal
(5) 
in which is the total number of roadways in the network. This procedure is repeated by drawing more network realizations until convergence of the quantity of interest (QoI) is achieved. The DepthFirst Search (DFS) algorithm tarjan1972depth (); korf1985depth () with a lineartime computational complexity of is utilized herein for the evaluation of twoterminal connectivity. For a given set of failure probabilities for bridges, and for a given MCS with network realizations, the expected twoterminal connectivity is estimated by
(6) 
In order to accelerate the twoterminal connectivity computations, the Monte Carlo calculations are performed in parallel rosenthal2000parallel () where different processors evaluate the network connectivity for different network realizations. In the next section, we explain the approach to train and use fast and accurate deep learning surrogates in place of Monte Carlobased DFS (exact) calculations.
3 Surrogate Model for TwoTerminal Connectivity
3.1 Deep Neural Networks
For notation brevity, single hidden layer neural networks are introduced first, since its subsequent generalization to multiple hidden layers, which makes a neural network deep, will be straightforward. Given the dimensional row vector as model input, the dimensional output of a standard single hidden layer neural network is in the form of
(7) 
in which and are weight matrices of size and , respectively, and are biases of size and , respectively. The function is an elementwise nonlinearity, commonly known as the activation function. In deep neural networks, the output of each activation function is transformed by a new weight matrix and a new bias, and is then fed to another activation function. For each new set of weight matrix and bias that is added to (7), a new hidden layer is added to the neural network. The capacity of neural networks can be easily increased by adding more hidden layers or more units to each hidden layer.
Popular choices of activation functions are Sigmoid, hyperbolic tangent (Tanh), and rectified linear unit (RELU). The RELU activation function has the form of . RELUs are getting increasingly popular in deep learning applications as, compared to Sigmoid and Tanh activations, they are faster and do not suffer from the vanishing gradient problem.
In order to calibrate the weight matrices and biases for a regression problem, we may use a Euclidean loss function as follows
(8) 
where is the mean squared error, is the set of observed inputs, is the set of observed outputs, and is the set of neural network output (model prediction) corresponding to the set of inputs . For a binary classification task, we may use a binary crossentropy loss function in the form of
(9) 
where is the binary crossentropy. Minimizing the loss function with respect to model parameters () will yield the calibrated model parameters (). For instance, for a binary crossentropy loss function, we have
(10) 
Minimizing the loss function is usually performed using backpropagation lecun2015deep (); jin2000improvements (). It consists of a twophase cycle; forward pass and backward pass. A forward pass takes the input to the network and propagates it through the layers of the network, one by one, to calculate the network output. A backward pass starts from the network output and propagates towards the input layer while calculating the gradients, layer by layer, using the chain rule.
3.2 Deep Neural Networks for TwoTerminal Reliability Analysis
The stepbystep procedure for construction of DNN surrogates that can be used to accelerate twoterminal reliability analysis of transportation systems is elaborated in this section. Two different surrogate models are developed in this study. The first model is hereinafter referred to as the classifier surrogate. It replaces the DFS algorithm to determine whether a particular sourcetoterminal connection exists. It does so for each MC sample, i.e. for each realized roadway failure and its corresponding topology. The input to this model is therefore a deterministic network topology in the form of a binary vector, and the output is a binary variable indicating the connection.
The second surrogate model, which we refer to as the endtoend surrogate, is designed to replace the topology realization, connectivity determination, and connectivity averaging modules (see Figure 1). It is used to immediately evaluate the average (expected) twoterminal connectivity given the roadway failure probabilities. It bypasses roadway status realizations from the failure probabilities, and thus saves computational time. Figures 2 and 3 show the proposed frameworks for constructing the classifier and endtoend surrogates, respectively, and how these surrogates are utilized in the evaluation of expected twoterminal connectivity.
3.3 Surrogate Performance Measures
In order to evaluate the accuracy of a DNN surrogate model, a number of performance measures are used in this study. They include QoI prediction accuracy , binary classification accuracy , sensitivity or True Positive Rate (TPR), and specificity or True Negative Rate (TNR) baldi2000assessing (). The last three measures are applicable to binary classification only.
The QoI prediction accuracy for connectivity is calculated as
(11) 
where and are the twoterminal connectivity calculated respectively using exact (DFS) connectivity check and the surrogate. The binary classification accuracy is calculated as
(12) 
where TP (True Positive) is the number of times the surrogate correctly predicts network survival, TN (True Negative) is the number of times the surrogate correctly predicts network failure, FP (False Positive) is the number of times the surrogate incorrectly predicts network survival, and FN (False Negative) is the number of times the surrogate incorrectly predicts network failure, satisfying . As more specific measures, True Positive Rate (TPR) and True Negative Rate (TNR) are calculated as
(13)  
(14) 
4 Case Study for the San JoseMountain View Transportation Network
The surrogatebased twoterminal connectivity analysis procedure described in Section 3.2 is applied to the transportation network that connects San Jose, CA to Mountain View, CA in the United States. This network is located in a region of high seismic activity. A sketch of this network is provided in Figure 4. The network consists of 39 bridges, 12 nodes, and 18 roadway links (out of which 14 have at least one bridge). Throughout the numerical examples presented in this section, the network is considered to be impacted by the 1989 Loma Prieta earthquake with varying magnitudes. The geographical coordinates for the epicenter of Loma Prieta earthquake are , .
A number of assumptions and choices were made throughout this section. First, bridges are assumed to be the only network components vulnerable to earthquakes, as commonly considered in the literature (e.g. bocchini2011stochastic ()). Second, the network is considered to be an undirected graph since the adjacent bridges on two different sides of the road share the same or very similar properties. Also, according to the HAZUSMH soil classification, the soil for the study area is determined to be of type D.
The NetworkX Python library schult2008exploring () was used for network connectivity evaluation using DFS algorithm, and the Keras deep learning library chollet2015keras () was used for construction of classifier and endtoend surrogates. The Python source codes for all the simulations presented in this section are made available on GitHub mohammad_amin_nabian_2017_846898 (). Computations in sections 4.1, 4.2 are conducted on a quad core 2.5 GHz MacBook Pro with 16 GB of RAM. Computations in sections 4.3, 4.4 are conducted on the Comet cluster from XSEDE (Extreme Science and Engineering Discovery Environment) resources [50], with 24 2.5 GHz CPU cores and 128 GB of DRAM.
4.1 Classifier Surrogate Training and Prediction
Following the framework represented in Figure 2, given a network realization, the classifier surrogate indicates whether a sourcetoterminal connection exists. The input and output to this surrogate are a binary vector of roadway conditions (failed or survived) and a binary connectivity indicator, respectively. In order to generate training and evaluation data sets, a total of 10,000 samples of earthquake magnitude , denoted by are drawn according to
(15) 
where is a random sample drawn from a truncated exponential distribution with a shape parameter and lower and upper bounds of 15, 0, and 1.5, receptively. Ninety percent of the samples are used for training, and the rest is left for surrogate evaluation. A sketch for the probability distribution of is provided in Figure 5. Training and evaluation samples are preferred to be drawn from an exponential distribution, and not a uniform distribution. This is due to the nonlinear relationship between earthquake moment magnitude and energy release kanamori1977energy (), leading to larger sensitivity of failure probabilities to magnitude perturbations when the nominal magnitude is larger; hence, the exponentially increasing distribution of training samples.
The classifier surrogate consists of 7 hidden layers with different dimensionalities (see Figure 6). RELU activation is adopted for hidden layers 1 through 6, while the Sigmoid activation is used in the last hidden layer. The Adam optimization algorithm kingma2014adam ()is used to minimize the binary crossentropy loss function (Equation 9). For 150 epochs and a batch size of 64, it took 83.05 seconds to train the classifier surrogate.
In order to evaluate the predictive performance of the trained classifier surrogate, we consider five different scenarios with earthquake magnitudes 6.7, 7.0, 7.3, 7.6, and 7.9 , and for each we use the trained surrogate for twoterminal connectivity evaluation. The surrogatebased results are compared versus exact connectivity results obtained using DFS algorithm. Figure 7 shows the survival probabilities for the 39 bridges subject to the five earthquake scenarios. For each earthquake event, given these survival probabilities and by using Equation 4 for calculating roadway failure probabilities, a total of 100,000 network realizations are generated. The twoterminal connectivity of each one of these network realizations is determined using the classifier surrogate and the DFS algorithm, and the resulting expected connectivities are compared in Figure 8. It is evident from the convergence plots that the surrogate and DFS results are in close agreement. The estimated expected values for twoterminal connectivity, as well as computational times, are compared in Table 1, and surrogate performance measures are reported in Table 2. Compared to exact connectivity check using DFS, the classifier surrogate predictions are about one order of magnitude faster, with accuracies of more than 99.9%.
Magnitude  DFS time (s)  Surrogate time (s)  




0.9769  0.9769  5.23  0.62  

0.9256  0.9255  5.40  0.62  

0.8635  0.8633  6.05  0.63  

0.7679  0.7679  5.71  0.62  

0.6263  0.6264  5.63  0.65 
Magnitude  TPR  TNR  




1.0000  1.0000  0.9996  

0.9998  0.9998  0.9989  

0.9995  0.9996  0.9990  

0.9995  0.9997  0.9988  

0.9995  0.9996  0.9993 
Next, we investigate the performance of the classifier surrogate in network connectivity prediction for a probabilistic earthquake event, i.e. for earthquakes with probabilistic magnitudes. Following cosentino1977truncated (); kang2008matrix (), it is assumed that the earthquake magnitude follows a truncated exponential distribution with the following pdf
(16) 
where and are the minimum and maximum of random magnitudes, which are set to 6.8 and 7.5, respectively. is the shape parameter and is set to 0.76 kang2008matrix () .
A total of 10,000,000 network realizations are generated, each corresponding to a random sample from the probabilistic magnitude and a random sample from roadway failure states. For each of these network realizations, the twoterminal connectivity is evaluated using DFS algorithm and the classifier surrogate, and the resulting expected connectivities are compared in Figure 9. Expected twoterminal connectivity using DFS and classifier surrogate is, respectively, 0.9002 and 0.9001. The DFS and surrogate computational time are, respectively, 603.97 and 53.89 seconds, and , TPR, and TNR are 0.9997, 0.9998, and 0.9989, respectively. Once again, classifier surrogate results are in close agreement with DFS results with accuracies of more than 99.9%, however, are achieved one order of magnitude faster.
4.2 UncertaintyAware TwoTerminal Reliability Analysis Using Classifier Surrogate
In this section, we investigate an additional layer of uncertainty in twoterminal reliability assessment. Specifically, we consider the residual variability in GMPE, which is due to the model fitting error. To quantify the impact of this residual variability, we start off by the probabilistic earthquake event defined in Equation 16, and then consider and at each bridge location to be normally distributed random variables with mean values calculated using equations 1 and 2, and standard deviations reported in graizer2016summary (). To study the classifier surrogate performance in this case, following the procedure represented in Figure 2, 10,000,000 network realizations are generated by consecutive sampling from earthquake magnitudes, and at bridge locations, and roadway failure states according to their failure probabilities. Figure 10 shows good agreement between expected twoterminal connectivities using DFS and the classifier surrogate. This is while the surrogate evaluation, compared to DFS, is about one order of magnitude faster, i.e. 574.48 vs. 53.08 seconds. The estimated expected twoterminal connectivity using DFS and classifier surrogate are, respectively, 0.6853 and 0.6853, and , TPR, and TNR are 0.9990, 0.9993, and 0.9985, respectively.
It should be noted that the evaluated twoterminal reliability in this section (0.69) is smaller than the one evaluated in the previous section (0.90). This highlights the importance of the additional uncertainties in ground motion intensity measures (e.g. and ), which is usually ignored in reliability studies (e.g. bocchini2011stochastic (); stern2017accelerated ()) and can lead to overestimation of the network reliability.
4.3 EndtoEnd Surrogate Training and Prediction
As mentioned earlier, two terminal connectivity calculations can be substantially accelerated by using an endtoend surrogate, which replaces the entire MCS as outlined in Figure 3. To numerically demonstrate this, we need to first train the endtoend surrogate. The training data can be generated using the DFS algorithm. Alternatively, we will use the previouslydeveloped classifier surrogate to produce the training data. It should be noted that this training data set is not exact, but according to the results in the previous sections the error is expected to be negligible and the computational speed up is expected to be substantial.
Figure 11 shows the architecture of the endtoend surrogate, with the input being the vector of roadway failure probabilities and the output the expected twoterminal connectivity. The surrogate consists of 5 hidden layers with different dimensionalities. Sigmoid activation is adopted for hidden layers. The Adam optimizer is used to minimize the Euclidean loss function (Equation 8). To generate training and evaluation data, a total of 3,000 magnitude samples are drawn, and for each magnitude sample, 100,000 topology samples are drawn whose twoterminal connectivities were evaluated using the classifier surrogate. For a batch size of 64 and 2,000 epochs, the endtoend surrogate training time (including generation of training and evaluation data and model calibration) was 351.99 seconds.
Using the trained endtoend surrogate, we study the twoterminal connectivity of the San JoseMountain View transportation network subject to a probabilistic earthquake event. Similar to Section 4.1, it is assumed that the earthquake magnitude follows a truncated exponential distribution. The lower and upper bounds for the magnitude variability are set to 6.8 and 7.5, respectively. Without loss of generality, the GMPE residual variabilities were ignored for simplicity. To test the surrogate, 10,000 magnitude samples are drawn and for each sample, the expected twoterminal connectivity is calculated using the endtoend surrogate. As the reference case, for each earthquake realization, a total of 100,000 topology realizations are drawn and their connectivity is evaluated using DFS algorithm, and the results are compared in Figure 12. As another way of demonstrating the surrogate accuracy, Figure 13 compares the DFS and surrogate predictions of connectivity for each earthquake realization. The expected twoterminal connectivity was estimated to be 0.9001 using both approaches while the computational times for DFS and endtoend surrogate were found to be 7,857.92 and 0.71 seconds, respectively.
4.4 Oneatatime Sensitivity Analysis Using EndtoEnd Surrogate
In this section, we demonstrate the application of the proposed endtoend surrogate in maintenance planning. In particular, we consider the optimal seismic retrofitting of bridges buckle2006seismic () where decision makers seek to improve the twoterminal reliability of the network. In this case, typically in the face of budget constraints, it is crucial to identify the bridges that are most influential on twoterminal reliability and prioritize them for repair. To this end, a oneatatime (OAT) sensitivity analysis can be performed komkov1986design (). It involves considering amplifications on the survival probabilities, one bridge at a time, while keeping the other bridges’ survival probabilities at their nominal values. These amplifications should reflect the expected outcomes of repair plans for each bridge. We assume that these retrofit plans will result in an amplification rate of for every bridge. Considering this rate, the expected twoterminal connectivities are then calculated using the DFS algorithm and endtoend surrogate, for the nominal and “retrofitted” networks. Here we consider a probabilistic earthquake event as defined in Equation 16 with a magnitude ranging between 7.3 and 7.9 . Table 3 shows the OAT sensitivity analysis results. For brevity, only the results for the three most and least sensitive components are shown. For the DFS results, for each earthquake realization, a total of 100,000 topology realizations are drawn. With no amplification, the expected twoterminal connectivity probability of the network subject to this probabilistic earthquake event is found to be 0.7641 using DFS and 0.7643 using the endtoend surrogate. This table also highlights the substantial computational savings that the endtoend surrogate can offer in repetitive processes, e.g. optimization, sensitivity analysis, or realtime riskinformed decision making.
Rank  Bridge ID  Improvement in connectivity (%) (DFS estimate)  Improvement in connectivity (%) (surrogate estimate)  DFS time (s)  Surrogate time (s) 



0  8.22  8.19  8361.22  0.75 

13  7.80  7.78  7855.76  0.72 

25  2.17  2.14  8334.18  0.74 

⋮  ⋮  ⋮  ⋮  ⋮ 

14  0.07  0.06  8089.66  0.75 

10  0.01  0.01  8131.76  0.74 

26  0.00  0.00  8079.06  0.75 
5 Conclusion
Approximations and uncertainties inherent in infrastructure systems reliability analysis on one hand and the associated computational challenge on the other hand motivate the utilization of fast and sufficiently accurate surrogates that can replace one or more computational modules in the analysis pipeline. The resulting surrogatebased reliability analysis can then facilitate optimal planning and management of infrastructure systems subject to natural hazards. In this paper, we studied the surrogates that are trained based on deep learning, and using a case study, highlighted how they can offer fast computation of infrastructure response with high accuracy. An important advantage of using deep learning in building surrogates for nonlinear system responses is its capability for automatic feature engineering/detection. This will remove the need to manually identify features for a given data, and make the approach broadly applicable to various nonlinear responses.
The proposed surrogatebased reliability analysis framework can be further extended by augmenting the training data to improve the prediction accuracy. An example of data augmentation for improving TNR can be created as follows. For each topology realization with no sourcetoterminal connectivity, we can generate multiple additional topology realizations by randomly (according to roadway failure probabilities) letting the survived roadways fail. These additional network realizations will not incur extra computational burden, as they are already known to be corresponding to a “noconnectivity” condition. Another extension to further improve the computational efficiency is to make use of graphic processing units (GPUs) in deep neural network surrogate training and prediction. Deep learning generally involves large matrix multiplications that are substantially parallelizable using GPUs, leading to significant acceleration.
Acknowledgement
This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI1053575.
References
References
 (1) S. Zacks, Introduction to reliability analysis: probability models and statistical methods, Springer Science & Business Media, 2012.
 (2) D. Godschalk, Natural hazard mitigation: Recasting disaster policy and planning, Island Press, 1999.
 (3) D. Paton, Disaster preparedness: a socialcognitive perspective, Disaster Prevention and Management: An International Journal 12 (3) (2003) 210–216.
 (4) M. Perry, Natural disaster management planning: A study of logistics managers responding to the tsunami, International Journal of Physical Distribution & Logistics Management 37 (5) (2007) 409–433.
 (5) C. E. Adie, et al., Holistic disaster recovery: Ideas for building local sustainability after a natural disaster, DIANE Publishing, 2001.
 (6) Y.M. Wang, J. Liu, T. M. Elhag, An integrated ahp–dea methodology for bridge risk assessment, Computers & industrial engineering 54 (3) (2008) 513–525.
 (7) R. Stern, J. Song, D. Work, Accelerated monte carlo system reliability analysis through machinelearningbased surrogate models of network connectivity, Reliability Engineering & System Safety 164 (2017) 1–9.
 (8) P. Bocchini, D. M. Frangopol, A stochastic computational framework for the joint transportation network fragility analysis and traffic flow distribution under extreme events, Probabilistic Engineering Mechanics 26 (2) (2011) 182–193.
 (9) L. Chang, A. S. Elnashai, B. F. Spencer, J. Song, Y. Ouyang, Transportations systems modeling and applications in earthquake engineering, Tech. rep., DTIC Document (2010).
 (10) P. Bocchini, D. M. Frangopol, Generalized bridge network performance analysis with correlation and timevariant reliability, Structural Safety 33 (2) (2011) 155–164.
 (11) M. A. Nabian, H. Meidani, Uncertainty quantification and pcabased model reduction for parallel monte carlo analysis of infrastructure system reliability, Tech. rep. (2017).
 (12) C. Liu, Y. Fan, F. Ordóñez, A twostage stochastic programming model for transportation network protection, Computers & Operations Research 36 (5) (2009) 1582–1590.
 (13) R. Faturechi, E. MillerHooks, Measuring the performance of transportation infrastructure systems in disasters: A comprehensive review, Journal of infrastructure systems 21 (1) (2014) 04014025.
 (14) S. Koziel, L. Leifsson, Surrogatebased modeling and optimization, Applications in Engineering.
 (15) N. V. Queipo, R. T. Haftka, W. Shyy, T. Goel, R. Vaidyanathan, P. K. Tucker, Surrogatebased analysis and optimization, Progress in aerospace sciences 41 (1) (2005) 1–28.
 (16) S. M. Wild, R. G. Regis, C. A. Shoemaker, Orbit: Optimization by radial basis function interpolation in trustregions, SIAM Journal on Scientific Computing 30 (6) (2008) 3197–3219.
 (17) J. P. Kleijnen, Kriging metamodeling in simulation: A review, European journal of operational research 192 (3) (2009) 707–716.
 (18) N. Tabatabaee, M. Ziyadi, Y. Shafahi, Twostage support vector classifier and recurrent neural network predictor for pavement performance modeling, Journal of Infrastructure Systems 19 (3) (2012) 266–274.
 (19) M. Ziyadi, I. L. AlQadi, Efficient surrogate method for predicting pavement response to various tire configurations, Neural Computing and Applications (2016) 1–13.
 (20) K. Hornik, M. Stinchcombe, H. White, Multilayer feedforward networks are universal approximators, Neural networks 2 (5) (1989) 359–366.
 (21) K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural networks 4 (2) (1991) 251–257.
 (22) Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444.
 (23) J. Schmidhuber, Deep learning in neural networks: An overview, Neural networks 61 (2015) 85–117.
 (24) I. Goodfellow, Y. Bengio, A. Courville, Deep learning, MIT Press, 2016.
 (25) H. B. Demuth, M. H. Beale, O. De Jess, M. T. Hagan, Neural network design, Martin Hagan, 2014.
 (26) J. Cheng, Q. Li, R.c. Xiao, A new artificial neural networkbased response surface method for structural reliability analysis, Probabilistic Engineering Mechanics 23 (1) (2008) 51–63.
 (27) M. Papadrakakis, N. D. Lagaros, Reliabilitybased structural optimization using neural networks and monte carlo simulation, Computer methods in applied mechanics and engineering 191 (32) (2002) 3491–3507.
 (28) H. M. Gomes, A. M. Awruch, Comparison of response surface and neural network with other methods for structural reliability analysis, Structural safety 26 (1) (2004) 49–67.
 (29) J. E. Hurtado, D. A. Alvarez, Neuralnetworkbased reliability analysis: a comparative study, Computer methods in applied mechanics and engineering 191 (1) (2001) 113–132.
 (30) C. Srivareeratana, A. Konak, A. E. Smith, Estimation of allterminal network reliability using an artificial neural network, Computers & Operations Research 29 (7) (2002) 849–868.
 (31) A. H. Elhewy, E. Mesbahi, Y. Pu, Reliability analysis of structures using neural network method, Probabilistic Engineering Mechanics 21 (1) (2006) 44–53.
 (32) J. B. Cardoso, J. R. de Almeida, J. M. Dias, P. G. Coelho, Structural reliability analysis using monte carlo simulation and neural networks, Advances in Engineering Software 39 (6) (2008) 505–513.
 (33) J. Zhang, R. O. Foschi, Performancebased design and seismic reliability analysis using designed experiments and neural networks, Probabilistic Engineering Mechanics 19 (3) (2004) 259–267.
 (34) W.H. Kang, J. Song, P. Gardoni, Matrixbased system reliability method and applications to bridge networks, Reliability Engineering & System Safety 93 (11) (2008) 1584–1593.
 (35) J. P. Stewart, J. Douglas, M. Javanbarg, Y. Bozorgnia, N. A. Abrahamson, D. M. Boore, K. W. Campbell, E. Delavaud, M. Erdik, P. J. Stafford, Selection of ground motion prediction equations for the global earthquake model, Earthquake Spectra 31 (1) (2015) 19–45.
 (36) J. J. Bommer, J. Douglas, F. Scherbaum, F. Cotton, H. Bungum, D. Fäh, On the selection of groundmotion prediction equations for seismic hazard analysis, Seismological Research Letters 81 (5) (2010) 783–793.
 (37) J. Douglas, Groundmotion prediction equations 19642016, Pacific Earthquake Engineering Research Center Berkeley, CA, 2017.
 (38) V. Graizer, E. Kalkan, Summary of the gk15 groundmotion prediction equation for horizontal pga and 5% damped psa from shallow crustal continental earthquakes, Bulletin of the Seismological Society of America 106 (2) (2016) 687–707.
 (39) V. Graizer, Update of the graizer–kalkan groundmotion prediction equations for shallow crustal continental earthquakes, US Geol. Surv. OpenFile Rept 1009 (2015) 98.
 (40) FEMA, Hazusmh mr3: Technical manual (2008).
 (41) R. Tarjan, Depthfirst search and linear graph algorithms, SIAM journal on computing 1 (2) (1972) 146–160.
 (42) R. E. Korf, Depthfirst iterativedeepening: An optimal admissible tree search, Artificial intelligence 27 (1) (1985) 97–109.
 (43) J. S. Rosenthal, Parallel computing and monte carlo algorithms, Far east journal of theoretical statistics 4 (2) (2000) 207–236.
 (44) W. Jin, Z. J. Li, L. S. Wei, H. Zhen, The improvements of bp neural network learning algorithm, in: Signal Processing Proceedings, 2000. WCCCICSP 2000. 5th International Conference on, Vol. 3, IEEE, 2000, pp. 1647–1649.
 (45) P. Baldi, S. Brunak, Y. Chauvin, C. A. Andersen, H. Nielsen, Assessing the accuracy of prediction algorithms for classification: an overview, Bioinformatics 16 (5) (2000) 412–424.
 (46) D. A. Schult, P. Swart, Exploring network structure, dynamics, and function using networkx, in: Proceedings of the 7th Python in Science Conferences (SciPy 2008), Vol. 2008, 2008, pp. 11–16.
 (47) F. Chollet, Keras, https://github.com/fchollet/keras (2015).

(48)
M. A. Nabian,
UIUCUQ/DeepLearningforReliabilityAnalysis:
Deep Learning for Accelerated Reliability Analysis of Infrastructure
Systems (Aug. 2017).
doi:10.5281/zenodo.846898.
URL https://doi.org/10.5281/zenodo.846898  (49) H. Kanamori, The energy release in great earthquakes, Journal of geophysical research 82 (20) (1977) 2981–2987.
 (50) D. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980.
 (51) P. Cosentino, V. Ficarra, D. Luzio, Truncated exponential frequencymagnitude relationship in earthquake statistics, Bulletin of the Seismological Society of America 67 (6) (1977) 1615–1623.
 (52) G. Buckle, I. Friedland, J. Mander, G. Martin, R. Nutt, M. Power, Seismic retrofitting manual for highway structures, MCEER, Buffalo, 2006.
 (53) V. Komkov, K. K. Choi, E. J. Haug, Design sensitivity analysis of structural systems, Vol. 177, Academic press, 1986.