Dynamic Prediction of ICU Mortality RiskUsing Domain Adaptation

Dynamic Prediction of ICU Mortality Risk Using Domain Adaptation


Early recognition of risky trajectories during an Intensive Care Unit (ICU) stay is one of the key steps towards improving patient survival. Learning trajectories from physiological signals continuously measured during an ICU stay requires learning time-series features that are robust and discriminative across diverse patient populations. Patients within different ICU populations (referred here as domains) vary by age, conditions and interventions. Thus, mortality prediction models using patient data from a particular ICU population may perform suboptimally in other populations because the features used to train such models have different distributions across the groups. In this paper, we explore domain adaptation strategies in order to learn mortality prediction models that extract and transfer complex temporal features from multivariate time-series ICU data. Features are extracted in a way that the state of the patient in a certain time depends on the previous state. This enables dynamic predictions and creates a mortality risk space that describes the risk of a patient at a particular time. Experiments based on cross-ICU populations reveals that our model outperforms all considered baselines. Gains in terms of AUC range from 4% to 8% for early predictions when compared with a recent state-of-the-art representative for ICU mortality prediction. In particular, models for the Cardiac ICU population achieve AUC numbers as high as 0.88, showing excellent clinical utility for early mortality prediction. Finally, we present an explanation of factors contributing to the possible ICU outcomes, so that our models can be used to complement clinical reasoning.

I Introduction

Data from patients in the Intensive Care Unit (ICU) are extensive, complex, and often produced at a rate far greater than intensivists can absorb. Monitoring ICU patients is becoming increasingly complicated, and systems that learn from ICU data to alert clinicians to the current and future risks of a patient are playing a significant role in the decision making process [36]. One of the main barriers in the deployment of these learning systems is the lack of generalisation of results as the learning effectiveness achieved in controlled environments often drops when the models are tested with different patient populations and conditions [2].

In this paper, we explore domain adaptation approaches to improve the accuracy of systems evaluated with mismatched training and testing conditions. We propose deep models that extract the domain-shared and the domain-specific latent features from ICU domains or patient sub-populations. Each domain corresponds to a different ICU type, such as cardiac, coronary, medical and surgical ICUs. This enables us to learn multiple models that are specific to each ICU domain, improving prediction accuracy over diverse patient populations. For this, we employ transference approaches that differ in terms of the choice of which layers to freeze or tune [1].

Our proposed models combine convolutional and recurrent components. While this combination has been investigated in prior work other than mortality prediction [44], here we capture local physiological interactions (e.g., heart rate, creatinine, systolic blood pressure) at the lower level using a Convolutional Neural Network (CNN) and extracts the long range dependencies based on convolved physiological signals at the higher level using a Long Short-Term Memory network (LSTM). Thus, our models exploit temporal information within vital signals and laboratorial findings to dynamically predict patient outcomes, i.e., the CNN component extracts features of varying abstract levels and the LSTM component ingests a sequence of these features to generate dynamic predictions for patient mortality.

As a consequence, the learned representations along with the predictions for a specific patient during the ICU stay form the corresponding patient trajectory and, thus, a mortality risk space can be obtained from a set of past patient trajectories. The fundamental benefit of analyzing future patient trajectories in the mortality risk space is the focus on dynamics, emphasizing the proximity to risky regions and the speed in which the patient condition changes. Thus, the mortality risk space enables clinicians to track risky trends and to gain insight into their treatment decisions.

I-a Contributions and Findings.

In this paper we elucidate the extent to which ICU mortality prediction may benefit from ICU domain adaptation. Thus, our main contributions are:

  • We present a combination of convolutional and recurrent architecture that offers a complementary temporal perspective of the patient condition. As a result, predictions based on information that is continuously collected over time can be dynamically updated as soon as new information becomes available [12]. Further, we employ Shapley additive explanations [31] over the ICU stay in order to provide interpretable real-time predictions to help physicians prevent risky trajectories in the ICU and to complement clinical reasoning [18].

  • We show that patients within different ICU domains form sub-populations with different marginal distributions over their feature spaces. Therefore, we propose to learn specific models for different ICU domains that are trained using different feature transference approaches, instead of learning a single model for different ICU domains. We show that the effectiveness of different feature transference approaches varies greatly depending on the factors that define the target domain.

  • We conducted rigorous experiments using the PhysioNet 2012 dataset [41], which comprises four different ICU domains. We show that multi-domain ICU data used for adaptation can significantly improve the effectiveness of the final model. Gains in terms of AUC range from 4% to 8% for early predictions, i.e., predictions based on data acquired during the first hours after admission.

  • We show that the patient representations along with the predictions provided by our models are meaningful in the sense that they form trajectories in a mortality risk space. Dynamics within this space can be very discriminative, enabling clinicians to track risky trends and to gain more insight into their treatment decisions.

Ii Background Research

Research on predicting ICU mortality is of great academic interest in medicine [8] and in clinical machine learning [21, 32]. A number of researchers have investigated how to correlate ICU data with patient outcomes. In one of the first studies [38], the authors identify parameters in patient data that correlate with outcomes. In what follows, we discuss previous work in contrast with ours.

Ii-a Mortality Prediction

The PhysioNet ICU Mortality Challenge 2012 [41] provided benchmark data that incorporate evolving clinical data for ICU mortality prediction. As [22] reported, this benchmark data fostered the development of new approaches, leading to up to 170% improvement over traditional risk scoring systems that do not incorporate such clinical data currently used in ICUs [14]. In what follows, we discuss previous work in contrast with ours.

Most of the current work uses the PhysioNet ICU Mortality Challenge 2012 data. The most effective approaches are based on learning discriminative classifiers for specific sub-populations. Authors in [11] proposed a robust SVM classifier, while authors in [5] proposed a logistic regression classifier. Authors in [42] also employed logistic regression classifiers, but coupled them with Hidden Markov Models in order to model time-series data. Shallow neural networks were evaluated in [45], while a tree-based Bayesian ensemble classifier was evaluated in [20]. Authors in [25] employed fuzzy rule-based systems, and authors in [35] proposed an approach that identifies and integrates information in motifs that are statistically over- or under-represented in ICU time series of patients. Authors in [29] used LSTMs to improve the classification of diagnoses.

More recently, authors in [27] proposed a Markov model that accumulates mortality probabilities. Likewise, authors in [3] proposed an approach that models the mortality probability as a latent state that evolves over time. Authors in [17] proposed an approach to address the problem of small data using transfer learning in the context of developing risk models for cardiac surgeries. They explored ways to build surgery-specific and hospital-specific models using information from other kinds of surgeries and other hospitals. Their approach is based on weighting examples according to their similarity to the target task training examples. The three aforementioned works are considered as baselines and compared with our approach.

Following [17], in this work we use feature transference, but in a quite different way, as follows: (i) instead of applying instance weighting, we employed a deep model that transfers domain-shared features; (ii) we studied a broader scenario that includes diverse ICU domains; and (iii) our models employ both spatial and temporal feature extraction, being able to predict patient outcomes dynamically.

Ii-B ICU Domains and Sub-Populations

Imbalanced data [6], sub-populations of patients with different marginal distributions over their feature spaces [37], and sparse data acquired from heterogeneous sources [15, 19] are issues that pose significant challenges for ICU mortality prediction.

Authors in [16] discussed problems due to the lack of consistency in how semantically equivalent information is encoded in different ICU databases. Authors in [6] discussed the problem of imbalanced ICU data, which occurs when one of the possible patient outcomes is significantly under-represented in the data. Further, since features are often imbalanced, some ICU domains have a significantly larger number of observations than others (e.g., respiratory failure in adults vs. children). In a recent work, authors in [7] proposed a mortality study based on the notion of burstiness, where high values of burstiness in time-series ICU data may relate to possible complications in the patient’s medical condition and hence provide indications on the mortality. Authors in [39] employed a variational recurrent neural network in order to capture temporal latent dependencies of multivariate time-series data.

While most studies on mortality prediction for ICU patients have assumed that one common risk model could be developed and applied to all the patients, authors in [37] advocated that this might fail to capture the diversity of ICU patients. As shown in [2], models built using patient data from particular age groups perform poorly on other age groups because the features used to train the models have different distributions across the groups.

Ii-C Our Work

None of the aforementioned approaches attempted to perform ICU domain adaptation, which is a core focus of our work. There is often a mismatch between different ICU domains or patient sub-populations, and domain adaptation seems to be a natural solution for learning more robust models, as different ICU domains share features that exhibit different distributions. While data in different ICU domains may vary, there are potentially shared or local invariant features that shape patients in different ICU domains.

Other focus of our work is to capture spatial and temporal features from time-series ICU data. Features are captured in a way that the state of the patient in a certain time depends on the previous state. This forms a risk space, and trajectories in this space allow to easily describe the state of the patient at a particular time, helping intensivists to estimate the patient progress from the current patient state.

Iii Methods

The task of predicting patient outcomes over time from ICU data is defined as follows. We have as input the training set, which consists of a sequence of observations of the form , where is a vector of values corresponding to physiological parameters associated with a patient at time , and is the outcome for the patient at time (i.e., whether or not the patient survived the hospitalization, replicated for each time ). The training set is used to construct a model that relates features within the sequence of observations to the patient outcome. The test set consists of a sequence of observations for which only the physiological parameters for the patient at time are available, while the corresponding patient outcome is unknown. The model learned from the training set is used to predict the outcome for patients in the test set. The task of dynamically predicting patients outcomes in the ICU has two important requirements:

  • It is a domain-specific problem, i.e., a prediction model learned from a sub-population (or ICU domain) is likely to fail when tested against data from other population [39]. Feature transferability is thus an appealing way to provide robustness to the prediction models [33, 47].

  • It is a time-sensitive problem, i.e., accurately predicting patient outcomes as early as possible may lead to earlier diagnosis and more effective therapies.

Next we present our model, which is built from multi-domain ICU time-series data and is designed to provide dynamically updated estimates of patient mortality.

Iii-a Network Architecture

Here, we introduce our deep model, referred to as CNN LSTM, which is composed of both convolutional and recurrent layers, as shown in Figure 1. Convolutional and recurrent components offer a complementary perspective of the patient condition, as follows: the convolutional layer encodes temporal physiological information locally, while the recurrent layer is designed to capture long range information and forget unimportant local information.

Specifically, our model employs one-dimensional CNN layers [26] followed by max-pooling layers, thus extracting correlations between physiological parameters measured in consecutive time periods. For instance, it may find that if both temperature and heart rate are increasing, the odds of survival decrease. In a complementary way, the recurrent layer (LSTM) learns how changes in observations for a patient affect the corresponding outcome. Intuitively, the recurrent layer captures temporal dependencies, enabling the estimation of patient progress from the current patient state. For instance, if the heart rate was low at the beginning of the stay and then becomes very high, then the odds of survival decrease. Finally, a dense layer takes the output of the recurrent layer and predicts the patient outcome.

\ncput*dynamic predictions\ncput*Dense\ncput*Dense\ncput*Dense\ncput*Dense

Fig. 1: Architecture for predicting outcomes over time. Each convolutional (CNN) layer is followed by a LSTM layer and different feature transference approaches are designed using this architecture.

In summary, our model works by passing each observation through a feature extractor and then the sequence model captures how the extracted features are associated with patient outcomes over time. Also, dropout operation is performed after each layer of the network.

As not all the descriptors and time-series were available for all records, we had to deal with the problem of missing values. If one variable (either a descriptor or a time-series) was never recorded for a given patient, we used the approach called “imputation” and replaced its features with value zero after normalization. Because of the normalization step, this approximately corresponds to replacing the missing raw variable with a measure of central tendency, which corresponds to the arithmetic mean for Gaussian-distributed variables and to the geometric mean for log-normal ones. In some cases, the time-series measurement was taken only in the first 24 hours or only during the next 24 hours. In this case, replacing with zero all the features related to the period with missing measurements could possibly create a non-existing improvement or deterioration trend. Instead, we duplicate the values from the available period, assuming stationarity conditions as default in absence of further measurements.

Cardiac Coronary Medical Surgical
N 874 577 1,481 1,067
Age 67.91 (5679) 69.22 (5981) 62.83 (5178) 60.50 (4876)
Male 530 (60.6%) 333 (57.7%) 753 (50.8%) 630 (59.0%)
Mortality Rate 4.9% (7.8%) 14.0% (14.6%) 18.6% (49.6%) 14.5% (28.0%)
Albumin (g/dL) 2.92 (2.43.5) 3.31 (2.93.6) 2.92 (2.53.3) 2.99 (2.53.5)
Alkaline phosphatase (IU/L) 74.93 (4683) 92.44 (59102) 126.15 (64138) 91.43 (5296)
Alanine transaminase (IU/L) 28.70 (1845) 68.14 (1978) 45.17 (1661) 72.11 (1784)
Aspartate transaminase (IU/L) 37.19 (2856) 32.41 (2655) 42.14 (2457) 34.90 (2453)
Bilirubin (mg/dL) 1.01 (0.41.1) 0.87 (0.40.9) 2.44 (0.41.6) 1.85 (0.51.5)
Cholesterol (mg/dL) 150.14 (114174) 163.59 (134189) 141.04 (111169) 157.87 (122184)
Creatinine (mg/dL) 1.04 (0.71.1) 1.58 (0.81.6) 1.64 (0.71.7) 1.12 (0.71.1)
Invasive diast. press. (mmHg) 58.85 (5166) 62.65 (5374) 54.97 (4870) 59.65 (5272)
Fractional inspired O2 0.91 (1.01.0) 0.82 (0.51.0) 0.72 (0.51.0) 0.72 (0.51.0)
Serum glucose (mg/dL) 129.28 (103145) 165.74 (114191) 155.02 (104175) 148.85 (114167)
Serum bicarbonate (mmol/L) 23.41 (2225) 23.31 (2126) 22.74 (1926) 23.44 (2126)
Hematocrit (%) 29.32 (25.332.8) 34.48 (30.737.8) 31.82 (27.936) 33.01 (29.136.8)
Heart rate (bpm) 85.43 (7991) 84.32 (6997) 95.61 (80110) 87.83 (74100)
Serum potassium (mEq/L) 4.49 (44.7) 4.28 (3.84.5) 4.19 (3.64.5) 4.07 (3.64.3)
Lactate (mmol/L) 2.76 (1.53.3) 2.76 (1.43) 2.58 (1.32.8) 2.65 (1.33.1)
Invasive mean press. (mmHg) 78.86 (6986) 86.14 (7399) 86.58 (6896) 87.13 (7398)
Serum sodium (mEq/L) 138.42 (136140) 137.82 (135140) 138.96 (136142) 139.33 (137142)
Non-invasive diast. press. (mmHg) 52.21 (4459) 61.15 (4972) 62.03 (5072) 62.42 (5273)
Non-invasive mean press. (mmHg) 71.53 (6279) 78.93 (6789) 80.55 (6891) 82.78 (7194)
Non-invasive syst. press. (mmHg) 110.88 (96125) 117.46 (101134) 121.78 (104138) 126.72 (108145)
Partial press. of art. CO2 (mmHg) 41.20 (3645) 40.61 (3545) 42.50 (3448) 41.01 (3545)
Partial press. of art. O2 (mmHg) 295.46 (218387) 181.58 (89248) 147.68 (78185) 188.24 (101250)
Arterial pH (0-14) 7.39 (7.357.44) 7.84 (7.317.43) 7.44 (7.37.42) 7.46 (7.327.43)
Platelets (cells/nL) 170.36 (117208) 241.44 (181283) 230.89 (143287) 219.19 (150268)
Respiration rate (bpm) 17.55 (1420) 19.74 (1623) 21.10 (1724) 18.95 (1621)
Invasive systolic press. (mmHg) 117.16 (105127) 117.65 (100139) 107.45 (95137) 123.33 (108148)
Temperature (C) 35.57 (35.536.6) 36.38 (3637.1) 36.77 (36.237.4) 36.51 (36.137.4)
Troponin-I (g/L) 6.77 (0.810.1) 10.05 (0.812.4) 5.59 (0.87) 7.02 (0.46.7)
Troponin-T (g/L) 1.51 (0.040.59) 2.78 (0.172.8) 0.33 (0.040.25) 0.22 (0.030.14)
Urine output (mL) 497.92 (120615) 365.62 (100500) 255.39 (70325) 389.29 (100500)
White blood cell (cells/nL) 12.98 (9.215.5) 12.31 (8.514.3) 13.33 (7.817) 12.37 (8.415.1)
TABLE I: Average patient physiological data. Mean, first and third quartiles within each physiological parameter. Mortality rate is concentrated in the Medical ICU (49.6% of all the deaths).

Iii-B Feature Transferability

Our goal is to train multi-domain models to predict patient outcomes over time, which is based on patient observations from multiple ICU domains. Although patients from a given ICU domain may be better represented by domain-specific features, there still exist some common features that permeate all other ICU domains.

The main intuition that we exploit for feature transferability is that the features must eventually transition from general to specific along our model and, accordingly to  [46], feature transferability drops significantly in higher layers with increasing domain discrepancy. In other words, the features computed in higher layers must depend strongly on a specific domain and prediction effectiveness suffers if this domain is discrepant from the target domain. Our proposal is to initialize the model with pretrained weights of source ICU domains, which are then fine-tuned with data from the target ICU domain. Since we are dealing with many domains simultaneously, we tested different transference approaches, which are detailed as follows:


No layer is kept frozen during fine-tuning, i.e., errors are back-propagated through the entire network during fine-tuning.


Only the convolutional layer is kept frozen during fine-tuning.


Convolutional and LSTM layers are kept frozen during fine-tuning, i.e., errors are back-propagated only thought the fully-connected layers during fine-tuning.


Only the convolutional layer is kept frozen during fine-tuning and other layers have their weights randomly initialized for fine-tuning.


Convolutional and LSTM layers are kept frozen during fine-tuning and weights in fully-connected layers are randomly initialized for fine-tuning.

Iv Experiments

In this section, we present the data we used to evaluate our multi-domain model for mortality prediction over time. Then, we discuss our evaluation procedure and report the results of our multi-domain model. In particular, our experiments aim to answer the following research questions:


Does domain adaptation improve mortality prediction? Do models that are specific to each ICU domain improve the state-of-the-art for mortality prediction?


Which feature transference approach is more appropriate to each ICU domain?


How effective and accurate are dynamic predictions?


How meaningful are the mortality risk spaces created from patient trajectories?

Iv-a Data and Domains

We use the publicly available dataset of multivariate clinical time-series of 4,000 patients from the PhysioNet 2012 challenge [41].The data for each patient includes age, gender, height, weight and 37 time-stamped physiological parameters measured during the first 48 hours of ICU stay. Patient outcomes, including mortality, are available. We resample the time series on an hourly basis and propagate measurements forward (or backward) in time to fill gaps. We scale each variable to fall into the interval. The source domain is composed of all ICU domains but the target one, which is used only during fine-tuning. In contrast to [6], we did not perform feature selection and, thus used the entire feature-set in all experiments.

Table I shows the average physiological data for patients in each ICU domain. The dataset also specifies the ICU domain to which the patient has been admitted: Cardiac Surgery, Coronary Care, Medical and Surgical. Physiological data differ greatly between patients admitted to different ICU domains. Figure 2 shows the frequency in which physiological parameters are measured within each ICU domain. Clearly, some ICU domains have a significantly larger number of observations than others (e.g., PaCO is much more frequently measured in the Cardiac ICU, while TroponinT is much more frequently measured in the Coronary ICU).

Fig. 2: (Color online) Relative frequency in which physiological parameters are measured in different ICU domains.

Iv-B Baselines

We considered the following methods in order to provide baseline comparison:

  • Shallow classifiers: Logistic Regression (LR), Support Vector Machines (SVM: Linear Kernel, C=0.1), Random Forest (RF: depth=10, random features, 200 trees). The main objective of using these baselines is to compare CNNLSTM with shallow models.

  • Training on Target (TT): A CNNLSTM model is trained using only the target domain data. No source domain data is used. The main objective of using this baseline is to assess the benefits of domain adaptation.

  • CNN and LSTM: A CNN and a LSTM model are trained using data from all domains. No fine-tuning is performed. The main objective of using this baseline is to assess the benefits of employing CNN and LSTM components together.

  • No tuning (NT): A CNNLSTM model is trained using data from all domains. No fine-tuning is performed. ICU source is included as an input variable, so the model is aware of the source. The main objective of using this baseline is to assess the benefits of domain adaptation.

  • Che et al., 2015 [9]: A deep network that uses data-driven prior-based regularization. The main objective of using this baseline is to compare our model with state-of-the-art results on the PhysioNet data.

  • Che et al., 2018 [10]: A recent Gated Recurrent Unit network which employs a missing value imputation approach which is similar to ours. Again, the main objective of using this baseline is to compare our model with state-of-the-art results on the PhysioNet data.

Model Cardiac Coronary Medical Surgical Avg
SVM 0.627 0.572 0.503 0.532 0.558
LR 0.629 0.601 0.510 0.517 0.564
RF 0.610 0.578 0.587 0.623 0.599
TT 0.821 0.769 0.722 0.727 0.759
LSTM 0.812 0.807 0.742 0.769 0.782
CNN 0.866 0.802 0.747 0.812 0.807
NT 0.876 0.833 0.737 0.801 0.812
NT 0.876 0.837 0.757 0.812 0.820
Che et al., 2015 0.853 0.802 0.760 0.785 0.800
Che et al., 2018 0.868 0.824 0.775 0.823 0.823
CNNLSTM 0.885 0.848 0.782 0.827 0.836
TABLE III: AUC numbers for different feature transference approaches. Numbers in bold indicate the best transference approach for each target ICU domain.
Target A1 A2 A3 A4 A5
Cardiac 0.852 0.885 0.829 0.849 0.858
Coronary 0.848 0.812 0.807 0.793 0.784
Medical 0.754 0.763 0.782 0.759 0.736
Surgical 0.822 0.827 0.808 0.818 0.788
Overall 0.819 0.822 0.806 0.804 0.791
TABLE II: AUC numbers for shallow and deep models. Numbers in bold indicate the best models for each ICU domain.

Iv-C Setup

We evaluate the effectiveness of the models using the standard Area Under the ROC Curve (AUC), as adopted by [9]. Like [20], we used five-fold cross validation and relevant hyper-parameters were found using a validation set. Each fold is split into three distinct subsets: one for training, with 64% of the patients, one for validation and parameter tuning, with 16% of the patients and the final set for testing the model, with the remaining 20% of the patients. As in other works [9, 10], test set leakage was prevented by ensuring that time-series data of a specific patient are either on the training or test set, and never on both [34, 30].

For CNNLSTM, learning rate was set to 0.001. We used Scaled Exponential Linear Unit [24] as non linear activations and a dropout probability of 0.2 for every layer. The 1D-CNN components employ 64 filters, kernel size was set to 5 with stride of 1. Max pooling size was set to 4. The LSTM components employ 70 neurons on the inner cell. Training was stopped after 15 epochs with no improvement. We used ADAM [23] in order to minimize the binary cross-entropy of the training set.

Fig. 3: (Color online) CNNLSTM AUC numbers for predictions performed using information within the first hours after the patient admission ( hours).

The results reported are the average of the five runs, and to ensure their relevance we assess the statistical significance of our measurements by means of a pairwise t-test [40] with pvalue . We perform a hand search for these hyper-parameters, tuning on the validation set, with early stopping. The best model was chosen according to the smallest loss on the validation set and are used to assess the overall performance of the models.

Iv-D Results and Discussion

The first experiment is devoted to answer Q1. Table III shows AUC numbers for each model. We report numbers for each ICU domain, and also the macro-averaged result. Clearly, CNNLSTM consistently outperforms all shallow baselines, and also [9]. Employing CNN and LSTM components together is beneficial, since NT is consistently superior than CNN and LSTM. Domain adaptation is beneficial for most of the domains. The only exception occurs with the Coronary domain for which performance remains statistically the same when compared with NT. Overall, CNNLSTM shows a macro-averaged AUC of 0.832.

The second experiment is concerned with Q2. Table III shows AUC numbers for CNNLSTM models learned following the different feature transference approaches. The best transference approach varies depending on the target ICU domain. Randomly initializing the weights for fine-tuning does not show to be the best approach, as A4 and A5 were not the best performers for any target domain. It seems that specific temporal patterns play an important role for mortality prediction in the Surgical domain, as A1 and A2 were the best approaches for this domain. For the Medical domain, A3 was the best approach, suggesting that features learned from other domains are effective. For the Cardiac and Coronary domains, A2 was the best transference approach, which indicates that specific features are important in this domain.

Fig. 4: (Color online) Gains over [10] at different prediction times ( hours).
Fig. 5: (Color online) Mortality risk space for different ICU domains. Regions in red are risky. Each axis is a t-SNE [43] non-linear combination of: (top row) physiological parameters, or (bottom row) features extracted by CNNLSTM.
Fig. 6: (Color online) Dynamics of 48-hour trajectories in different ICU domains. Red curves are computed from trajectories associated with patients that have died. Blue curves are computed from trajectories associated with patients that survived.

The next set of experiments is devoted to answer Q3. Figure 3 shows AUC numbers obtained with predictions performed using information acquired within the first hours after the admission. AUC increases as more information is acquired. From the first 5 to 20 hours, the slopes associated with Cardiac and Coronary domains increase much faster than the slopes associated with Medical and Surgical domains. Figure 4 shows the gains obtained when compared with [9] at different prediction times. Early predictions performed by the CNNLSTM architecture are much more accurate than those performed by [9], particularly in the first hours after admission. The 1020 hours period concentrates the more impressive gains, which vary from 4% (Medical) to almost 8% (Coronary).

The last set of experiments is concerned with Q4, i.e., to assess how meaningful are the mortality risk spaces. Figure 5 shows risk spaces for each ICU domain. These spaces are obtained by gathering patient trajectories, that is, the coordinates (i.e., CNNLSTM representations) along with the predicted outcome at each time. Risk spaces can also be obtained from raw data and, in this case, the coordinates are simply the entire feature-vector. Risk spaces created from CNNLSTM representations are much more meaningful than the corresponding spaces obtained from raw data.

Time is also encoded in the risk spaces, and thus we can exploit dynamics, such as the distance to the death centroid or the speed in which the patient condition changes. Figure 6 shows such dynamics in mortality risk spaces obtained from CNNLSTM representations. Dynamics associated with the mortality risk space for the Cardiac and Coronary ICU domains are highly discriminative since red and blue curves are separated in the first hours after the patient admission. This may explain the high AUC numbers obtained in these domains. Patients show distinct dynamics, depending on the ICU domain, i.e., patients admitted to the Cardiac and Surgical units move much faster than patients admitted to the Coronary and Medical units. Also, the speed increases over time for patients admitted to the Coronary and Medical units.

Iv-E Feature Importance Estimates

In order to interpret our model we chose to use a model agnostic representation of feature importance, where the impact of each feature on the model is represented using Shapley Additive Explanations, or simply SHAP [31]. SHAP values provide a theoretically justified method for allocation of credit among a group. In our models, the group is a set of interpretable input feature values, and the credit is the prediction made by the model when given those input feature values. Specifically, feature importance is defined as the change in prediction probability when a feature is observed vs. unknown. Some feature values have a large impact on the prediction, while others have little impact. Unless otherwise stated, we used our CNNLSTM model.

Fig. 7: (Color online) A patient who has survived the hospitalization in the Cardiac ICU. Each row shows a physiological parameter and how its importance varies with time. Diameter is proportional to the parameter value. Color indicates SHAP values: red points push the mortality risk higher, while blue points push the mortality risk lower.

Figure 7 shows a summary plot associated with a patient who has survived hospitalization. During the first hours of stay, the patient showed a mix of features contributing to survival and features contributing to death. The overall picture improved after the first 20 hours after admission, and the mortality risk has decreased significantly. Interesting to notice that our model was able to capture known but complex relationships, such as high glucose values inhibiting HCO3 [13].

Fig. 8: (Color online) A patient who has not survived the hospitalization in the Medical ICU. Each row shows a physiological parameter and how its importance varies with time. Diameter is proportional to the parameter value. Color indicates SHAP values: red points push the mortality risk higher, while blue points push the mortality risk lower.

Figure 8 shows a summary plot associated with a patient who has not survived hospitalization. The patient showed a large number of features contributing to death. Medical interventions have stabilized some of the physiological parameters, but then other parameters started contributing to death. In particular, low urine output is often used as a marker of acute kidney injury [28] and long-term low urine output increases lactate levels [4]. Despite changes in the physiological parameter values, mortality risk was always high for this patient.

V Conclusions

ICU mortality prediction is a domain-specific problem. Thus, a prediction model learned from a sub-population of patients is likely to fail when tested against data from other population. We investigated this problem by considering four sub-populations of patients that were admitted to different ICU domains. We showed that patients within a specific ICU domain are physiologically different from patients within other domains. Nevertheless, patients across ICU domains still share basic characteristics. This motivates us to propose mortality prediction models based on domain adaptation. Specifically, our models learn domain invariant representations from time series ICU data while transferring the complex temporal dependencies between ICU sub-populations. The proposed models employ temporal feature extractors, being thus able to perform dynamic predictions during the ICU stay, potentially leading to earlier diagnosis. Finally, our models produce a mortality risk space, and the dynamics associated with patient trajectories are meaningful and can be very discriminative, enabling clinicians to track risky trends and to gain insight into their treatment decisions. Our models provide significant gains (4% to 8%) for early predictions, i.e., predictions within the first hour period after admission. Gains (2.5% to 5%) are also observed for predictions performed based on information acquired during the first 48 hours after admission.

Acknowledgements. We thank the partial support given by the project MASWEB - Models, Algorithms and Systems for the Web (grant FAPEMIG/PRONEX/MASWeb APQ-01400-14), and authors’ individual grants and scholarships from CNPq, Fapemig and Kunumi. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Tesla K40 GPU used for this research.


  1. A. Albuquerque, T. Amador, R. Ferreira, A. Veloso and N. Ziviani (2018) Learning to rank with deep autoencoder features. In Proc. of the IJCNN, pp. 1–8. Cited by: §I.
  2. B. Alemayehu and K. Warner (2004) The lifetime distribution of health care costs. Health Services Res. 3 (39), pp. 627–642. Cited by: §I, §II-B.
  3. K. C. Barajas and R. Akella (2015) Dynamically modeling patient’s health state from electronic medical records: A time series approach. In Proc. of the 21st ACM SIGKDD, pp. 69–78. Cited by: §II-A.
  4. R. Bellomo (2002) Bench-to-bedside review: lactate and the kidney. Crit Care 6 (4), pp. 322–326. Cited by: §IV-E.
  5. D. Bera and M. Nayak (2012) Mortality risk assessment for ICU patients using logistic regression. Computing in Cardiology, pp. 493–496. Cited by: §II-A.
  6. S. Bhattacharya, V. Rajan and H. Shrivastava (2017) ICU mortality prediction: A classification algorithm for imbalanced datasets. In Proc. of the 31st AAAI, pp. 1288–1294. Cited by: §II-B, §II-B, §IV-A.
  7. L. Bonomi and X. Jiang (2017) A mortality study for ICU patients using bursty medical events. In Proc. of the 33rd IEEE ICDE, pp. 1533–1540. Cited by: §II-B.
  8. X. Cai, O. Concha, E. Coiera, F. Martin-Sanchez, R. Day, D. Roffe and B. Gallego (2016) Real-time prediction of mortality, readmission, and length of stay using electronic health record data. JAMIA 23 (3), pp. 553–561. Cited by: §II.
  9. Z. Che, D. Kale, W. Li, M. Bahadori and Y. Liu (2015) Deep computational phenotyping. In Proc. of the 21st ACM SIGKDD, pp. 507–516. Cited by: 5th item, §IV-C, §IV-D, §IV-D.
  10. Z. Che, S. Purushotham, K. Cho, D. Sontag and Y. Liu (2018) Recurrent neural networks for multivariate time series with missing values. Scientific Reports 8, pp. 1–12. Cited by: Fig. 4, 6th item, §IV-C.
  11. L. Citi and R. Barbieri (2012) Physionet 2012 challenge: predicting mortality of ICU patients using a cascaded SVM-GLM paradigm. Computing in Cardiology, pp. 257–260. Cited by: §II-A.
  12. G. Dafé, A. Veloso, M. Zaki and W. Meira Jr. (2015) Learning sequential classifiers from long and noisy discrete-event sequences efficiently. Data Min. Knowl. Discov. 29 (6), pp. 1685–1708. Cited by: 1st item.
  13. S. Futakuchi, H. Ishiguro, S. N. S, S. Ko, K. Fujiki, A. Yamamoto, M. Nakakuki, Y. Song, M. Steward, T. Kondo and H. Goto (2009) High glucose inhibits hco3 and fluid secretion in rat pancreatic ducts. Pflugers Archiv: European Journal of Physiology 459 (Nov), pp. 215–226. Cited by: §IV-E.
  14. J-R. L. Gall, S. Lemeshow and F. Saulnier (1993) A new simplified acute physiology score (SAPS II) based on a european/north american multicenter study. JAMA, Journal of the American Medical Association, pp. 2957–2963. Cited by: §II-A.
  15. M. Ghassemi, M. Pimentel, T. Naumann, T. Brennan, D. Clifton, P. Szolovits and M. Feng (2015) A multivariate time-series modeling approach to severity of illness assessment and forecasting in ICU with sparse, heterogeneous clinical data. In Proc. of the 29th AAAI, pp. 446–453. Cited by: §II-B.
  16. J. Gong, T. Naumann, P. Szolovits and J. Guttag (2017) Predicting clinical outcomes across changing electronic health record systems. In Proc. of the 23rd ACM SIGKDD, pp. 1497–1505. Cited by: §II-B.
  17. J. Gong, T. Sundt, J. Rawn and J. Guttag (2015) Instance weighting for patient-specific risk stratification models. In Proc. of the 21st ACM SIGKDD, pp. 369–378. Cited by: §II-A, §II-A.
  18. I. Hata, A. Veloso and N. Ziviani (2013) Learning accurate and interpretable classifiers using optimal multi-criteria rules. JIDM 4 (3), pp. 204–219. Cited by: 1st item.
  19. V. Huddar, B. K. Desiraju, V. Rajan, S. Bhattacharya, S. Roy and C. Reddy (2016) Predicting complications in critical care using heterogeneous clinical data. IEEE Access 4, pp. 7988–8001. Cited by: §II-B.
  20. A. Johnson, N. Dunkley, L. Mayaud, A. Tsanas, A. Kramer and G. Clifford (2012) Patient specific predictions in the intensive care unit using bayesian ensemble. Computing in Cardiology, pp. 249–252. Cited by: §II-A, §IV-C.
  21. A. Johnson, M. Ghassemi, S. Nemati, K. Niehaus, D. Clifton and G. Clifford (2016) Machine learning and decision support in critical care. Proc. of the IEEE 104 (2), pp. 444–466. Cited by: §II.
  22. A. Johnson, A. Kramer and G. Clifford (2014) Data preprocessing and mortality prediction: the physionet/cinc 2012 challenge revisited. Computing in Cardiology, pp. 157–160. Cited by: §II-A.
  23. D. Kingma and J. Ba (2015) Adam: a method for stochastic optimization. In Proc. of the 3rd ICLR, Cited by: §IV-C.
  24. G. Klambauer, T. Unterthiner, A. Mayr and S. Hochreiter (2017) Self-normalizing neural networks. In Proc. of the 30th NIPS, pp. 972–981. Cited by: §IV-C.
  25. M. Krajnak, J. Xue, W. Kaiser and W. Balloni (2012) Combining machine learning and clinical rules to build an algorithm for predicting ICU mortality risk. Computing in Cardiology, pp. 401–404. Cited by: §II-A.
  26. A. Krizhevsky, I. Sutskever and G. Hinton (2012) ImageNet classification with deep convolutional neural networks. In Proc. of the 26th NIPS, pp. 1106–1114. Cited by: §III-A.
  27. D. H. Lee and E. Horvitz (2017) Predicting mortality of intensive care patients via learning about hazard. In Proc. of the 31st AAAI, pp. 4953–4954. Cited by: §II-A.
  28. M. Legrand and D. Payen (2011) Bench-to-bedside review: lactate and the kidney. Ann Intensive Care, pp. 1–13. Cited by: §IV-E.
  29. Z. Lipton, D. Kale, C. Elkan and R. Wetzel (2016) Learning to diagnose with LSTM recurrent neural networks. In Proc. of the 4th ICLR, Cited by: §II-A.
  30. R. Lourenço, A. Veloso, A. Pereira, W. Meira Jr., R. Ferreira and S. Parthasarathy (2014) Economically-efficient sentiment stream analysis. In Proc. of the 37th ACM SIGIR, pp. 637–646. Cited by: §IV-C.
  31. S. Lundberg and S. Lee (2017) A unified approach to interpreting model predictions. In Proc. of the 30th NIPS, pp. 4768–4777. Cited by: 1st item, §IV-E.
  32. Y. Luo, Y. Xin, R. Joshi, L. Celi and P. Szolovits (2016) Predicting ICU mortality risk by grouping temporal trends from a multivariate panel of physiologic measurements. In Proc. of the 30th AAAI, pp. 42–50. Cited by: §II.
  33. A. Marczewski, A. Veloso and N. Ziviani (2017) Learning transferable features for speech emotion recognition. In Proc. of the ACM Multimedia 2017, pp. 529–536. Cited by: 1st item.
  34. C. Masdeval and A. Veloso (2015) Mining citizen emotions to estimate the urgency of urban issues. Inf. Syst. 54, pp. 147–155. Cited by: §IV-C.
  35. S. McMillan, C. Chia, A. Esbroeck, I. Rubinfeld and Z. Syed (2012) ICU mortality prediction using time series motifs. Computing in Cardiology, pp. 265–268. Cited by: §II-A.
  36. G. McNeill and D. Bryden (2013) Do either early warning systems or emmergency response teams improve hospital patient survival?. Resuscitation 84 (12), pp. 1652–1667. Cited by: §I.
  37. N. Nori, H. Kashima, K. Yamashita, S. Kunisawa and Y. Imanaka (2017) Learning implicit tasks for patient-specific risk modeling in ICU. In Proc. of the 31st AAAI, pp. 1481–1487. Cited by: §II-B, §II-B.
  38. V. Patel, E. Shortliffe, M. Stefanelli, P. Szolovits, M. Berthold, R. Bellazzi and A. Abu-Hanna (2009) The coming of age of artificial intelligence in medicine. Artificial Intelligence in Medicine 46 (1), pp. 5–17. Cited by: §II.
  39. S. Purushotham, W. Carvalho, T. Nilanon and Y. Liu (2017) Variational recurrent adversarial deep domain adaptation. In Proc. of the 5th ICLR, Cited by: §II-B, 1st item.
  40. T. Sakai (2014) Statistical reform in information retrieval?. SIGIR Forum 48 (1), pp. 3–12. Cited by: §IV-C.
  41. I. Silva, G. M. D. Scott, L. Celi and R. Mark1 (2012) Predicting in-hospital mortality of ICU patients: the physionet/computing in cardiology challenge. Computing in Cardiology, pp. 245–248. Cited by: 3rd item, §II-A, §IV-A.
  42. A. Vairavan, L. Eshelman, S. Haider, A. Flower and A. Seiver (2012) Prediction of mortality in an intensive care unit using logistic regression and hidden markov model. Computing in Cardiology, pp. 393–396. Cited by: §II-A.
  43. L. van der Maaten and G. Hinton (2008) Visualizing high-dimensional data using t-sne. Journal of Machine Learning Research 9 (Nov), pp. 2579–2605. Cited by: Fig. 5.
  44. D. Wang and E. Nyberg (2015) A long short-term memory model for answer sentence selection in question answering. In Proc. of the 53rd ACL Conference, pp. 707–712. Cited by: §I.
  45. H. Xia, B. Daley, A. Petrie and X. Zhao (2012) A neural network model for mortality prediction in ICU. Computing in Cardiology, pp. 261–264. Cited by: §II-A.
  46. J. Yosinski, J. Clune, Y. Bengio and H. Lipson (2014) How transferable are features in deep neural networks?. In Proc. of the 28th NIPS, pp. 3320–3328. Cited by: §III-B.
  47. G. Zuin, L. Chaimowicz and A. Veloso (2018) Learning transferable features for open-domain question answering. In Proc. of the IJCNN, pp. 1–8. Cited by: 1st item.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description