PIVETed-Granite: Computational Phenotypes through Constrained Tensor Factorization
It has been recently shown that sparse, nonnegative tensor factorization of multi-modal electronic health record data is a promising approach to high-throughput computational phenotyping. However, such approaches typically do not leverage available domain knowledge while extracting the phenotypes; hence, some of the suggested phenotypes may not map well to clinical concepts or may be very similar to other suggested phenotypes. To address these issues, we present a novel, automatic approach called PIVETed-Granite that mines existing biomedical literature (PubMed) to obtain cannot-link constraints that are then used as side-information during a tensor-factorization based computational phenotyping process. The resulting improvements are clearly observed in experiments using a large dataset from VUMC to identify phenotypes for hypertensive patients.
Computational phenotyping is the process of algorithmically deriving cohesive sets of clinical characteristics from collections of patient documentation like electronic health records (EHRs) (Pathak et al., 2013). For a set of computational phenotypes to be useful to clinicians, they should be 1) sparse (i.e., have relatively few elements) and 2) diverse (i.e., have few overlapping elements). Most importantly, they should map to clinically relevant concepts. Variations of a tensor factorization method called CANDECOMP/PARAFAC (CP) decomposition have shown potential in deriving computational phenotypes with these characteristics. CP decomposition factors multiway arrays, or tensors, into sets of rank-one components. In the application of computational phenotyping, the non-zero elements of each component can be interpreted as elements of a phenotype. Figure 1 shows examples of computational phenotypes derived through tensor factorization. Ho et al. (2014a) were the first to show tensor decomposition could be applied to tensors constructed from count data extracted from EHRs to derive phenotypes, a large number of which were clinically relevant. Subsequent models have been developed with the goal of deriving sparse, diverse, and interpretable phenotypes (Ho et al., 2014b; Wang et al., 2015; Henderson et al., 2017). Wang et al. (2015) incorporated guidance information from domain experts into the tensor decomposition process, but obtaining input from domain experts may not always be possible. Henderson et al. (2017) introduced a CP model called Granite with angular constraints to encourage diverse phenotypes and sparsity constraints to derive succinct phenotypes but found there was a trade-off between the diversity and the clinical meaningfulness of components.
One possible weakness of using CP decomposition is that there can be noise between and across the modes (i.e., elements appear together that do not belong together). In computational phenotyping, this noise could manifest as a medication and diagnosis co-occurring in a component but not actually having a clinical relationship. This weakness can degrade the interpretability of the fits (Henderson et al., 2017). Few tensor decomposition methods applied to clinical data have used supervision or domain expertise to increase the number of interpretable components (Wang et al., 2015). Like many problems in machine learning, incorporating supervision can be challenging and costly in terms of the time and domain expertise necessary for gathering labels or domain-specific constraints.
In this work, we explore a new proxy for domain-expertise via side information extracted using PIVET (Henderson et al., 2018), a phenotype validation tool. The goal is to increase the number of meaningful components in the CP decomposition process without human input. PIVET analyzes a corpus of publicly available medical journal articles to create evidence sets for candidate phenotypes. Our method, which we refer to as PIVETed-Granite, involves a novel application of PIVET to provide side information in the form of a cannot-link matrix between the diagnosis and medication modes of a tensor constructed from a set of EHRs. By automatically leveraging available biomedical literature, PIVETed-Granite enables more concise and diverse phenotypes that are discriminative and interpretable.
2. PIVETed-Granite Framework
Background. We briefly describe tensors and the process of decomposing them but refer the reader to (Kolda and Bader, 2009) for a more detailed account. A tensor is an -way or -mode array that is used to represent -dimensional relationships. We use CANDECOMP/PARAFAC (CP) decomposition (Harshman, 1970) to factor tensors. For a tensor with modes, the CP decomposition is
where each component of the sum is a rank-one tensor formed by taking the outer product of N vectors, . The representation is shorthand notation with the weight vector and the factor matrix where denotes the column of . CP decompositions are usually fit using a loss function that makes assumptions about the underlying distribution that generated the data in the tensor. When CP decomposition is applied to tensors constructed from EHRs, each rank-one tensor can be thought of a phenotype.
Problem Formulation. PIVETed-Granite is built on existing nonnegative CP decomposition algorithms that model the observed data using the Poisson distribution(Ho et al., 2014b, a; Henderson et al., 2017). We focus on a -mode tensor where the three dimensions are (1) patients, (2) diagnoses, and (3) medications, and each element is a count of the number of times a patient received a diagnosis and medication prescription in a given period of time. An observed tensor, is approximated as the sum of -way rank-one tensors , which are the patient, diagnosis, and medication factor matrices, respectively. To discourage specified diagnosis and medication pairs from appearing together in the same phenotype, PIVETed-Granite introduces a cannot-link matrix between the diagnosis () and the medication () factor matrices. The optimization problem for the observed tensor is:
For count data, the loss function is KL-divergence (2). An angular penalty term (3) discourages any factors from being too similar, where similarity is defined as the cosine angle between two factor vectors, and an penalty term controls the growth of the size of the factors (4) (See (Henderson et al., 2017) for details).
Incorporating PIVET. In Equation 5, is a binary cannot-link matrix defined as follows:
We construct with PIVET. PIVET calculates the lift for each (diagnosis, medication) pair (i.e., ) based on analysis of biomedical journal articles. A lift of much greater than indicates diagnosis and medication co-occur often and therefore may have a clinical relationship with one another, and a value of or less means diagnosis and medication do not co-occur often in the corpus and may not have a clinical relationship. In this work, we use . The terms in Equation 5 are of the form , and only contribute to the objective function if the diagnosis and the medication appear in the component. Since (5) is a soft constraint if there is actually a relationship between in the data, they can still appear together in components. However, if the relationship is weak in the data, these elements will be discouraged from appearing together in a phenotype.
Dataset Description. To explore the feasibility of using guidance from PIVET, we constructed a tensor from the diagnosis and medication counts of 1622 patients from the Synthetic Derivative (SD), a de-identified EHR database gathered at the VUMC(Roden et al., 2008). A panel of domain experts developed sets of characteristics (i.e., billing and medical codes) to identify patients as case and control for a set of diseases(Ritchie et al., 2010). Using these specifications, we included 304 resistant hypertension case patients and 399 resistant hypertension control patients in the tensor. For each case patient, we counted the medication and diagnosis interactions that occurred two years before they received the hypertension diagnosis. For each non-case patient, we counted the interactions that occurred two years before their last interaction with the VUMC. In their raw form, the diagnosis codes (International Classification of Diseases (ICD-9) system) capture a very detailed level of information. We use PheWAS coding to aggregate the diagnosis codes into broader categories(Denny et al., 2013) and Medical Subject Headings (MeSH) pharmacological terms from the RxClass RESTful API to group the medications into more general categories111https://rxnav.nlm.nih.gov/RxClassAPIs.html. These coarser hierarchies produced a tensor with 1622 patients by 1325 diagnoses by 148 medications.
|Average Number of Non-Zeros||Cosine Similarity|
|Marble||2803253.42 (194914.35)||26.72 (1.3)||7.01 (0.37)||8.44 (0.22)||0.07 (0.01)||0.01 (0.01)||0.24 (0.01)|
|Granite||2311866.35 (27826.92)||70.45 (1.69)||18.21 (1.06)||12.31 (0.31)||0.18 (0.01)||0.02 (0.01)||0.12 (0.01)|
|PIVETed-Granite||2224824.04 (19758.83)||57.21 (2.65)||5.78 (0.2)||5.89 (0.56)||0.20 (0.02)||0.03 (0.02)||0.05 (0.02)|
Quantitative Evaluation. We compare PIVETed-Granite to two baseline models, Granite and Marble (Ho et al., 2014a). Table 1 shows fit quality (KL-divergence), sparsity, and diversity measures for the three models (). PIVETed-Granite was the best fit to the data with the lowest KL-divergence, and also resulted in the smallest number of non-zero elements in the diagnosis and medication modes. Additionally, PIVETed-Granite resulted in diagnosis factors that were comparably diverse to those of Granite and medication factors that were more diverse than Granite. We also evaluate the effect of the cannot-link weight on the percentage of (diagnosis, medication) cannot-link pairs present in the factor matrices. In Figure 2, as increases, the percentage of cannot-link pairs decreases.
Additionally, we evaluated the discriminative capabilities of PIVETed-Granite in a prediction task where the patient factor matrix served as the feature matrix. We compared the performance of PIVETed-Granite, Granite, and Marble using logistic regression to predict which patients were hypertension case and control. The model ran with five 80-20 train-test splits, and the optimal LASSO parameter for the model was learned using 10-fold cross-validation. Table 2 shows the AUC for PIVETed-Granite, Granite, and Marble. The patient factor matrix derived using PIVETed-Granite resulted in the most discriminative model in this task.
Qualitative Exploration. To evaluate the effect of the cannot-link matrix on the decomposition process we initialized PIVETed-Granite and Granite fits with the same factors and then examined the differences between the fitted factors. Figure 3 shows one phenotype from each method initialized from the same factors. While the phenotypes are similar to one another, PIVETed-Granite’s characteristics form a more succinct, focused characterization of heart disease complicated with type 2 diabetes. Additionally, the Granite phenotype contains many cannot-link combinations (e.g., (“Fracture of foot”, “antidotes”)) whereas the PIVETed-Granite phenotype does not. The cannot-link constraints seem to result in phenotypes that are descriptive and cohesive.
As a way to qualitatively explore the clinical meaningfulness of the discovered phenotypes we identified patients who experienced acute myocardial infarctions (AMI), which resulted in a cohort of unique patients within the tensor. In Figure 1, we show the phenotypes with the highest proportions of AMI patients. These automatically generated phenotypes seem to give nuanced descriptions of patients who have AMIs. For example, in Phenotype one of the diagnoses is congestive heart failure, which is primarily caused by acute myocardial infarctions (Cahill and Kharbanda, 2017). Type-2 diabetes patients (Phenotype ) are also more likely to experience heart attacks and have more negative outcomes from them (Lago and Nesto, 2009).
|Method||AUC (st. dev.)|
4. Discussion and Conclusion
Adding guidance in the form of constraints to computational phenotyping models can help improve the quality of the fit and shows promise in increasing the clinical meaningfulness of derived phenotypes. However, obtaining informative constraints can be difficult and expensive in regard to time and effort required by domain experts. We show how to leverage publicly available information in the form of medical journals to guide the decomposition process to discriminative and interpretable phenotypes. PIVETed-Granite derived phenotypes that were more discriminative, more diverse, and sparser than two competing baseline models. In the future, we plan to further analyze the clinical interpretability of the PIVETed-Granite-derived phenotypes and experiment with the threshold used to create the cannot-link constraint matrix. Incorporating cannot-link constraints between modes is a general method that can be applied to many domains. In this application, the quality of the auxiliary information provided by PIVET seems to be high, but in other applications, it may not be. Our next step is to study how to incorporate auxiliary information when that side information is noisy.
- Cahill and Kharbanda (2017) Thomas J Cahill and Rajesh K Kharbanda. 2017. Heart failure after myocardial infarction in the era of primary percutaneous coronary intervention: Mechanisms, incidence and identification of patients at risk. World journal of cardiology 9, 5 (2017), 407.
- Denny et al. (2013) Joshua C Denny, Lisa Bastarache, Marylyn D Ritchie, and Robert J et al Carroll. 2013. Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data. Nat. Biotechnol. 31, 12 (Nov. 2013), 1102–1111.
- Harshman (1970) Richard A Harshman. 1970. Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multimodal factor analysis. UCLA Working Papers in Phonetics 16 (1970), 1–84.
- Henderson et al. (2017) Jette Henderson, Joyce C. Ho, Abel N Kho, Joshua C Denny, Bradley A Malin, Jimeng Sun, and Joydeep Ghosh. 2017. Granite: Diversified, Sparse Tensor Factorization for Electronic Health Record-Based Phenotyping. In Fifth IEEE Int. Conf. on Healthc. Inform. IEEE, 214–223.
- Henderson et al. (2018) Jette Henderson, Junyuan Ke, C. Joyce Ho, Joydeep Ghosh, and C. Byron Wallace. 2018. Phenotype Instance Verification and Evaluation Tool (PIVET): A Scaled Phenotype Evidence Generation Framework Using Web-Based Medical Literature. J. Med. Internet Res. 20, 5 (04 May 2018), e164.
- Ho et al. (2014b) Joyce C Ho, Joydeep Ghosh, Steve R Steinhubl, Walter F Stewart, Joshua C Denny, Bradley A Malin, and Jimeng Sun. 2014b. Limestone: High-throughput candidate phenotype generation via tensor factorization. J. Biomed. Inf. 52 (Dec. 2014), 199–211.
- Ho et al. (2014a) Joyce C Ho, Joydeep Ghosh, and Jimeng Sun. 2014a. Marble: High-throughput phenotyping from electronic health records via sparse nonnegative tensor factorization. In Proc. of ACM Int. Conf. on Knowl. Discov. Data Min. 115–124.
- Kolda and Bader (2009) Tamara G Kolda and Brett W Bader. 2009. Tensor decompositions and applications. SIAM Rev. 51, 3 (2009), 455–500.
- Lago and Nesto (2009) Rodrigo M Lago and Richard W Nesto. 2009. Type 2 diabetes and coronary heart disease: focus on myocardial infarction. Current diabetes reports 9, 1 (2009), 73–78.
- Pathak et al. (2013) Jyotishman Pathak, Abel N Kho, and Joshua C Denny. 2013. Electronic health records-driven phenotyping: challenges, recent advances, and perspectives. Journal of the Am. Med.Inf. Assoc. (JAMIA) 20, e2 (2013), e206–11.
- Petersen and Pedersen (2008) Kaare Brandt Petersen and Michael Syskind Pedersen. 2008. The Matrix Cookbook. Technical University of Denmark (2008).
- Ritchie et al. (2010) Marylyn D Ritchie, Joshua C Denny, Dana C Crawford, Andrea H Ramirez, Justin B Weiner, et al. 2010. Robust replication of genotype-phenotype associations across multiple diseases in an electronic medical record. Am. J. Hum. Genet. 86, 4 (2010), 560–572.
- Roden et al. (2008) D. M. Roden, J. M. Pulley, M. A. Basford, G. R. Bernard, E. W. Clayton, J. R. Balser, and D. R. Masys. 2008. Development of a large-scale de-identified DNA biobank to enable personalized medicine. Clin. Pharmacol. Ther. 84, 3 (2008), 362–369.
- Wang et al. (2015) Yichen Wang, Robert Chen, Joydeep Ghosh, Joshua C Denny, Abel Kho, You Chen, Bradley A Malin, and Jimeng Sun. 2015. Rubik: Knowledge guided tensor factorization and completion for health data analytics. In Proc. of ACM SIGKDD Int. Conf. on Know. Disc. and Data Min. 1265–1274.