Predicting Infectiousness for Proactive Contact Tracing
The COVID-19 pandemic has spread rapidly worldwide, overwhelming manual contact tracing in many countries and resulting in widespread lockdowns for emergency containment. Large-scale digital contact tracing (DCT)
Until pharmaceutical interventions such as a vaccine become available, control of the COVID-19 pandemic relies on nonpharmaceutical interventions such as lockdown and social distancing. While these have often been successful in limiting spread of the disease in the short term, these restrictive measures have important negative social, mental health, and economic impacts. Digital contact tracing (DCT), a technique to track the spread of the virus among individuals in a population using smartphones, is an attractive potential solution to help reduce growth in the number of cases and thereby allow more economic and social activities to resume while keeping the number of cases low.
Most currently deployed DCT solutions use binary contact tracing (BCT), which sends a quarantine recommendation to all recent contacts of a person after a positive test result. While BCT is simple and fast to deploy, and most importantly can help curb spread of the disease (Abueg et al., 2020), epidemiological simulations by Hinch et al. (2020) suggest that using only one bit of information about the infection status can lead to quarantining many healthy individuals while failing to quarantine infectious individuals. Relying only on positive test results as a trigger is also inefficient for a number of reasons: (i) Tests have high false negative rates (Li et al., 2020); (ii) Tests are administered late, only after symptoms appear, leaving the asymptomatic population, estimated 20%-30% of cases (Gandhi et al., 2020), likely untested; (iii) It is estimated that infectiousness is highest before symptoms appear, well before someone would get a test (Heneghan et al., 2020), thus allowing them to infect others before being traced, (iv) Results typically take at least 1-2 days, and (v) In many places, tests are in limited supply.
Recognizing the issues with test-based tracing, Gupta et al. (2020) propose a rule-based system leveraging other input clues potentially available on a smartphone (e.g. symptoms, pre-existing medical conditions), an approach they call feature-based contact tracing (FCT). Probabilistic (non-binary) approaches, using variants of belief propagation in graphical models, e.g. (Baker et al., 2020; Satorras & Welling, 2020; Briers et al., 2020), could also make use of features other than test results to improve over BCT, although these approaches rely on knowing the social graph, either centrally or via distributed exchanges between nodes. The latter solution may require many bits exchanged between nodes (for precise probability distributions), which is challenging both in terms of privacy and bandwidth. Building on these works, we propose a novel FCT methodology we call proactive contact tracing (PCT), in which we use the type of features proposed by Gupta et al. (2020) as inputs to a predictor trained to output proactive (before current-day) estimates of expected infectiousness (i.e. risk of having infected others in the past and of infecting them in the future). The challenges of privacy and bandwidth motivated our particular form of distributed inference where we pretrain the predictor offline and do not assume that the messages exchanged are probability distributions, but instead just informative inputs to the node-level predictor of infectiousness.
We use a recently proposed COVID-19 agent-based simulation testbed (Gupta et al., 2020) called COVIsim to compare PCT to other contact tracing methods under a wide variety of conditions. We develop deep learning predictors for PCT in concert with a professional app-development company, ensuring inference models are appropriate for legacy smartphones. By leveraging the rich individual-level data produced by COVIsim to train predictors offline, we are able to perform individual-level infectiousness predictions locally to the smartphone, with sensitive personal data never required to leave the device. We find deep learning based methods to be consistently able to reduce the spread of the disease more effectively, at lower cost to mobility, and at lower adoption rates than other predictors. These results suggest that deep learning enabled PCT could be deployed in a smartphone app to help produce a better trade-off between the spread of the virus and the economic cost of mobility constraints than other DCT methods, while enforcing strong privacy constraints.
1.1 Summary of technical contributions
We examine the consequential problem of COVID-19 infectiousness prediction and propose a new method for contact tracing, called proactive contact tracing (see Sec. 2).
In order to perform distributed inference with deep learning models, we develop an architectural scaffolding whose core is any set-based neural network. We embed two recently proposed networks, namely Deep Sets (Zaheer et al., 2017) and Set Transformers (Lee et al., 2018) and evaluate the resulting models via the COVIsim testbed (Gupta et al., 2020) (see Sec. 3.1).
To our knowledge the combination of techniques in this pipeline is entirely novel, and of potential interest in other settings where privacy, safety, and domain shift are of concern. Our training pipeline consists of training an ML infectiousness predictor on the domain-randomized output of an agent-based epidemiological model, in several loops of retraining to mitigate issues with (i) non-stationarity and (ii) distributional shift due to predictions made by one phone influencing the input for the predictions of other phones. (see Sec. 3.2)
To our knowledge this is the first work to apply and benchmark a deep learning approach for probabilistic contact tracing and infectiousness risk assessment. We find such models are able to leverage weak signals and patterns in noisy, heterogeneous data to better estimate infectiousness compared to binary contact tracing and rule-based methods (see Sec. 4)
2 Proactive Contact Tracing
Proactive contact tracing (PCT) is an approach to digital contact tracing which leverages the rich suite of features potentially available on a smartphone (including information about symptoms, preexisting conditions, age and lifestyle habits if willingly reported) to compute proactive estimates of an individual’s expected infectiousness. These estimates are used to (a) provide an individualized recommendation and (b) propagate a graded risk message to other people who have been in contact with that individual (see Fig. 1). This stands in contrast with existing approaches for contact tracing, which are either binary (recommending all-or-nothing quarantine to contacts), or require centralized storage of the contact graph or other transfers of information which are incompatible with privacy constraints in many societies. Further, the estimator runs locally on the individual’s device, such that any sensitive information volunteered does not need to leave the device.
In Section 2.1, we formally define the general problem PCT solves. In Section 2.2, we describe how privacy considerations inform and shape the design of the proposed framework and implementation. Finally, in Section 3.1, we introduce deep-learning based estimators of expected infectiousness, which we show in Section 4 to outperform DCT baselines by a large margin.
2.1 Problem Setup
We wish to estimate infectiousness of an agent on day , given access to locally observable information now on day and over the past days () Some of the information available on day is static, including reported age, sex, pre-existing conditions, and lifestyle habits (e.g. smoking), denoted , the health profile. Other information is measured each day: the health status of reported symptoms and known test results. Finally, also includes information about encounters in the last days, grouped in for day . Thus:
The information about the encounters from day is subject to privacy constraints detailed in Section 2.2 but provides indications about the estimated infectiousness of the persons encountered at , given the last available information by these contacts as of day , hence these contacts try to estimate their past infectiousness a posteriori. Our goal is thus to model the history of the agent’s infectiousness (in the last days), which is what enables the recommendations of PCT to be proactive and makes it possible for an infected asymptomatic person to receive a warning from their contact even before they develop symptoms, because their contact obtained sufficient evidence that they were contagious on the day of their encounter. Formally, we wish to model , where is the vector of present and past infectiousness of agent and specifies the parameters of the predictive model. In our experiments we only estimate conditional expectations with a predictor , with an estimate of the conditional expected per-day present and past infectiousness .
The predicted expected values are used in two ways. First, they are used to generate messages transmitted on day to contacts involved in encounters on day .
These messages contain the estimates of the expected infectiousness of at day , quantized to 4 bits for privacy reasons discussed in section 2.2.
Second, the prediction for today is also used to form
a discrete recommendation level regarding the behavior of agent at day via a recommendation mapping , i.e. .
Here we make two important observations about contact tracing: (1) There is a trade-off between decelerating spread of disease, measured by the reproduction number , or as number of cases,
2.2 Privacy-preserving PCT
One of the primary concerns with the transmission and centralization of contact information is that someone with malicious intent could identify and track people. Even small amounts of personal data could allow someone to infer the identities of individuals by cross-referencing with other sources, a process called re-identification (El Emam et al., 2011). Minimizing the number of bits being transmitted and avoiding information that makes it easy to triangulate people is a protection against big brother attacks, where a central authority with access to the data can abuse its power, as well as against little brother attacks, where malicious individuals (e.g. someone you encounter) could use your information against you. Little brother attacks include vigilante attacks: harassment, violence, hate crimes, or stigmatization against individuals which could occur if infection status was revealed to others. Sadly, there have been a number of such attacks related to COVID-19 (Russell, 2020). To address this, PCT operates with (1) no central storage of the contact graph; (2) de-identification (Sweeney, 2002) and encryption of all data leaving the phones; (3) informed and optional consent to share information (only for the purpose of improving the predictor); and (4) distributed inference which can achieve accurate predictions without the need for a central authority. Current solutions (Woodhams, 2020) share many of these goals, but are restricted to only binary CT. Our solution achieves these goals while providing individual-level graded recommendations.
Each app records contacts which are defined by Health Canada (2020) as being an encounter which lasts at least 15 minutes with a proximity under 2 meters. Once every 6 hours, each application processes contacts, predicts infectiousness for days to , and may update its current recommended behavior. If the newly predicted infectiousness history differs from the previous prediction on some day (e.g. typically because the agent enters new symptoms, receives a negative or positive test result, or receives a significantly updated infectiousness estimate), then the app creates and sends small update messages to all relevant contacts. These heavily quantized messages reduce the network bandwidth required for message passing as well as provide additional privacy to the individual. We follow the communication and security protocol for these messages introduced by Alsdurf et al. (2020). We note that a naive method would tend to over-estimate infectiousness in these conditions, because repeated encounters with the same person (a very common situation, for example people living in the same household) should carry a lower predicted infectiousness than the same number of encounters with different people. To mitigate this over-estimation while not identifying anyone, messages are clustered based on time of receipt and risk level as in Gupta et al. (2020).
3 Methodology for Infectiousness Estimation
3.1 Distributed Inference
Distributed inference networks seek to estimate marginals by passing messages between nodes of the graph to make predictions which are consistent with each other (and with the local evidence available at each node). Because messages in our framework are highly constrained by privacy concerns in that we are prevented from passing distributions or continuous values between identifiable nodes, typical distributed inference approaches (e.g. loopy belief propagation (Murphy et al., 1999), variational message passing (Winn & Bishop, 2005), or expectation propagation (Minka, 2001)) are not readily applicable. We instead propose an approach to distributed inference which uses a trained ML predictor for estimating these marginals (or the corresponding expectations). The predictor is trained offline on simulated data to predict expected infectiousness from the locally available information for agent on day . The choice of ML predictor is informed by several factors. First and foremost, we expect the input (a component of ) to be variable-sized, because we do not know a priori how many contacts an individual will have on any day. Further, privacy constraints dictate that the contacts within the day cannot be temporally ordered, implying that the quantity is set-valued. Second, given that we are training the predictor on a large amount of domain-randomized data from many epidemiological scenarios (see Section 3.2), we require the architecture to be sufficiently expressive. Finally, we require the predictor to be easily deployable on edge devices like legacy smartphones.
To these ends, we construct a general architectural scaffold in which any neural network that maps between sets may be used. In this work, we experiment with Set Transformers (Lee et al., 2018) and a variant of Deep Sets (Zaheer et al., 2017). The former is a variant of Transformers (Vaswani et al., 2017), which model pairwise interactions between all set elements via multi-head dot-product attention and can therefore be quite expressive, but scales quadratically with elements. The latter relies on iterative max-pooling of features along the elements of the set followed by a broadcasting of the aggregated features, and scales linearly with the number of set elements (see Figure 2).
In both cases, we first compute two categories of embeddings: per-day and per-encounter. Per-day, we have embedding MLP modules of health status, for the health profile, and a linear map for the day-offset . Per-encounter, the cluster matrix can be expressed as the set where is in the set of putative anonymous persons encountered by at , is the risk level sent from to and is the number of repeated encounters between and at . Accordingly, we define the risk level embedding function and for the number of repeated encounters. is parameterized by an embedding matrix, while by the vector
which resembles the positional encoding in (Vaswani et al., 2017) and counteracts the spectral bias of downstream MLPs (Rahaman et al., 2019; Mildenhall et al., 2020). We tried alternative encoding schemes like thermometer encodings (Buckman et al., 2018), but they did not perform better in our experiments. Next, we join the per-day and per-encounter embeddings to obtain and respectively. Where is the concatenation operation, we have:
The union of and forms the input to a set neural network , that predicts infectiousness:
The trunk of both models – deep-set (DS-PCT) and set-transformer (ST-PCT) – is a sequence of 5 set processing blocks (see Figure 2). A subset of outputs from these blocks (corresponding to ) is processed by a final MLP to yield . As a training objective for agent , we minimize the Mean Squared Error (MSE) between (which is generated by the simulator) and the prediction . We treat each agent as a sample in batch to obtain the sample loss The net loss is the sum of over all agents .
3.2 Training Procedure
Unlike existing contact tracing methods like BCT and the NHSx contact tracing app (Briers et al., 2020) that rely on simple and hand-designed heuristics to estimate the risk of infection, our core hypothesis is that methods using a machine-learning based predictor can learn from patterns in rich but noisy signals that might be available locally on a smartphone. In order to test this hypothesis prior to deployment in the real world, we require a simulator built with the objective to serve as a testbed for app-based contact tracing methods. We opt to use COVIsim (Gupta et al., 2020), which features an agent specific virology model together with realistic agent behaviour, mobility, contact and app-usage patterns. With the simulator, we generate large datasets of samples comprising the input and target variables defined in sections 2.1 and 3.1. Most importantly for our prediction task, this includes a continuous-valued infectiousness parameter as target for each agent.
We use runs of the simulator to generate this dataset, where each simulation is configured with parameters randomly sampled from selected intervals (See Appendix Domain randomization). These parameters include the adoption rate of the contact tracing app, initial fraction of exposed agents in the population, the likelihood of agents ignoring the recommendations, and strength of social distancing and mobility restriction measures. There are two reasons for sampling over parameter ranges: First, the intervals these parameters are sampled from reflect both the intrinsic uncertainty in the parameters, as well as the practical setting and limitations of an app-based contact tracing method (e.g. we only care about realistic levels of app usage). Second, randomly sampling these key parameters significantly improves the diversity of the dataset, which in turn yields predictors that can be more robust when deployed in the real world. The overall technique resembles that of domain-randomization (Tobin et al., 2017; Sadeghi & Levine, 2016) in the Sim2Real literature, where its efficacy is well studied for transfer from RGB images to robotic control, e.g. (Chebotar et al., 2018; OpenAI et al., 2018).
We use runs for training and the remaining for validation (full training and other reproducibility details are in Appendix Appendix 4: Experimental details). The model with the best validation score is selected for online evaluation, wherein we embed it in the simulator to measure the reduction in the as a function of the average number of contacts per agent per day. While the evaluation protocol is described in section 4, we now discuss how we mitigate a fundamental challenge that we share with offline reinforcement learning methods, that of auto-induced distribution shift when the model is used in the simulation loop (Levine et al., 2020; Krueger et al., 2020).
To understand the problem, consider the case where the model is trained on simulation runs where PCT is driven by ground-truth infectiousness values, i.e. an oracle predictor. The predictions made by an oracle will in general differ from ones made by a trained model, as will resulting dynamics of spread of disease. This leads to a distribution over contacts and epidemiological scenarios that the model has not encountered during training. For example, oracle-driven PCT might even be successful in eliminating the disease in its early phases – a scenario unlikely to occur in model-driven simulations. For similar reasons, oracle-driven simulations will also be less diverse than model-driven ones. To mitigate these issues, we adopt the following strategy: First, we generate an initial dataset with simulations driven by a noisy-oracle, i.e. we add multiplicative and additive noise to the ground-truth infectiousness to partially emulate the output distribution resulting from trained models. The corresponding noise levels are subject to domain-randomization (as described above), resulting in a dataset with some diversity in epidemiological scenarios and contact patterns. Having trained the predictor on this dataset (until early-stopped), we generate another dataset from simulations driven by the thus-far trained predictor, in place of the noisy-oracle. We then fine-tune the predictor on the new dataset (until early stopped) to obtain the final predictor. This process can be repeated multiple times, in what we call iterative retraining. We find three steps yields a good trade-off between performance and compute requirement.
We evaluate the proposed PCT methods, ST-PCT and DS-PCT, and benchmark them against: test-based BCT; a rule-based FCT Heuristic method proposed by Gupta et al. (2020)
EXP1: In Figure 3,
we plot the Pareto frontier between spread of disease () and the amount of restriction imposed. To traverse the frontier at 60% adoption rate, we sweep through multiple values of the simulator’s global mobility scaling factor, a parameter which controls the strength of social distancing and mobility restriction measures.
Each method uses agents for days with random seeds, and we plot the resulting number of contacts per day per human
We fit a Gaussian Process Regressor to the simulation outcomes and highlight the average reduction in obtained in the vicinity of , finding
DS-PCT and ST-PCT reduce to below 1 at a much lower cost to mobility on average.
For subsequent plots, we select the number of contacts per day per human such that the no-tracing baseline yields a realistic post-lockdown of around (Brisson & et al., 2020)
EXP2: Figure 4 compares various DCT methods in terms of their cumulative cases and their fraction of false quarantine: the number of healthy agents the method wrongly recommends to quarantine.
Again, all DCT methods have a clear advantage over the no-tracing baseline, while the number of agents wrongly recommended quarantine is much lower for ML-enabled PCT.
EXP3: In Figure 5 Left we plot the bootstrapped distribution of mean for different DCT methods. Recall is an estimate of how many other agents an infectious agent will infects, i.e. even a numerically small improvement in could yield an exponential improvement in the number of cases. We find that both PCT methods yield a clear improvement over BCT and the rule-based heuristic, all of which significantly improve over the no-tracing baseline. We hypothesize this is because deep neural networks are better able to capture the non-linear relationship between features available on the phone, interaction patterns between agents, and individual infectiousness.
EXP4: In Figure 5 Right we investigate the effect of iterative retraining on the machine-learning based methods, DS-PCT and ST-PCT. We evaluate all 3 iterations of each method in the same experimental setup as in Figure 5 and find that DS-PCT benefits from the 3 iterations, while ST-PCT saturates at the second iteration. We hypothesize that this is a form of overfitting, given that the set transformer (ST) models all-to-all interactions and is therefore more expressive than deep-sets (DS).
EXP5: In Figure 6, we analyze the sensitivity of the various methods to adoption rate, which measures what percent of the population actively uses the CT app. Adoption rate is an important parameter for DCT methods, as it directly determines the effectiveness of an app. We visualize the effect of varying the adoption rate on the reproduction number . Similarly to prior work (Abueg et al., 2020), we find that all DCT methods improve over the no-tracing baseline even at low adoptions and PCT methods dominate at all levels of adoption.
Our results demonstrate the potential benefit of digital contact tracing approaches for saving lives while reducing mobility restrictions and preserving privacy, independently confirming previous reports (Ferretti et al., 2020; Abueg et al., 2020). Of all methods in our study, we find that deep learning based PCT provides the best trade-off between restrictions on mobility and reducing the spread of disease under a range of settings, making it a potentially powerful tool for saving lives in a safe deconfinement. This area of research holds many interesting avenues of future work in machine learning, including: (1) The comparison of methods for fitting parameters of the epidemiological simulator data using spatial information (not done here to avoid the need for GPS), (2) Using reinforcement learning to learn (rather than expert hand-tune) the mapping from estimated infectiousness and individual-level features (number of contacts, age, etc.)
Accurate and practical models for contact tracing are only a small part of the non-pharmaceutical efforts against the pandemic, a response to which is a complex venture necessitating cooperation between public health, government, individual citizens, and scientists of many kinds - epidemiologists, sociologists, behavioural psychologists, virologists, and machine learning researchers, among many others. We hope this work can play a role in fostering this necessary collaboration.
This project is motivated by the goal of saving more lives than manual tracing or standard digital tracing which are both putting people in only two categories. This can be achieved by providing early graded warning of possible infectiousness, even before a person has symptoms, thanks to the combination of risk levels propagating across smartphones and the ability to predict those risk levels with machine learning. The potential to save lives is of huge benefit, but it must be appropriately weighed against the costs. This kind of mobile application raises serious questions about privacy, dignity and democracy which cannot be ignored. We contributed to a much longer paper which considers these social impact questions at length (a full citation will be provided in the non-anonymous version of the paper). See also (Bengio et al., 2020) on the privacy of COVID-19 tracing apps, which is aligned with this work. We created a non-profit organization - a form of data trust - independent of governments and corporations to manage the project, with a board which includes representatives of civil society in addition to experts in public health, epidemiology, ethics, and machine learning, as well as representatives of governments. This organization would disappear (along with any data) at the end of the pandemic, and its sole mission is to protect the health, privacy and dignity of people in accordance with the principles in the Montreal Declaration for the Responsible Development of AI.
Of note here is how these considerations have already imposed serious constraints on the information available to the risk predictor and training procedure (see Section 2.2 above). Clearly, people who do not have a smartphone (around 15% in our country) would not be able to receive recommendations. This might mean that they would not be able to help prevent others in their community from being infected by them, but they would benefit from a reduction in spread of the disease due to others who are equipped with a smartphone and using the app. There are also hardware makers building cheap Bluetooth devices (e.g. 5$) which could be used by such people (think especially about older people and children, in particular, who might not be able to use a smartphone). Failing to convince enough users to adopt the app could mean that the privacy risks taken by those who use the app and share their data with the ML server might result in little collective benefit. To mitigate this, the proposed app provides a coarse hotspot map functionality to Public Health by aggregating risk statistics over a neighbourhood or small town. This would be useful even if a very small fraction of the population (like 10%) were to use the app. Such a map would provide warnings of increasing risk in a neighborhood even before the outbreak translates into increased hospitalizations (because symptoms and contact information are often available long before a test result is obtained). Even at that low level of participation, the data collected would be of great value in terms of epidemiological modeling, i.e., to better understand the disease and its transmission, and thus guide public health policies. Another social question of interest is whether an app like this would favor particular groups in society, for example, the most wealthy (because they have more smartphones and possibly more app uptake will be seen in their community). We propose that such issues be monitored by using census data to correlate prediction error with socio-economic status and race (on a per-neighborhood basis). If such disparities are found, then a mitigating strategy would be to reweigh (or oversample) examples from these disadvantaged communities in order to reduce the error bias.
We thank all members of the broader COVI project https://covicanada.org for their dedication and teamwork; it was a pleasure and honour to work with such a great team. We also thank the Mila and Empirical Inference communities, in particular Xavier Bouthillier, Vincent Mai, Gabriele Prato, and Georgios Arvanitidis, for detailed and helpful feedback on early drafts. The authors gratefully acknowledge the following funding sources: NSERC, IVADO, CIFAR. This project could not have been completed without the resources of MPI-IS cluster, Compute Canada & Calcul Quebec, in particular the Beluga cluster. In gratitude we have donated to a project to help understand and protect the St. Lawrence beluga whales for whom the cluster is named https://baleinesendirect.org/
Appendix 1: Glossary of bolded terms
Finding the people who have been in contact with an infected person, typically using information such as test results and phone surveys, and recommending that they quarantine themselves to prevent further spread of the disease.
Manual contact tracing
Method for contact tracing using trained professionals who interview diagnosed individuals to identify people that have come into contact with an infected person and recommend a change in their behavior (or further testing).
Digital contact tracing (DCT)
Predominantly smartphone-based methods of identifying individuals at high risk of contracting an infectious disease based on their interactions, mobility patterns, and available medical information such as test results.
Binary contact tracing (BCT)
Methods which use binary information (e.g. tested positive or negative) to perform contact tracing, thus putting users in binary risk categories (at-risk or not-at-risk).
Proactive contact tracing (PCT)
DCT methods which produce graded risk information to mitigate spread of disease, thus putting users in graded risk categories (more-at-risk or less-at-risk), potentially using non-binary clues like reported symptoms which are available earlier than test results.
Agent-based epidemiological models (ABMs)
Also called individual-based, this type of simulation defines rules of behaviour for individual agents, and by stepping forward in time, contact patterns between agents are generated in a “bottom up” fashion. Contrast with Population-level epidemiological models.
Risk of infection
Expected infectiousness, or probability of infection for a susceptible agent by an infectious agent given a qualifying contact (15+ minutes at under 2 meters).
An encounter between 2 agents which lasts at least 15 minutes with a distance under 2 meters (Health Canada, 2020).
A version of the risk of infection quantized into 16 bins (for reducing the number of bits being exchanged, for better privacy protection). The thresholds are selected by running the domain randomization with separate seeds and grouping risk messages such that there are an approximately uniform number in each bin.
A table which maps floating point risk scalars into one of 16 discrete risk levels.
A table of which recommendation level should be associated with a given risk level. Each recommendation level comes with a series of specific recommendations (e.g. “Limit contact with others”, “wash hands frequently”, “wear a mask when near others”, “avoid public transportation”, etc) which should be shown for a given risk level. Reminders of these recommendations may be sent via push notification more or less often depending on the recommendation level.
Population-level epidemiological models
This type of model fits global statistics of a population with a mathematical model, typically sets of ordinary differential equations. The equations track some statistics over time, typically counts of people in each of a few “compartments”, e.g. the number of Infected and Recovered, and the parameters of the equations are often tuned to match statistics collected from real data. It does not give any information about the individual-level contact patterns which give rise to these statistics.
Compartment model (SEIR)
A compartment model tracks the counts of agents in each of several mutually-exclusive categories called “compartments”. In an SEIR model, the 4 compartments are Susceptible, Exposed, Infectious, and Recovered (see entries for each of these words).
At risk of catching disease, but not infected.
Exposed or infected
Infected with the disease, (i.e. potentially carries some viral load).
Carries sufficient viral load to transmit the disease to others. In real life, this typically means that the virus has multiplied sufficiently to overwhelm the immune system.
No longer carries measurable viral load, after having been infected. In real life this is typically measured by two successive negative lab tests.
Viral load is the number of actual viral RNA in a person as measured by a lab test.
Effective viral load
A term we introduce representing a number between 0 and 1 which we use as a proxy for viral load. It could be converted to an actual viral load via multiplying by the maximum amount of viral RNA detectable by a lab test.
Degree to which an agent is able to transmit viruses to another agent. A factor in determining whether an encounter between a susceptible and an infectious agent results in the susceptible agent being exposed (it is a property of the infector agent; this does not consider environmental factors or the other agent’s susceptibility; see Transmission probability).
The proportion of individuals in a population who are infected over some time period.
How long a location remains infectious after an infected person has visited it and shed viruses there.
The chance that one agent infects another during an encounter. Infectiousness of the infected agent, susceptibility of the other agent, both their behaviour (e.g. usage of masks) and other environmental factors all play a role in determining this probability.
Also, colloquially referred to as a lab test. In this context, this means a Polymerase Chain Reaction (PCR) test used to amplify viral DNA samples, typically obtained via a nasopharangeal swab.
When an infected person shows no symptoms. For COVID-19, many people remain asymptomatic for the entire course of their disease.
The number of days from exposure (infection) until an agent shows symptoms. In real life this is the average amount of time it takes the viral RNA to multiply sufficiently in / burst out of host cells and cause an immune response, which is what produces the symptoms.
Reproduction number ()
The average number of other agents an agent infects, measured over a certain window of time. We follow Gupta et al. (2020) and approximate by computing the infection tree and taking the ratio of the number of infected children divided by the number of parents who are recovered infectors.
The average number of days between successive symptom onsets (i.e. between the incubation time of the infector and the infectee).
The number of days between successive infections (i.e. between the exposure of the infector and the infectee). Generation time is difficult to measure directly so one can estimate it from the serial interval.
A method of generating a training dataset by sampling from several distributions, each corresponding to a different setting of parameters of a simulator. This allows to generate a dataset which covers a potentially wide range of settings, improving generalization to environments which may not be well covered by the simulator in any one particular setting chosen a priori.
Baseline for the Set Transformer which uses ground-truth infectiousness levels as “predictions”. Provides an expected upper-bound performance for the predictors and can be used to provide risk level inputs when pre-training a predictors.
Multi-Layer Perceptron (MLP)
A risk prediction model which concatenates all inputs (after heuristically aggregating to a fixed size vector the set-valued inputs), feeds these through several fully-connected neural network layers and is trained to predict infectiousness. See Section ML architectures and baseline details:.
Set Transformer (ST) Model
A risk prediction model which uses a modified set transformer to process and attend to inputs, and is trained to predict infectiousness. See Section ML architectures and baseline details:.
Deep Set (DS) Model
Similar structure as the Set Transformer, but using pooling instead of self-attention. This allows it to use less memory and compute as compared to the Set Transformer.
Heuristic (HCT) Model
A rule-based method for predicting risk histories and corresponding current behavior levels developed in Gupta et al. (2020).
The proportion of the total population which uses a digital contact tracing application.
The technique of varying the parameters of a simulation environment to create a broad distribution of training data.
The process of matching anonymous data with publicly available data so as to determine which person is the owner.
Big brother attacks
“Big brother” is a reference to the 1984 book by George Orwell where a highly regulated government surveilled and controlled its citizenry. The term “big brother attacks” references the idea that an organized group (e.g., state, federal, academic or financial institution) may try to gain control of your data.
Little brother attacks
Little brother attacks references the idea that potential security threats are also posed by smaller actors like individuals, criminals, and data brokers.
Extra-judicial violence by an individual. In the context of this paper, specifically as a result of a recommendation shown to the user or gained through other means.
Degree of Restriction
Ratio between the number of people given a recommendation in the highest level of restriction (i.e. quarantine) relative to other categories.
Global mobility scaling factor
A simulator parameter enabling us to vary the amount of contacts which may lead to infection in the simulator. It allows us to scale the mobility to simulate pre-lockdown or post-lockdown environments.
Auto-induced distribution shift
A change in distribution of data observed by an agent or algorithm as a result of the agent or algorithm’s actions.
The process of generating data using some method in a simulator, training a model on this data in a supervised manner, then evaluating the model in the simulator to produce more data.
The Pareto frontier is a way of evaluating a tradeoffs in a set of policies and environments with multi-dimensional outputs (e.g., viral spread and mobility).
Appendix 2: Motivating Example
Appendix 3: Comparison of approach to related work
Epidemiological Modeling and Simulations. Given that modeling contact-tracing requires capturing past interactions, it is mathematically complicated to consider the dynamic nature of network interactions. An agent-based model (simulator), on the other hand, gives us maximum flexibility to incorporate real data and/or assumptions easily and emulate the effect of personalized policies (such as resulting from the proposed app). Models with binary contagion and random-walk mobility are ubiquitous (Stevens, 2020). The appeal of such models is their simplicity; they are easy to code, fast to run, and can give a general picture of some aspects of disease spread. There are other models with increasing complexity of either the mobility model, contagion model, agent demographics, or some combination of these, e.g. (Verity et al., 2020; Vespignani et al., 2020; Wood et al., 2020; Lorch et al., 2020; Hinch et al., 2020).
GLEAM (Global Epidemic And Mobility model) (Vespignani et al., 2020) is an off-the-shelf simulation platform for epidemics which offers mobility patterns and demographic information, and uses a generic format for defining how a disease depends on these two things. Similarly, FRED (Framework for Reconstructing Epidemiological Dynamics) provides an open-source, agent-based model with realistic social networks and US demographics. JJ et al. (2013), and (Wood et al., 2020) build an intervention-planning tool on top of this simulator. These works are comparable to our simulator only; they do not do any contact tracing or individual risk prediction.
The work presented in (Lorch et al., 2020) and (Ferretti et al., 2020) is closely related to ours. A detailed mobility model for interactions is presented in (Lorch et al., 2020), but the epidemiological model of the disease is much simpler there. First, the contact graph is built based on their mobility model. The next step is an implementation of various policy interventions by health authorities, which includes contact tracing. This is done instead of building a contact graph that takes into account policy interventions and contact tracing apps at an individual level. Next, most similarly to our work, (Ferretti et al., 2020) proposes an app-based tracing and recommendations. However, their epidemiological model is a very simple differential-equation model, and interventions for controlling the disease are computed analytically.
Risk estimation approaches. While the applications deployed so far are based primarily on binary contact tracing as discussed above, some probabilistic risk estimation approaches, similar to ours, have been developed for other diseases or applications, and have begun to be applied to COVID-19. For example, (Baker et al., 2020) uses the susceptible-infected-recovered (SIR) model and describes the dynamical process of infection propagation using the dynamical message passing equations from (Lokhov et al., 2014); the probability of each node (person) to be in a specific state (S,I or R) is estimated via the dynamic message-passing (DMP) algorithm, which belongs to the family of local message-passing methods similar to belief propagation (BP) algorithm (Yedidia et al., 2000) for estimating marginal probability distributions over the network nodes; despite BP being only an approximate inference method, not guaranteed to converge to the correct marginals when the underlying graphical model has (undirected) cycles, it demonstrated remarkable performance in various applications. A similar approach based on BP was also discussed in (Murphy, 2020). Furthermore, there are recent extensions of belief propagation approach graph neural nets (Satorras & Welling, 2020). Also, another recent work uses Gibbs sampling and SEIR model for their test-trace-isolate approach (Herbrich et al., 2020). However, such approaches typically rely on the knowledge of the social interaction graph, which is not available in our case due to privacy and security constraints.
To the best of our knowledge, our work is the first to use an approach based on detailed agent-based epidemiological model together with a model of phone app messaging to generate simulated data for training an ML-based predictor of individual-level risk.
Digital contact tracing for COVID-19 (Hinch et al., 2020; Hellewell et al., 2020; Aleta et al., 2020; Grantz et al., 2020) study, either via simulations or mathematical models, the conditions under which BCT can be effective. Toward addressing the issues with BCT, (Hinch et al., 2020) show in simulation that using self-reported symptoms in addition to test results can greatly help control an outbreak. Probabilistic (non-binary) approaches to the problem of contact tracing (e.g. Baker et al. (2020); Satorras & Welling (2020); Briers et al. (2020)) typically assume full access to location histories and contact graph, an unacceptable violation of privacy in most places in the world. As a result, these methods are most often used for predicting overall patterns of disease spread.
Agent-Based Models as a generative process Our use of an agent-based simulator (Gupta et al., 2020) as a generative model allows us to generate fine-grained (continuous) values for expected infectiousness with realistic contact patterns. Most works performing probabilistic inference for disease modeling use a simple differential equation generative model, which does not characterize individual behaviour but rather the dynamics of transition between each of several mutually exclusive disease states. Such models make many simplifying assumptions, such as contact patterns based on random walks, which make them unsuitable for individual-level prediction of infectiousness; they are typically used instead to infer latent variables such as infectiousness that would be consistent with the population-level statistics generated by the differential equation model, and/or to predict statistics of spread of the disease in a population, e.g. (ref, ref). While agent-based models are widely used in epidemiological literature to model the spread of disease (see e.g. (ref) for review), to our knowledge we are the first to use an ABM as a generative model for training a deep learning-based infectiousness predictor.
Distributed inference and belief propagation Belief propagation in graphical models is often used for disease spread modeling, e.g. Fan et al. (2016). Some recent works have applied this to COVID-19; for example, Baker et al. (2020) use the susceptible-infected-recovered (SIR) model and describe the process of infection propagation using the dynamical message passing equations from Lokhov et al. (2014). A work concurrent with ours follows a similar justification for modeling a latent parameter of expected infectiousness, using an SEIR model with inference via (Herbrich et al., 2020). However, these approaches rely on a centralized social graph or a large number of bits exchanged between nodes, which is challenging both in terms of privacy and bandwidth. This challenge motivated our particular form of distributed inference where we pretrain the predictor and do not assume that the messages exchanged are probability distributions, but instead just informative input to the node-level predictor.
Appendix 4: Experimental details
ML architectures and baseline details:
Binary contact tracing
quarantines app-users who had high-risk encounters with an app-user who receives a positive PCR test. Under BCT1, if Alice gets a positive test result, then every user who encountered Alice within 14 days of her receiving the positive test result is sent a message which places them in app-recommendation level 3 (quarantine) for 14 days. Formally, .
Recall that in Section 3 we proposed two parameterizations of the model (DS and ST in Figure 2). In the first proposal, we use a set transformer (ST) to model interactions between the elements in the set . We now describe the precise architecture of the model used.
The model comprises 5 embedding modules, namely: the health status embedding , health profile embedding , day offset embedding , risk message embedding and an embedding of the number of repeated encounters.
The model was trained for 160 epochs on a domain randomized dataset (see below) comprising samples. We used a batch-size of 1024, resulting in training steps. The learning rate schedule is such that the first steps are used for linear learning-rate warmup, wherein the learning rate is linearly increased from to , followed by a cosine annealing schedule that decays the learning rate from to in steps.
Inspired by research in hyper-parameter search (Bergstra & Bengio, 2012) and recent advances in deep reinforcement learning (Tobin et al., 2017) we created the transformer’s training data by sampling uniformly in the following ranges:
Initial proportion of exposed people
Oracle additive noise
Oracle multiplicative noise
Global mobility scaling factor
Symptoms dropout: likelihood of not reporting some symptoms
Symptoms drop-in: Likelihood of falsely reporting symptoms
Quarantine dropou (test)t : likelihood of not quarantining when recommended to quarantine due to a positive test
Quarantine dropout (household) : likelihood of not quarantining when recommended to quarantine because a household member got a positive test
All-levels dropout : likelihood of not following app-recommended behavior and instead exhibit pre-pandemic behavior
COVIsim Gupta et al. (2020) models app adoption proportional to smartphone usage statistics.
|% of population with app||Uptake required to get that %|
Our ML experiments use approximately 250 days training time on GPUs while simulations required approximately 41 days of CPU time. All CPU time was run on compute using renewable resources.
- *,, Equal contributions, alphabetically sorted; aMila, Québec; bUniversity of Oxford; cMax-Planck Institute for Intelligent Systems Tübingen; dUniversity of Ottawa; e McGill University; fUniversité de Montreal; gÉcole Polytechnique de Montreal; hThe Alan Turing Institute; iHEC Montréal
- All bolded terms are defined in the Glossary; Appendix 1.
- Open source code available at https://github.com/mila-iqia/COVI-ML
- The recommendations for each level are those proposed by Gupta et al. (2020), hand-tuned by behavioural experts to lead to a reduction in the number of contacts. Full compliance with recommendations is not assumed.
- Note that the the number of cases is highly non-stationary; it grows exponentially over time, even where , which is in the exponent, is constant.
- While we did experiment with additional methods like linear regression and MLPs, they did not improve the performance over the rule-based FCT heuristic proposed by Gupta et al. (2020).
- To estimate this number, we use the GP regression fit in figure 3 and consider the value for which the mean of the NT process is at . We find an estimate at , and select only the runs yielding effective number of contacts that lie within the interval .
- Note that the baseline no tracing method also has false quarantines, because household members of an infected individual are also recommended quarantine, irrespective of whether they are infected.
- Matthew Abueg, Robert Hinch, Neo Wu, Luyang Liu, William J M Probert, Austin Wu, Paul Eastham, Yusef Shafi, Matt Rosencrantz, Michael Dikovsky, Zhao Cheng, Anel Nurtay, Lucie Abeler-Dörner, David G Bonsall, Michael V McConnell, Shawn O’Banion, and Christophe Fraser. Modeling the combined effect of digital exposure notification and non-pharmaceutical interventions on the covid-19 epidemic in washington state. medRxiv, 2020. doi: 10.1101/2020.08.29.20184135. URL https://www.medrxiv.org/content/early/2020/09/02/2020.08.29.20184135.
- Alberto Aleta, David Martin-Corral, Ana Pastore y Piontti, Marco Ajelli, Maria Litvinova, Matteo Chinazzi, Natalie E Dean, M Elizabeth Halloran, Ira M Longini Jr, Stefano Merler, et al. Modeling the impact of social distancing, testing, contact tracing and household quarantine on second-wave scenarios of the covid-19 epidemic. medRxiv, 2020.
- Hannah Alsdurf, Edmond Belliveau, Yoshua Bengio, Tristan Deleu, Prateek Gupta, Daphne Ippolito, Richard Janda, Max Jarvie, Tyler Kolody, Sekoul Krastev, Tegan Maharaj, Robert Obryk, Dan Pilat, Valerie Pisano, Benjamin Prud’homme, Meng Qu, Nasim Rahaman, Irina Rish, Jean-Francois Rousseau, Abhinav Sharma, Brooke Struck, Jian Tang, Martin Weiss, and Yun William Yu. Covi white paper. arXiv preprint arXiv:2005.08502, 2020.
- Antoine Baker, Indaco Biazzo, Alfredo Braunstein, Giovanni Catania, Luca Dall’Asta, Alessandro Ingrosso, Florent Krzakala, Fabio Mazza, Marc Mézard, Anna Paola Muntoni, Maria Refinetti, Stefano Sarao Mannelli, and Lenka Zdeborová. Epidemic mitigation by statistical inference from contact tracing data, 2020.
- Yoshua Bengio, Richard Janda, Yun William Yu, Daphne Ippolito, Max Jarvie, Dan Pilat, Brooke Struck, Sekoul Krastev, and Abhinav Sharma. The need for privacy with public digital contact tracing during the covid-19 pandemic. The Lancet Digital Health, 2020.
- James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(10):281–305, 2012. URL http://jmlr.org/papers/v13/bergstra12a.html.
- Mark Briers, Marcos Charalambides, and Chris Holmes. Risk scoring calculation for the current nhsx contact tracing app. arXiv preprint arXiv:2005.11057, 2020.
- Marc Brisson and et al. Épidémiologie et modélisation de l’évolution de la covid-19 au québec. INSPQ, 2020. URL https://www.inspq.qc.ca/covid-19/.
- Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=S18Su—CW.
- Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, and Dieter Fox. Closing the sim-to-real loop: Adapting simulation randomization with real world experience, 2018.
- Khaled El Emam, Elizabeth Jonker, Luk Arbuckle, and Bradley Malin. A systematic review of re-identification attacks on health data. PloS one, 6(12):e28071, 2011.
- Kai Fan, Chunyuan Li, and Katherine Heller. A unifying variational inference framework for hierarchical graph-coupled hmm with an application to influenza infection. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pp. 3828–3834. AAAI Press, 2016.
- Luca Ferretti, Chris Wymant, Michelle Kendall, Lele Zhao, Anel Nurtay, Lucie Abeler-Dörner, Michael Parker, David Bonsall, and Christophe Fraser. Quantifying sars-cov-2 transmission suggests epidemic control with digital contact tracing. Science, 368(6491), 2020. URL https://doi.org/10.1126/science.abb6936.
- Monica Gandhi, Deborah S. Yokoe, and Diane V. Havlir. Asymptomatic transmission, the achilles’ heel of current strategies to control COVID-19. New England Journal of Medicine, 382(22):2158–2160, 2020. URL https://doi.org/10.1056/NEJMe2009758.
- Kyra H Grantz, Elizabeth C Lee, Lucy D’Agostino McGowan, Kyu Han Lee, C Jessica E Metcalf, Emily S Gurley, and Justin Lessler. Maximizing and evaluating the impact of test-trace-isolate programs. medRxiv, 2020.
- Prateek Gupta, Tegan Maharaj, Martin Weiss, Nasim Rahaman, Hannah Alsdurf, Abhinav Sharma, Nanor Minoyan, Soren Harnois-Leblanc, Victor Schmidt, Pierre-Luc St. Charles, Tristan Deleu, Andrew Williams, Akshay Patel, Meng Qu, Olexa Bilaniuk, Gáetan Marceau Caron, Pierre-Luc Carrier, Satya Ortiz-Gagne, Marc-Andre Rousseau, David Buckeridge, Joumana Ghosn, Yang Zhang, Bernhard Schölkopf, Jian Tang, Irina Rish, Christopher Pal, Joanna Merckx, Eilif B. Muller, and Yoshua Bengio. COVIsim: an agent-based model for evaluating methods of digital contact tracing, 2020.
- Health Canada. Public health management. https://www.canada.ca/en/public-health/services/diseases/2019-novel-coronavirus-infection/health-professionals/interim-guidance-cases-contacts.html, April 2020.
- Joel Hellewell, Sam Abbott, Amy Gimma, Nikos I Bosse, Christopher I Jarvis, Timothy W Russell, James D Munday, Adam J Kucharski, W John Edmunds, Fiona Sun, et al. Feasibility of controlling covid-19 outbreaks by isolation of cases and contacts. The Lancet Global Health, 2020.
- Carl Heneghan, Jon Brassey, and Tom Jefferson. Sars-cov-2 viral load and the severity of COVID-19, 2020. URL https://www.cebm.net/covid-19/sars-cov-2-viral-load-and-the-severity-of-covid-19/. Accessed: 2020-06-02.
- Ralf Herbrich, Rajeev Rastogi, and Roland Vollgraf. Crisp: A probabilistic model for individual-level covid-19 infection risk estimation based on contact data, 2020.
- Robert Hinch, W Probert, A Nurtay, M Kendall, C Wymant, Matthew Hall, and C Fraser. Effective configurations of a digital contact tracing app: A report to nhsx. en. In:(Apr. 2020). Available here. url: https://github. com/BDI-pathogens/covid-19_instant_tracing/blob/master/Report, 2020.
- Grefenstette JJ, Brown ST, Rosenfeld R, Depasse J, Stone NT, Cooley PC, Wheaton WD, Fyshe A, Galloway DD, Sriram A, Guclu H, Abraham T, and Burke DS. Fred (a framework for reconstructing epidemic dynamics): An open-source software system for modeling infectious diseases and control strategies using census-based populations. BMC Public Health, 2013.
- David Krueger, Tegan Maharaj, and Jan Leike. Hidden incentives for auto-induced distributional shift. arXiv preprint arXiv:2009.09153, 2020.
- Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R. Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks, 2018. URL https://arxiv.org/abs/1810.00825.
- Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020.
- Dasheng Li, Dawei Wang, Jianping Dong, Nana Wang, He Huang, Haiwang Xu, and Chen Xia. False-negative results of real-time reverse-transcriptase polymerase chain reaction for severe acute respiratory syndrome coronavirus 2: role of deep-learning-based ct diagnosis and insights from two cases. Korean journal of radiology, 21(4):505–508, 2020.
- Andrey Y Lokhov, Marc Mézard, Hiroki Ohta, and Lenka Zdeborová. Inferring the origin of an epidemic with a dynamic message-passing algorithm. Physical Review E, 90(1), 2014. URL https://doi.org/10.1103/PhysRevE.90.012801.
- Lars Lorch, Heiner Kremer, William Trouleau, Stratis Tsirtsis, Aron Szanto, Bernhard Schölkopf, and Manuel Gomez-Rodriguez. Quantifying the effects of contact tracing, testing, and containment, 2020.
- Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. arXiv preprint arXiv:2003.08934, 2020.
- Thomas P. Minka. Expectation propagation for approximate bayesian inference. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, UAI’01, pp. 362–369, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc. ISBN 1558608001.
- Kevin Murphy. Bayesian contact tracing. "Personal Correspondance.", 2020.
- Kevin P Murphy, Yair Weiss, and Michael I Jordan. Loopy belief propagation for approximate inference: an empirical study. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, pp. 467–475, 1999.
- OpenAI, Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, Jonas Schneider, Szymon Sidor, Josh Tobin, Peter Welinder, Lilian Weng, and Wojciech Zaremba. Learning dexterous in-hand manipulation, 2018.
- Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In International Conference on Machine Learning, pp. 5301–5310. PMLR, 2019.
- A Russell. The rise of coronavirus hate crimes. The New Yorker, 2020.
- Fereshteh Sadeghi and Sergey Levine. Cad2rl: Real single-image flight without a single real image. arXiv preprint arXiv:1611.04201, 2016.
- Victor Garcia Satorras and Max Welling. Neural enhanced belief propagation on factor graphs, 2020.
- Harry Stevens. Why outbreaks like coronavirus spread exponentially, and how to "flatten the curve", 2020. URL https://www.washingtonpost.com/graphics/2020/world/corona-simulator/.
- Latanya Sweeney. K-anonymity: A model for protecting privacy. Int. J. Uncertain. Fuzziness Knowl.-Based Syst., 10(5):557–570, October 2002. ISSN 0218-4885. doi: 10.1142/S0218488502001648. URL https://doi.org/10.1142/S0218488502001648.
- Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world, 2017.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017.
- Robert Verity, Lucy C Okell, Ilaria Dorigatti, Peter Winskill, Charles Whittaker, Natsuko Imai, Gina Cuomo-Dannenburg, Hayley Thompson, Patrick GT Walker, Han Fu, et al. Estimates of the severity of coronavirus disease 2019: a model-based analysis. The Lancet infectious diseases, 2020.
- Alessandro Vespignani, Matteo Chinazzi, Jessica T. Davis, Kunpeng Mu, Ana Pastore y Piontti, Nicole Samay, Xinyue Xiongand Corrado Gioannini, Maria Litvinova, Paolo Milano, Daniela Paolotti, Marco Quaggiotto, Luca Rossi, Michele Tizzoni, and Ivan Vismara. Gleam project. https://www.gleamproject.org/, 2020. Accessed: 2020-06-11.
- John Winn and Christopher M Bishop. Variational message passing. Journal of Machine Learning Research, 6(Apr):661–694, 2005.
- Frank Wood, Andrew Warrington, Saeid Naderiparizi, Christian Weilbach, Vaden Masrani, William Harvey, Adam Scibior, Boyan Beronov, and Ali Nasseri. Planning as inference in epidemiological models. arXiv preprint arXiv:2003.13221, 2020.
- Samuel Woodhams. Covid-19 digital rights tracker - contact tracing apps. https://docs.google.com/spreadsheets/d/1_BCKlMuniEhzvpQ-ha0jhdksvqdINUAUHA8J9LSr_Dc/edit#gid=0, 2020. Accessed: 2020-06-11.
- Jonathan S. Yedidia, William T. Freeman, and Yair Weiss. Generalized belief propagation. In Proceedings of the 13th International Conference on Neural Information Processing Systems, pp. 668–674, 2000. URL https://dl.acm.org/doi/10.5555/3008751.3008848.
- Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. In Advances in neural information processing systems, pp. 3391–3401, 2017.