Now you're going to hear from my colleague, Dr. Amy Wesolowski about assessing the value of some specific infectious disease transmission models used in the context of an Ebola outbreak in the early days of the COVID-19 pandemic. Hi, I'm Amy Wesolowski, an Assistant Professor in epidemiology at the Johns Hopkins Bloomberg School of Public Health. My research focuses on the use of mathematical models to understand patterns of disease transmission with a focus on vector-borne and vaccine preventable diseases. Today, we're going to go over to different case studies to help illustrate some of the points raised in the course. These vignettes are designed to emphasize some important concepts in the course, as well as show how models have been integrated into public health decision-making. We will focus on two recent outbreaks, the Ebola outbreak in West Africa, from 2014-2016, and the global COVID-19 pandemic. For the first case study on Ebola, we will investigate limitations with predictions during the initial stages of the outbreak and how limited data and model assumptions impacted the accuracy of these models. Then we will discuss how more detailed data was able to help improve the accuracy of these models since they were able to better capture the behaviors relevant for disease transmission. But we will also talk about how there are still some limitations for the use of these models, more generally in the response. First, a brief introduction to the Ebola outbreak that took place in West Africa from 2014-2016. Initial cases were identified in rural areas of Guinea in late 2013, followed by transmission in more urban areas. An outbreak was declared by the World Health Organization, WHO, at the end of March in 2014. This outbreak spread to multiple countries but was primarily focused in West Africa, including Liberia, Guinea, and Sierra Leone, shown in red, with limited outbreaks and isolated cases observed elsewhere, shown in orange and yellow. In total, the outbreak lasted 2.5 Years with a total of 28,000 cases and over 11,000 deaths. It was the largest and most complex Ebola outbreak reported. There were a number of ways that models were used to answer questions and informed decision-making at different stages of the Ebola outbreak, with a reported case counts for the three most affected countries shown above. Early in the outbreak, models were used to predict the number of cases and deaths. Models were also used to estimate what resources may be needed and cost-analysis for these resources. Finally, models were also able to try to estimate what would be the regional and global spread of the outbreak by combining data on travel with models of transmission to estimate importation rates. As the outbreak progressed, models continued to be relevant for many of the same questions, predicting the number of cases and deaths, resources needed, and cost-analysis. However, as data improved and was made available on finer spatial and temporal scales, models were increasingly being used to better estimate patterns of transmission. The effectiveness of different interventions such as isolation treatment and safe burial practices. We will start by focusing on the use of the models in the early stages of the outbreak and how these models were used to predict the number of cases and deaths. One of the first estimates was produced by the CDC and use data at the initial stages of the outbreak to estimate the cumulative number of cases in Liberia and Sierra Leone. This initial model relied on weekly national level data that was made available by the WHO on probable and confirmed reported cases. At this stage, the data was quite limited. However, given the broad public health importance, models were fit to these data, with an example shown in blue on the right. These models assumed that the outbreak would continue to grow exponentially. Based on these assumptions and the data that was available, they estimated a reproductive number of about 1.5-2.5. Overall estimated, there would be between 8,000 and 21,000 cases by the end of September in 2014 in Liberia and Sierra Leone. These models assumed a very large rate of growth resulting in about 1.5 million cases in these two countries by January 2015. Models of early outbreak dynamics are often very sensitive to these assumptions and difficult to produce, since limited data make it difficult for modelers to really understand the epidemiological patterns that are driving transmission dynamics. The basis for this model was a simple, Susceptible, Infected, Incubated, Infectious, and Recovered model shown above. However, given that this was early in the outbreak, a number of parameters and assumptions were made based on historic outbreaks and other setting, which is a standard approach when limited data is available. The model also investigated how different intervention may impact total case counts based on the timing of when these interventions would be put in place. They also provided a tool to use to explore possible outbreak scenarios and help assist in planning resources. This model is really limited by the data that was available and as a result, used estimates such as the incubation period based off previous Ebola outbreaks. At the time, this model fit the data fairly well and there were suggestions of exponential growth in some settings. However, in retrospect, we do realize now that this model do not accurately take into account heterogeneity and transmission patterns that impact the behaviors and other interventions that would be employed and put in place that change the trajectory of the outbreak. During the outbreak, a number of models were used to estimate similar things like resource needs. But given increased availability of finer temporal and spatial scale data, models were able to better estimate the effectiveness of different interventions, and reflect the actual transmission patterns and change observed during the outbreak. In total, there were about 28,000 suspected, probable, and confirmed cases in Liberia, Sierra Leone, and Guinea, which was much lower than the estimate from the CDC model. The reason these models were so much higher was that they did not necessarily take into account local transmission patterns. That were being driven more by isolated outbreaks that burned out after a few generations of disease transmission instead of exponential growth. Increasingly, models became available to allow more accurate forecasts of transmission. An example of comparisons between the forecasted versus observed cases are shown on the right. In the end, these models estimated a much lower value of R naught, but overall, they still tended to overestimate the total number of cases than were observed. In the end, a number of models were developed to answer policy relevant questions during the Ebola outbreak. But they were still always limited by the data that was available. Without detailed data, it's difficult to estimate epidemiologically important parameters. For example, estimates of the incubation period, case fatality ratio, and R-naught can vary widely, making models less useful. While early estimates were useful to help raise awareness on the possible magnitude of the outbreak and understand how different interventions could change the trajectory of case counts. In general, estimates at the beginning of the outbreak are often very difficult given the limited information. Increased surveillance and more detailed data would have helped improve these estimates. However, this information might be difficult to obtain in these settings. Further, this outbreak demonstrated how models need to be continuously updated to better incorporate interventions and behavioral practices such as safe burial practices in the case of Ebola. While many models were produced to understand the Ebola outbreak, there were only some limited uses by public health officials in Ebola affected countries limiting their use. Now, we will go to our second case study on the COVID-19 global pandemic. This vignette will focus on the COVID-19 global pandemic, as well as compare and contrast how different model structures are able to produce different estimates and learn how models may be adapted for different populations to inform public health decision-making. As a brief introduction, SARS CoV-2, Severe Acute Respiratory Syndrome, coronavirus two causes coronavirus disease 2019 or COVID-19. It was originally from a zoonotic reservoir from a bat and primarily transmitted via close contact from respiratory droplets. Starting in late December of 2019, there were cases reported in Wu Han, China of pneumonia of unknown origin. Soon afterwards, there was a lock-down implemented in Wu Han, China, followed by cases reported outside of mainland China. A public health emergency was declared by the WHO in late January 2020, and a number of outbreaks reported in other settings, such as on cruises in South Korea, Italy, Brazil, and the US. By mid-March, many countries were implementing a range of non-pharmaceutical interventions or NPIs, such as restricted gatherings, lock downs, stay-at-home orders, travel restrictions and business and school closures. The majority of these lockdowns remained in place, although this varied by setting. For example, the US started to lift some of these restrictions in May 2020. Given uncertainties in the transmission of SARS CoV-2, global economic impact and overall impact of cases, hospitalizations and deaths, models have been used since the start of the pandemic to answer a number of questions for policymakers. Particularly in these settings where uncertainty is high, but decisions need to be made quickly, models can provide an additional tool to policymakers by systematically integrating the information that is available. Many of these models have been used to predict the total number of cases, hospitalizations and deaths. As well as the impact of non-pharmaceutical interventions, such as business and school closures. There have also been a wide range of models developed to better understand the risk globally and regionally, particularly before many countries had reported their first case of COVID-19. Much of this work focused on flattening the curve, i.e, how different non-pharmaceutical interventions could be put in place to help reduce the demand on health care workers. As the pandemic has progressed, there had been a range of questions and uses of models, including understanding the allocation of vaccines, different testing strategies, and the emergence of variants of concerns. Modeling has been used by local, regional, and global decision-makers, particularly at the start of the pandemic, as they were making decisions with very limited information. In particular, a model and report that was developed by Imperial College was used to emphasize how quickly SARS-CoV-2 may spread and the burden this would put on the health care system if no interventions were put in place. This model estimated a very large outbreak with a high number of deaths that exceeded hospital capacity. An example figure from this model is shown on the right. This report was very influential in raising awareness to the possible impact of the COVID-19 pandemic if no interventions were put in place. While there is no universal model of SARS-CoV-2 transmission, many models include a general susceptible in blue, exposed in orange, infected in red, and recovered in green model. These models often include other factors, such as introduction rates, asymptomatic and symptomatic transmission, as well as projected hospitalizations, ICU needs and deaths. Non-pharmaceutical interventions are often included in these models that effectively reduce the contact rate between an individual's and reduce the value of R naught. Given uncertainties in transmission at the start of the pandemic, the assumptions in these models are incredibly important. For example, the Institute for Health Metrics and Evaluation model was very popular since it produced estimates for many locations in an easy to use format. However, this model relied on statistical methods that resulted in an overly optimistic projection with an example output shown to the right. It is important to evaluate our models with available data and investigate our assumptions to provide more reliable projections. Specific policies may also dictate the type of model you need to construct. For example, to investigate the effectiveness of contact tracing and the isolation of cases, you may need to use an individual-based model. Since we will need to follow exactly who infects who, which may be more difficult from some of the population-based approaches. An example modeled schematic is shown from the London School of Hygiene and Tropical Medicine that provided guidance on how effective contact tracing and case isolation may be. In general, the specific questions or policies of interests may help inform the model structure you need to use. Given the abundance of models that are available and different approaches, it is now possible to produce not only single model results, but ensemble models that combine these outputs from many different models, such as the COVID-19 scenario modeling hub. These models are able to not only forecast, but also investigate different types of scenarios to help inform decision-making. For example, these models may include different combinations of NPIs and compliance to these NPIs to evaluate what might be possible for a range of scenarios on the impact of keys counselor or deaths. Finally, it is important to always make sure our model is able to answer questions relevant to our population of interests. In particular, understanding the local context and how to integrate that into COVID-19 models can help make these models more informative. Some examples of ways models can be adapted to better reflect the transmission dynamics in a particular population, include taking into account the age structure or demography of the population or also taking into account different patterns of transmission, including different time-varying R-naught values. Models may also need to be adapted to include different controls or intervention measures, including the ones that have actually been put in place in a particular location. In general, high-quality data is needed to evaluate these models for a particular location. Models could and should continue to evolve as our understanding of transmission and policy relevant questions also change. The COVID-19 pandemic has demonstrated how models can be used as an additional tool to inform decision-making. However, caution should be taken when interpreting and evaluating models. It is important for models to be adapted for local context, produce robust results that are accurately capturing the behavior is relevant to transmission and be informed by data whenever possible.
