Pyfectious: An individual-level simulator to discover optimal containment policies for epidemic diseases

Author summaryPyfectious is an agent-based simulator with the capability to serve as an environment for reinforcement learning agents to discover novel control high-resolution agent-based policies that are hard for humans to discover. Pyfectious introduces several novelties which are unprecedented in the existing popular simulators in epidemiology. It constructs the population structure of a city without needing too detailed information by a novel probabilistic assignment method that is unparalleled to existing population synthesizers. The proposed disease propagation algorithm offers a multi-resolution functionality that allows running Pyfectious for large-population cities on normal computers. The modeling details can be easily traded-off with computational demand requiring minimal effort by the end user. The control and monitoring components are designed in an event-triggered fully flexible way by providing a rich action space from which effective policies are hoped to be discovered by advanced RL methods which are otherwise impossible for humans to find due to the immense complexity of the problem. An extensive set of experiments are included to illustrate various aspects of Pyfectious and also to briefly showcase its use as an RL environment which is hoped to help the automatic discovery of epidemic control policies upon bringing together RL scientists and epidemiologists. Simulating the spread of infectious diseases in human communities is critical for predicting the trajectory of an epidemic and verifying various policies to control the devastating impacts of the outbreak. Many existing simulators are based on compartment models that divide people into a few subsets and simulate the dynamics among those subsets using hypothesized differential equations. However, these models lack the requisite granularity to study the effect of intelligent policies that influence every individual in a particular way. In this work, we introduce a simulator software capable of modeling a population structure and controlling the disease's propagation at an individualistic level. In order to estimate the confidence of the conclusions drawn from the simulator, we employ a comprehensive probabilistic approach where the entire population is constructed as a hierarchical random variable. This approach makes the inferred conclusions more robust against sampling artifacts and gives confidence bounds for decisions based on the simulation results. To showcase potential applications, the simulator parameters are set based on the formal statistics of the COVID-19 pandemic, and the outcome of a wide range of control measures is investigated. Furthermore, the simulator is used as the environment of a reinforcement learning problem to find the optimal policies to control the pandemic. The obtained experimental results indicate the simulator's adaptability and capacity in making sound predictions and a successful policy derivation example based on real-world data. As an exemplary application, our results show that the proposed policy discovery method can lead to control measures that produce significantly fewer infected individuals in the population and protect the health system against saturation.