The spread of infectious diseases such as COVID-19 presents many challenges to healthcare systems and infrastructures across the world, exacerbating inequalities and leaving the world’s most vulnerable populations most affected. Given their density and available infrastructure, refugee and internally displaced person (IDP) settlements can be particularly susceptible to disease spread. Non-pharmaceutical public health interventions can be used to mitigate transmission, and modeling efforts can provide crucial insights on the potential effectiveness of such interventions to help inform decision making processes. In this paper we present an agent-based modeling approach to simulating the spread of disease in refugee and IDP settlements. The model, based on the JUNE open-source framework, is informed by data on geography, demographics, comorbidities, physical infrastructure and other parameters obtained from real-world observations and previous literature. Furthermore, we present a visual analytics tool which allows decision makers to distill insights by comparing the results of different simulations and scenarios. Through simulating their effects on the epidemiological development of COVID-19, we evaluate several public health interventions ranging from increasing mask wearing compliance to the reopening of learning institutions. The development and testing of this approach focuses on the Cox’s Bazar refugee settlement in Bangladesh, although our model is designed to be generalizable to other informal settings.