World + 6 more

Comparability of COVID-19 Epidemiological Data: A case study of six countries

Originally published


Executive Summary


Cross-country comparisons of COVID-19 data are important to understand differences in the burden of COVID-19, determine countries’ relative success containing the virus and guide policies. The comparison of COVID-19 epidemiological data across countries is, however, challenging due to differences in terms of how data are collected and reported.

Research questions The objective of this research was to assess issues associated with comparing national-level COVID-19 epidemiological data in six countries: Bangladesh Burkina Faso, Colombia, DRC, Nigeria and Syria. Specifically, this report sought to address the following questions:

What is the state of the COVID-19 pandemic in Burkina Faso, Colombia, DRC, Nigeria, Bangladesh, and Syria?
How do indicators used to measure COVID-19 testing, cases and mortality differ across the six countries?
What factors may have an impact on the accuracy of COVID-19 indicators and observed differences across countries?

What COVID-19 indicators and information should be reported to increase comparability across countries?
Methodology This research consisted of two data collection methods: secondary data review and semistructured key informant interviews. A review of the grey and peer-reviewed literature was conducted, and key informants were interviewed in Burkina Faso and Nigeria Key findings: This analysis revealed that data collection, measurement and reporting practices for COVID-19 testing, case identification, and mortality vary greatly across the six countries. As a result of data availability and quality issues, COVID-19 measures often either underestimate or overestimate the number of people tested, cases identified and people dying from COVID-19 to varying degrees across countries. Factors that lead to differences in COVID-19 data comparability include: variability in testing strategies including testing availability, eligibility criteria, cost of testing and contact tracing efforts; differing case definitions and use of COVID-19 tests; and overall lack of documentation regarding how indicators are measured. Ambiguous information is particularly prevalent for mortality calculations. Cross-country comparisons are also subject to key differences in reporting practices and contextual factors that may not be documented. When they are not addressed, these country-specific biases and cross-country differences lead to biased comparisons.

Recommendations: Based on the findings, the following set of recommendations are proposed to improve the comparability of data across countries:

To countries:
Publish online and in-country bulletins the definitions of the COVID-19 measures used. Changes in definitions and measurements should also be documented and communicated.
Publish information on processes for data collection and reporting, including reporting sample and processing times. This information should be updated regularly.
Share COVID-19 data updates online using the same time interval and location (e.g., government website). Ideally, updates should be posted daily and any delay in reporting should be documented and explained.
Make COVID-19 datasets readily available online. The dataset should be updated using a defined time interval (e.g., twice a week).
Document and communicate contextual factors that may affect the interpretation of the reported data (strike by laboratory personnel or medical doctors).
Publish multiple COVID-19 data measures. For instance, both the number of tests conducted, and the number of people tested should be reported. Countries with stronger registration systems should aim to publish excess mortality in addition to other mortality measures such as case fatality ratios.
Report the number of tests and number of positive cases separately for travelers versus suspected cases.

Researchers/policy makers

When comparing data across countries, use the same data sources to minimize differences in reporting. Any differences that could affect comparisons should be documented.
International websites should specify the reason they omit data for certain countries and provide information on their data collection practices Local, regional, and international health organizations should stress the importance of COVID-19 data quality, comprehensiveness, reliability, and timeliness and provide support and guidance to strengthen data quality. This is particularly important as these data may be used in the future to inform resource allocation such as vaccine distribution.