The World Health Organization’s (WHO) recent data disclosure on excess COVID-19 deaths in India has caused controversy. Between 2020 and 2021, the WHO estimated that there were 4.7 million excess deaths, directly or indirectly attributable to the pandemic, in India. This figure is an astounding 10-fold higher than the estimates of the government of India and the highest of any country globally, contributing to a harrowing one-third of global deaths.
The Indian Ministry of Health and Family Welfare issued a sharply worded reply, questioning the WHO’s methodology of data collection. The government of India (GOI) had sought to delay the revised WHO report on the COVID-19 death toll prior to its publication due to these outstanding reservations, which mainly relate to the WHO’s mathematical modelling. The GOI argued that since it had published and made publicly available “authentic” national-level data on COVID-19 deaths based on its Civil Registration System (a body that collates subnational data from all Indian states), such modelling and its consequent projections were superfluous and lacked academic rigor.
Jon Wakefied, a member of the WHO’s technical advisory group responsible for the modelling, said that the GOI’s criticism was misplaced since the report’s methodology was consistent for those countries for whom subnational data was available. The WHO’s supplementary report on “methods for estimating the excess mortality associated with the COVID-19 pandemic” supports this claim. India was placed in a grouping with other countries such as Argentina, Indonesia, and Turkey. The range of data considered for India included official figures and the registered number of deaths at the state level as well as data gathered by journalists through “Right to Information” requests, an official public information disclosure mechanism.
Although these discussions have taken on a somewhat partisan shape in the public sphere, the GOI’s reservations on and rebuttal of the WHO’s report should be read as part of a wider set of debates ongoing in the field of research methodology on the uses and limits of statistical modelling, especially in developing and culturally heterogenous geographies with unwieldy demographics.
There is good reason for skepticism about GOI’s statistics, not just because of the obvious electoral effects of higher COVID-19 death tolls on the fortunes of various political outfits at the state level in India, but also due to the limitations of India’s administrative machinery and data gathering apparatus. But if the purpose of such WHO reports is to improve upon national level deficiencies, and help us better understand the “health burden attributable to COVID-19” to affect appropriate changes in health policy, it is equally important to consider if the WHO’s methods proved useful in this respect.
A valuable intervention in this respect was made by journalist Shekhar Gupta of The Print. Even if one considers the limitations of India’s data gathering efforts and the worst possible cases of undercounting, it beggars belief to suggest the GOI missed out on reporting 92 percent of COVID-19 related deaths in India. In this respect, as Gupta rightly argues, the WHO’s findings seem to run afoul of the test of reasonableness, especially considering India’s Civil Registration System has become more robust and efficient with the passage of time and registration percentages for deaths have gone up year-on-year.
Considering a wider range of mortality data in India, Amitabh Sinha similarly wrote for the Indian Express that the WHO report “flies in the face of overall death data, historical trends in death reporting, and Covid-19 death compensation claims from states.” India has a Supreme Court-monitored compensation process for COVID-19 deaths. Even the claims filed for compensation under this process, although more than the official state numbers, are nowhere close to WHO estimates.
One of the GOI’s contentions – that national level statistical modelling is not practically applicable to a country as diverse as India – needed to be taken more seriously by the WHO’s technical advisory group. Neither the test positivity rate and nor the testing rate was consistent across Indian states. Moreover, even data gathering and reporting was not equally efficient, with some states performing better than their peers. However, the WHO’s report substitutes the complexity of data gathering in India with statistics. It places India in a special Tier II category of countries whose official data is deemed untrustworthy for serious consideration, requiring the intervention of mathematical modelling. It is worth highlighting here that the same WHO reports found that China in comparison had the lowest excess COVID-19 deaths in the world, despite a deeper crisis of authenticity with respect to its data regime.
The report on excess COVID-19 deaths only worsens the severe crisis of legitimacy facing the WHO since the beginning of the pandemic. Instead of promoting national data regimes on health in countries like India to be more accountable and transparent, this only creates more opportunity for misinformation and manipulation. If the purpose of the WHO study was to enable better policymaking and coordination, it has singularly failed to achieve this objective in India. In doing so, it has also brought to light the difficulties of statistical modelling, most aptly captured in a quote commonly attributed to British Prime Minister Benjamin Disraeli: “There are three kinds of lies: lies, damned lies, and statistics.”