Why rank countries by health performance?

Abstract

In 1978, from a little-known region of what was then the USSR, emerged a WHO/UNICEF statement of intent with the slogan “Health for all by the year 2000”. That year has passed, leaving the Alma-Ata declaration largely unfulfilled. Indeed in some parts of the world the situation has worsened, and not just because of AIDS and civil unrest. Yet the failure of Alma-Ata is often viewed positively: the declaration was never meant to be taken literally as a target that everyone would be healthy by last year, and it is argued, reasonably, that the slogan has kept the issue of primary care to the forefront of the debate in WHO and other United Nations agencies. But this is a card—labelling a failure a success because the matter was worth raising—that must be played sparingly. As this week's Lancet shows (pp 1671, 1685), The world health report 2000, published a year ago, continues to attract critical attention. Does it matter that the criticisms are serious provided the underlying objective, which is the use of national performance indices to improve health in all countries, is worthy, as it clearly is? If WHO is to become a science-led global policy body, the answer has to be Yes.WHO's director-general, Dr Gro Harlem Brundtland, conceded that “many of the concepts and measures used in the report require refinement and development” – mild words to set beside some recent comments. While avoiding the alliterative “fatally flawed”, this week's critics come close to it. WHO has listened, at least to the extent of announcing in January of this year a review of the methods used. Earlier this month, the Pan American Health Organization held a regional consultation, which included a commentary from R Paul Shaw, an economist with the World Bank Institute, Washington, DC. Focusing on the financial fairness index, one of several in The world health report 2000, he neatly turned upside-down the intuitively absurd result that a country such as Colombia could be fairer than Canada. In asking if this index's rankings “make sense in terms of what we know”, he exposes flaws in the report's use of statistical modelling. The methodological criticism about the report's index of inequality (p 1671) is of the same sort.No methodology will ever be transparent enough for some critics, and the team at the WHO Global Programme on Evidence for Health Policy may well feel aggrieved at the criticisms the report has been facing. They need not. What went wrong with this exercise—and there is no denying that the criticism is a blow for WHO—is that the organisation forgot that this is a research project. The authors rightly stressed that the data collection, analytical methods, and summary indicators were “new”. Sensibly, they wrote a more or less readable report, leaving experts to search in (and occasionally for) other documents with more technical material. Equally wisely, they placed the rankings of 191 countries in appendices. Nonetheless it was the apparent absurdity of some of those rankings that attracted media attention, and the multitude of indices used did not help. Has WHO learned nothing from the experience of others using a football league approach to hospital statistics, school performance, and the like? As Vicente Navarro reminds us, Spain's quality of life fell unbelievably from rank 7th to 21st in a single year in a similar exercise.WHO's director-general has promised that “measurement of health systems performance will be a regular feature of all World health reportsfrom now on”. That is a sensible objective but on the timing Dr Brundtland should think again. There first needs to be better consensus, among policymakers and academics alike, about the reliability of the methods and information sources. Then should come agreement on just one or two indices (eg, disability-adjusted life expectancy) rather than several. At that time it might be acceptable to reintroduce the rankings. In 1978, from a little-known region of what was then the USSR, emerged a WHO/UNICEF statement of intent with the slogan “Health for all by the year 2000”. That year has passed, leaving the Alma-Ata declaration largely unfulfilled. Indeed in some parts of the world the situation has worsened, and not just because of AIDS and civil unrest. Yet the failure of Alma-Ata is often viewed positively: the declaration was never meant to be taken literally as a target that everyone would be healthy by last year, and it is argued, reasonably, that the slogan has kept the issue of primary care to the forefront of the debate in WHO and other United Nations agencies. But this is a card—labelling a failure a success because the matter was worth raising—that must be played sparingly. As this week's Lancet shows (pp 1671, 1685), The world health report 2000, published a year ago, continues to attract critical attention. Does it matter that the criticisms are serious provided the underlying objective, which is the use of national performance indices to improve health in all countries, is worthy, as it clearly is? If WHO is to become a science-led global policy body, the answer has to be Yes. WHO's director-general, Dr Gro Harlem Brundtland, conceded that “many of the concepts and measures used in the report require refinement and development” – mild words to set beside some recent comments. While avoiding the alliterative “fatally flawed”, this week's critics come close to it. WHO has listened, at least to the extent of announcing in January of this year a review of the methods used. Earlier this month, the Pan American Health Organization held a regional consultation, which included a commentary from R Paul Shaw, an economist with the World Bank Institute, Washington, DC. Focusing on the financial fairness index, one of several in The world health report 2000, he neatly turned upside-down the intuitively absurd result that a country such as Colombia could be fairer than Canada. In asking if this index's rankings “make sense in terms of what we know”, he exposes flaws in the report's use of statistical modelling. The methodological criticism about the report's index of inequality (p 1671) is of the same sort. No methodology will ever be transparent enough for some critics, and the team at the WHO Global Programme on Evidence for Health Policy may well feel aggrieved at the criticisms the report has been facing. They need not. What went wrong with this exercise—and there is no denying that the criticism is a blow for WHO—is that the organisation forgot that this is a research project. The authors rightly stressed that the data collection, analytical methods, and summary indicators were “new”. Sensibly, they wrote a more or less readable report, leaving experts to search in (and occasionally for) other documents with more technical material. Equally wisely, they placed the rankings of 191 countries in appendices. Nonetheless it was the apparent absurdity of some of those rankings that attracted media attention, and the multitude of indices used did not help. Has WHO learned nothing from the experience of others using a football league approach to hospital statistics, school performance, and the like? As Vicente Navarro reminds us, Spain's quality of life fell unbelievably from rank 7th to 21st in a single year in a similar exercise. WHO's director-general has promised that “measurement of health systems performance will be a regular feature of all World health reportsfrom now on”. That is a sensible objective but on the timing Dr Brundtland should think again. There first needs to be better consensus, among policymakers and academics alike, about the reliability of the methods and information sources. Then should come agreement on just one or two indices (eg, disability-adjusted life expectancy) rather than several. At that time it might be acceptable to reintroduce the rankings. World Health Report 2000: inequality index and socioeconomic inequalities in mortalityMonitoring of inequality in health has become an increasingly important task of development agencies. We compared the inequality index as published in the World Health Report 2000 with available evidence on socioeconomic inequalities in mortality in 15 industrialised and 43 less-developed countries. We found that the World Health Report index does not correspond with international variations in the size of socioeconomic inequalities in mortality. These findings indicate that the index should not be interpreted as a reflection of socioeconomic inequalities in health, nor should the index be used to replace the indices developed to monitor socioeconomic inequalities in health. Full-Text PDF Healthy life expectancy in 191 countries, 1999We describe here the methods used to produce the first estimates of healthy life expectancy (DALE) for 191 countries in 1999. These were based on estimates of the incidence, prevalence, and disability distributions for 109 disease and injury causes by age group, sex, and region of the world, and an analysis of 60 representative health surveys across the world. We used Sullivan's method to compute healthy life expectancy for men and women in each WHO member country. Japan had the highest average healthy life expectancy of 74·5 years at birth in 1999. Full-Text PDF