On MPIs and MDGs

Conan, what is best in life? "It is 1/3 crushing your enemies, 1/3 seeing them driven before you, and 1/3 hearing the lamentation of their women."

Conan, what is best in life? “It is 1/3 crushing your enemies, 1/3 seeing them driven before you, and 1/3 hearing the lamentation of their women.”

Sabine Alkire and Andy Sumner have released a short paper suggesting that the Multidimensional Poverty Index (MPI) be used as a `headline indicator’ for the post-2015 Millennium Development Goals (MDGs). If you’re unfamiliar with the MPI, you can read up on it here. Alkire and Sumner are suggesting that whatever indicators emerge out of the inevitable post-2015 intellectual bloodbath be aggregated into a single index using the same method that is used for the current MPI. This has excited some people, including Duncan Green, who thinks it will be useful in inducing governments to take the post-2015 goals seriously:

That in turn would allow the post2015 process to generate more traction on national governments (the lack of which is the subject of my paper) through league tables. Imagine if every year, all countries (including the rich ones) are ranked on a comprehensive human development table that (unlike the Human Development Index and other similar efforts) has buy in and recognition from across the international community. Each annual report would pick out the countries that have risen/fallen relative to the others. Regional tables could compare India and Bangladesh, or Peru and Bolivia, to generate extra public interest and pressure on decision makers.

I’ll go out and say it: I think this is a really bad idea. It combines the two things that make  two things that make me uncomfortable about both the MPI and the MDGs – arbitrary weights on different indicators/goals and an inflexibility to local preferences.

I’ll use a very basic example: let’s say that the next set of MDGs focuses on two things: hunger and access to clean water. After what will bound to be a seriously convoluted process, someone will agree on internationally-agreed weights on these two things. Let’s say the weights are fifty-fifty, that the final index puts just as much weight on a person who is hungry as one who does not have access to clean water.

Now consider a fictional country, Bigmacistan, which has a culture that sees hunger as being the ultimate state of poverty, much more than clean water. If Bigmacistan were allowed to assign its own weights, it would prefer 3/4 of the total weight to go to hunger and 1/4 to clean water. In fact, given limited resources, Bigmacistan will choose to combat poverty in a way that is not only seen as sub-optimal by the post-MDG framework, but would result in a fall in its global rankings, even if every single person in Bigmacistan is in agreement with its national emphasis on hunger. So differences in MPI 2.0 rankings not only reflect aggregate differences in each country’s success in fighting poverty, but differences in the structure of national social welfare functions.

What one could do is let countries set their own weights (I’ve argued that this is the only way the MPI could even be useful for governments in the long run), but this would never appease the technocrats, because once weights start varying across countries, country rankings start making even less sense.

One could argue that, if there are some indicators that we can reach a reasonably broad consensus on, then imposing these preferences on other countries might be defensible. Unfortunately, this still doesn’t adequately justify the use of the MPI, especially if they are used for annual rankings. Imagine the Bigmacistan actually cares as much about clean water as it does about hunger, but realises that, given its own complex context, it needs to deal with its hunger problem before it will have the capacity to deal with its water access problem. It draws up a national plan which ends hunger by 2020 and then improves access to water by 2025. Yet, from 2015 onwards, Bigmacistan is hounded by donors, NGOs and the media for its poor performance on the MPI 2.0 due to its lack of concern for those living without water.

Finally, any time we want to say anything interesting about the MPI 2.0, we’ll still have to unpack it into its composite indicators, a point Claire Melamed makes on Duncan’s blog:

Say the MPI 2.0, or whatever you called it, went up, or down, in a given country. You’d need an extra layer of data analysis – always fatal as that’s the point you lose people’s attention – to know why. It could be that health outcomes got a lot better, but education outcomes got a bit worse, and so the overall MPI score went up a bit. This would neither be helpful for policy makers, nor tell you much about what people think is important, and it would all be much too complicated to generate any campaigning or political energy anyway.

I do think MPI has its uses, but could we please avoid creating another worldwide indicator that doesn’t tell us very much and imposes what will ultimately be imposing fairly arbitrary weights on individual countries?

Weighing the dimensions of poverty

You have to know these things when you're the King

Mead Over at the CGD discusses a recent debate between Martin Ravallion, frequent critic of the multidimensional poverty index (MPI), and James Foster, one of its inventors. For those of you not familiar with the MPI, you can read about it here, where I briefly explain and discuss the measure.

Ravallion criticises the MPI for using potentially arbitrary weights to combine several different measures of poverty into single, hard to interpret index. When researchers start assigning weights to create composite indices, they are implicitly making judgements on how we value different measures of poverty. For instance, if our MPI of interest is constructed using 2 x(access to water) + 1 x (asset poverty) (gross simplification), we are implicitly saying we care about access to water twice as much as we do about asset poverty.

This had led Ravallion (and others) to suggest that governments consider each measure separately, rather than compiling them into something that lacks an immediate interpretation:

Ravallion asked the audience to remember their last appointment with a physician for a general physical examination – a “routine checkup.” He asked, “Would you really want the doctor to summarize all the information on all the tests with a single composite index of you health?”

Foster objects, noting that battlefield medics will collapse all these measures into one index (i.e who is closest to death?) in order to establish priority. Mead Over manages to get to the heart of the disagreement: Ravallion’s approach makes no assumption of the MPI-user’s value function (i.e., what they care about most), where Foster’s assumes that someone is going to be imposing some sort of a value/objective function, in which case weighting and merging into an index makes sense.

If we wanted to establish a value function for weighting the MPI, what might it look like? Over suggests using some empirics to help us determine which measures of poverty are to be given higher priority (i.e. a higher weight in the MPI).

Continue reading

Questionable parentage

Gabriel Demombynes over at the World Bank blog has s0me more interesting things to say about the Multidimensional Poverty Index (MPI). There’s one claim he makes a claim which I find particularly interesting:

The MPI is a descendant of the earlier Human Development Index and is similar to the various Unsatisfied Basic Needs indices long used in many countries.

Several others, including Duncan Green, have also stated that the MPI is a natural follow-on from the Human Development Index (HDI), which I’m not sure is correct, as the two have a very different conceptual basis.

As its name implies, the MPI falls into a class of indices known as poverty measures. While they can get quite complex and opaque, the more basic of these have a similar approach: First we have to pick a welfare measure. This could really be anything that is measurable, but is most commonly income, consumption or asset wealth. Then comes the surprisingly contentious task of choosing a threshold, under which people will be classified as being poor if they do not meet it. These poverty lines can be absolute or relative, the latter indicating a greater concern for inequality than absolute deprivation. Counting the poor gives us a final tally of those living below the poverty line.

The MPI is an extension of this approach, instead using a range of indicators wrangled together a multidimensional poverty line. While single-dimension poverty lines make very precise statements about people along one dimension (Person i can only be not-poor if their income Xi exceeds the poverty threshold P (Xi >P), multidimensional lines can classify two households as being poor even when they face vastly different circumstances. For example: two people might be equally unhealthy, but one has enough asset wealth to be classified as “not-poor”. The MPI also tries to include information on the severity of poverty, for those that face many different deprivations all at once, a conceptually similar approach to the poverty gap and squared poverty gap indices.

The MPI, like the other poverty measures that came before it, focuses on a particular segment of the population, discarding all information about the non-poor. Because it is derived by counting individuals whole fall into a pre-specified condition, it is best thought of as a way to describe the state of this sub-population, rather than as a comprehensive indicator.

In contrast, the Human Development Index was intended to be used to make statements about the overall progress of a country’s development. While all of its components are aggregated from individual or household information, or from counting those in a certain condition (i.e. those that are literate, or who have died this year), they do not give the same type of insight. The education component is similar (we are just counting those who are in the state of literacy or who are enrolled in school), but with GNI and life expectancy, we aren’t really counting anything, we’re expressing moments and expectations from interesting country-wide distributions. We cannot say “X number of people have an HDI of Y.”

The HDI was initially introduced as an alternative to just relying on income as a measure of human welfare. This way of looking at the world, which became very popular following Sen’s work on the capabilities approach, also motivates the MPI as an alternative to only considering poverty in income. The weakness in both the indices is in their method with dealing with multidimensionality – by using ad hoc methods of averaging different dimensions together to come up with a single number.

So, when describing the MPI to someone new, one might refer to it as “an extension of traditional income-based poverty measures, taking into account the multidimensional nature of poverty, much as the Human Development Index considers the multidimensional nature of development. Both consider just measuring income, or consumption, to be insufficient,” rather than as a natural follow on from the HDI.

How does the MPI measure up?

Duncan Green introduces us to the new Multi-Dimensional Poverty Index (MPI), developed by the Oxford Poverty and Human Development Initiative (OPHI):

The MPI brings together 10 indicators of health (child mortality and nutrition), education (years of schooling and child enrolment) and standard of living (access to electricity, drinking water, sanitation, flooring, cooking fuel and basic assets like a radio or bicycle). It’s thus a logical extension of its predecessor, UNDP’s pioneering Human Development Index, launched in the first Human Development Report back in 1990, which combined life expectancy, education (literacy + enrolment rates) and GDP per capita.

The measure, like the HDI, is part of an attempt to get a “better measure” of poverty, by including many non-income indicators. While I think most would agree that policymakers and researchers should always consider non-income indicators of welfare, does it make sense to average them out into a single index?

What precisely are we measuring when the HDI for a given country increases by .01? These questions always seem to lead back to the original indicators: “A advanced in rank because of education improvements” or  “B is lower than C despite being richer, because life expectancy in B is much lower.” Given that we need to unpack these indices to figure out what’s going on, why do we bother to pack them in the first place?

Duncan, always open for a healthy debate, has already posted a criticism of the MPI by Martin Ravallion of the World Bank, which questions the implicit values placed on different indicators when they are weighted:

The index is essentially adding up “apples and oranges” without knowing their relative price. When one measures aggregate consumption from household-survey data for the purpose of measuring poverty, as in the World Bank’s “$1 a day” measures, one relies on economic theory, which says that (under certain conditions) market prices provide the correct weights for aggregation. We have no such theory for an index like the MPI. A decision has to be taken, and no consensus exists on how the multiple dimensions should be weighted to form the composite index.

On closer scrutiny, the embedded trade-offs (stemming from the weights chosen by the analyst) can be questioned, and may be unacceptable to many people.  In the context of the HDI, I pointed out 15 years ago that by aggregating GDP per capita with life expectancy the HDI implicitly put a value on an extra year of life, and I showed that this value rises from a very low level in poor countries to a remarkably high level in rich ones (4-5 times GDP per capita).   If it was made clearer to users, I expect that they would question this trade-off embedded in the HDI.

The MPI index faces the same problem. How can one contend (as the MPI does implicitly) that the death of a child is equivalent to having a dirt floor, cooking with wood, and not having a radio, TV, telephone, bike or car?  Or that attaining these material conditions is equivalent to an extra year of schooling (such that someone has at least 5 years) or to not having any malnourished family member?  These are highly questionable value judgments. Sometimes such judgments are needed in policy making at country level, but we would not want to have them buried in some aggregate index.  Rather, they should be brought out explicitly in the specific country and policy context, which will determine what trade off is considered appropriate; any given dimension of poverty will have higher priority in some countries and for some policy problems than others.

One could continue to argue about the weights – but Ravallion’s argument will still stand. I fail to see why these indices amount to anything more than intellectual exercises – while the HDI has got us all thinking about other things than income, has it really been useful as a method of actually measuring development? Is the MPI likely to do any better with poverty?

Oxford Poverty and Human Development Initiative (OPHI)