On MPIs and MDGs

Conan, what is best in life? "It is 1/3 crushing your enemies, 1/3 seeing them driven before you, and 1/3 hearing the lamentation of their women."

Conan, what is best in life? “It is 1/3 crushing your enemies, 1/3 seeing them driven before you, and 1/3 hearing the lamentation of their women.”

Sabine Alkire and Andy Sumner have released a short paper suggesting that the Multidimensional Poverty Index (MPI) be used as a `headline indicator’ for the post-2015 Millennium Development Goals (MDGs). If you’re unfamiliar with the MPI, you can read up on it here. Alkire and Sumner are suggesting that whatever indicators emerge out of the inevitable post-2015 intellectual bloodbath be aggregated into a single index using the same method that is used for the current MPI. This has excited some people, including Duncan Green, who thinks it will be useful in inducing governments to take the post-2015 goals seriously:

That in turn would allow the post2015 process to generate more traction on national governments (the lack of which is the subject of my paper) through league tables. Imagine if every year, all countries (including the rich ones) are ranked on a comprehensive human development table that (unlike the Human Development Index and other similar efforts) has buy in and recognition from across the international community. Each annual report would pick out the countries that have risen/fallen relative to the others. Regional tables could compare India and Bangladesh, or Peru and Bolivia, to generate extra public interest and pressure on decision makers.

I’ll go out and say it: I think this is a really bad idea. It combines the two things that make  two things that make me uncomfortable about both the MPI and the MDGs – arbitrary weights on different indicators/goals and an inflexibility to local preferences.

I’ll use a very basic example: let’s say that the next set of MDGs focuses on two things: hunger and access to clean water. After what will bound to be a seriously convoluted process, someone will agree on internationally-agreed weights on these two things. Let’s say the weights are fifty-fifty, that the final index puts just as much weight on a person who is hungry as one who does not have access to clean water.

Now consider a fictional country, Bigmacistan, which has a culture that sees hunger as being the ultimate state of poverty, much more than clean water. If Bigmacistan were allowed to assign its own weights, it would prefer 3/4 of the total weight to go to hunger and 1/4 to clean water. In fact, given limited resources, Bigmacistan will choose to combat poverty in a way that is not only seen as sub-optimal by the post-MDG framework, but would result in a fall in its global rankings, even if every single person in Bigmacistan is in agreement with its national emphasis on hunger. So differences in MPI 2.0 rankings not only reflect aggregate differences in each country’s success in fighting poverty, but differences in the structure of national social welfare functions.

What one could do is let countries set their own weights (I’ve argued that this is the only way the MPI could even be useful for governments in the long run), but this would never appease the technocrats, because once weights start varying across countries, country rankings start making even less sense.

One could argue that, if there are some indicators that we can reach a reasonably broad consensus on, then imposing these preferences on other countries might be defensible. Unfortunately, this still doesn’t adequately justify the use of the MPI, especially if they are used for annual rankings. Imagine the Bigmacistan actually cares as much about clean water as it does about hunger, but realises that, given its own complex context, it needs to deal with its hunger problem before it will have the capacity to deal with its water access problem. It draws up a national plan which ends hunger by 2020 and then improves access to water by 2025. Yet, from 2015 onwards, Bigmacistan is hounded by donors, NGOs and the media for its poor performance on the MPI 2.0 due to its lack of concern for those living without water.

Finally, any time we want to say anything interesting about the MPI 2.0, we’ll still have to unpack it into its composite indicators, a point Claire Melamed makes on Duncan’s blog:

Say the MPI 2.0, or whatever you called it, went up, or down, in a given country. You’d need an extra layer of data analysis – always fatal as that’s the point you lose people’s attention – to know why. It could be that health outcomes got a lot better, but education outcomes got a bit worse, and so the overall MPI score went up a bit. This would neither be helpful for policy makers, nor tell you much about what people think is important, and it would all be much too complicated to generate any campaigning or political energy anyway.

I do think MPI has its uses, but could we please avoid creating another worldwide indicator that doesn’t tell us very much and imposes what will ultimately be imposing fairly arbitrary weights on individual countries?

4 thoughts on “On MPIs and MDGs

  1. Duncan

    March 8, 2013 at 4:01pm

    Thanks Matt
    Wonks want dashboards and complexity/accuracy; campaigners and journos want single indicators and simplicity/communicability. Discuss. On Claire’s point, exactly the same criticism applies to GDP (first say ‘it’s going up/down, then unpack’). Seems like quite a productive exercise actually – attract people’s attention, then get them to think a bit harder.

  2. David Jinkins

    March 11, 2013 at 2:07pm

    The indicator is worse than just uninformative. In order to look good in the eyes of wonks, governments will misdirect resources from where they are actually needed to where they will increase the indicator.

    I am reminded of the EU’s erstwhile requirement that Turkey reduce the share of Agriculture in GDP before it could be allowed into the EU. The Turkish government started hatching plans to reduce the share of agriculture in ways that made many Turks worse off.

  3. Sabina

    March 12, 2013 at 7:27am

    Hi Matt,
    Great to hear your views. Two things to add to the mix:

    1 – on national indicators and preferences – absolutely critical – and happening!

    The UNDP MPI is a global measure that compares countries using the same indicators and weights – which is actually quite useful as a starting point.

    Alongside, governments are designing national and state MPIs whose indicators mirror their values. For example Colombia has an official MPI with 5 dimensions and 15 indicators reflecting their national plan. The state of Minas Gerais in Brazil has a state-level MPI that is embedded in their integrated social policy delivery programmes. Mexico has had an official multidimensional poverty index since 2009, with income plus 6 social rights. Others are under development. Just like governments mainly use national poverty lines, although the MDGs focus on $1.25/day poverty, so too national MPIs will prove critical for policy, even if MPI 2.0 is useful for basic comparisons.

    2 – on weights. Perhaps you want measures that give basically the same policy advice if people hold a range of weights. That way you can put yours in, and another person can use hers, and the policies will be similar. Good news: it’s possible to do that. See http://www.ophi.org.uk/ophi-brief-on-mpi-robustness/

    I think we need both: a dashboard, and an MPI 2.0 headline indicator, that monitors some post-2015 MDGs, and shows who is suffering most deprivations at the same time. And unlike GDP, the MPI can be unfolded really easily, to see what it’s made of.

  4. Abhijeet

    March 12, 2013 at 11:28am

    Hi Matt,

    Thanks for a great post (and the previous ones on land deals and governance). I share your concerns about the MPI, although Sabina’s comment is very reassuring about national targets. i have a couple more of my own:

    1. Incentives to governments: Different indicators going into the MPI have different elasticities with respect to policy – e.g. shifting malnutrition in South Asia is a lot harder than providing electricity. The MPI implicitly encourages the governments to go for the easier indicators to budge, not necessarily the one with the greatest social and economic benefits. To be fair, this is an issue with all poverty measures (headcount ration encourages governments to focus on the marginally poor, not the poorest) but the MPI is the only one which allows policy-makers to game the dimensions they choose to intervene in and claim they’ve made life *multidimensionally* better!

    2. Much is missing: A key issue with the MDGs is that much is missing – an obvious example is the quality of education (as separate from enrolment) – and I don’t think the MPI changes that much (not OPHI’s fault, more a data availability issue). More to the point, adding an indicator to a dashboard is easy, adding it to a poverty measure makes the measure incomparable over time. If MPI is the way post-2015 MDGs get formed, we will be with the exact dimensions and the weights as now for a very long time….

    3. Robustness? Sabina posted a link to the robustness analysis with the MPI. Kudos to OPHI for having considered this issue carefully and transparently. However, it doesn’t answer my key worry – are *changes* to the MPI stable to the use of other arbitrary weights? Put differently, we know that the cross-sectional rank of countries looks pretty stable with different weights but does their progress over time also look stable? This is central to a discussion about whether the MPI will be a good addition to the post-2015 targets: the MDGs are much more about movement towards a target than the level of target (or the rank of countries on the levels). [BTW, this is something that can be checked easily with available data by OPHI.]

    Cheers,

    Abhijeet

Comments are closed.