Counting desires

Preference aggregation can be tricky business

A key assumption behind the Global Burden of Disease project is that it is possible to come up with a “Disability Weight” for each health state.  Diseases conditions that are considered worse than other carry higher disability weights than others.  A very important issue in the development of such weights is the question of who should define these conditions?  Should those who have the conditions be the best judge or are they biased?  Should healthy people who have never experienced these conditions be the judge?  Should doctors decide?  Should policy makers?  Should health economists (gasp!!)?

In the past, the GBD has relied upon “expert opinion” to make such decisions.  Well, it seems for the next update of the GBD, which is currently underway, you can also be an expert.  I came across a link to the following survey earlier today that allows you to have some input in these weights.

That’s Karen Grepin discussing an attempt to aggregate beliefs over disease burdens to better define the weights given to different ailments. This is a very similar exercise to preference aggregation, where we attempt to construct a unified set of beliefs that will govern public policy. The result is something approaching a social welfare function, which allows us to make statements like “Society strictly prefers A to B.” One way of doing this is to get a sample of individuals to compare different states and to try and tease out an overall ordinal ranking of these states. Using Grepin’s example, each person has to make a pairwise comparison:

The first person has swelling and tenderness in the testicles and pain during urination.

The second person has lost part of both legs, leaving pain, tingling, and frequent sores in the stumps. The person has great difficulty moving around and has episodes of depression, anxiety and flashbacks to the injury.

By asking enough people to compare different states with different combinations of symptoms, we can tease out their overall ranking of those symptoms – how this is done can sometimes be contentious and quite technical. That ranking then represents the best approximation of everyone’s relative rankings of disease burden.

This isn’t a new method of trying to better calibrate policy to the wishes of many. As the US Army drew closer to victory in the European theatre during WWII, there was some concern over which soldiers would be allowed to return home first. Instead of making an arbitrary decision as to what characteristics led to early demobilisation or imposing a lottery, the Army actually surveyed several thousand active servicemen, asking them to rank soldiers with different circumstances (taken from Peyton Young’s book Equity: in theory and practice):

After the war when the Army starts releasing soldiers back to civilian life, which of each of these two groups of men do you think should be release first?

  1. Men with dependents OR men over 30  years of age
  2. Men who have been in the army longest OR men with dependents?
  3. Men over 30 years of age OR men who have served overseas
  4. Men who have served overseas OR men who have been in the army longest?

Based on how respondents ranked the various ‘states’, the Army was able to aggregate those preferences and derive a point system for which each circumstance (having dependents, being older, etc) was weighted. Servicemen with the most points were given priority for early release. The fairness in the system was inherent in the way is was derived: through aggregation of the beliefs and preferences of those that would have to live with its consequences.

This attempt to re-calibrate the weights placed on disability is similar in its approach, save for a significant flaw: the sampled respondents are self-selecting their way in (anyone can go and contribute here, as I already have). The responses will be primarily from the rich, healthy, internet-using population. The survey seems to ask questions which could be used to weight responses, (the age, location and socioeconomic status of the respondent), but I can’t imagine there will be enough of a response from areas of the world which actually have the higher disease burdens to overcome this.

At least it is a step in the right direction. Imagine a future where the priorities areas for aid were not decided by experts and technocrats at a global level, but instead were the result of careful preference aggregation from the populations who stand to benefit? I long for the day where DFID, before deciding where to spend its money in Zambia, runs a representative survey which asks:

Rank these three outcomes:

20 new public schools OR 5 new hospitals OR 3,000 new jobs

4 thoughts on “Counting desires

  1. Ranil Dissanayake

    July 28, 2010 at 7:21am

    Yes, but the problem is that people don’t always ask for what they need. You’re assuming that people will be far-sighted and rational. I think that’s optimistic. Particularly when they’re asked to rank intangibles, like a ‘a better healthcare system’; ‘a more rational budget process’ compared with ‘twenty new schools’.

    I know it’s fashionable to say ‘we should ask people’ and be all democratic about everything we do. But you can take it too far. People don’t always make the best decisions for themselves. People voted for Bush, for God’s sake. Twice.

  2. Matt

    July 28, 2010 at 9:48am

    Points taken – a few things to note though:

    These exercises are rather far flung from what we might think of democracy. For example, the Global Burden of Disease project isn’t just going to take every submitted result as a “vote”, choosing one to win over others – they are likely going to re-weight them based on which demographics are under-represented. Even then, the methods they use to derive an aggregate ranking from pairwise choices involves some possibly-arbitrary decisions which might be considered “undemocratic.” Democracy has also some frequency, while these sort of projects are usually one-offs.

    Also, these exercises rarely involve one-to-one translations of the resulting ranking into policy – I’d be surprised if Global Burden of Disease project won’t do some tinkering, as I imagine the Army did to renconcile the soldier’s wishes with their own needs for occupation. I wasn’t suggesting that people in developing countries start a Cameronesque practice of voting on aid flows (in Malawi, for example, they would just vote for more fertiliser), but that maybe, just possibly, it might be useful information for donors to have. It doesn’t have to translate directly into policy, and even if it did one might limit it to areas where rationality might be a little more assured.

    Finally, most of the arguments made for these exercises aren’t arguments made over efficiency or welfare, but those made on fairness. In the end, I think it’s reasonable to argue that the first two can trump the third.

    And a final comment on Bush – actually the majority of Americans voted for Gore in the first election – it’s because of less direct democratic institutions like the electoral college and the Supreme Court that he was elected.

  3. Ranil Dissanayake

    July 28, 2010 at 10:14am

    all fair points, particularly that on Gore! I’d forgotten he won the popular vote in 2000. I always remember the conversation I had that day.

    ‘So it all comes down to Florida? Great, the Governor is bound to order a re-count, with some clarity on how the ‘dimpled’ returns are counted.’
    ‘The Governor is a republican.’
    ‘Hmm. But maybe he’s one of the mavericks.’
    ‘His name is Jeb Bush.’
    ‘D’oh!’

  4. Roving Bandit

    July 30, 2010 at 9:13am

    I long for the day when DFID gives Zambians cash, the government of Zambia taxes its citizens, and then the government of Zambia before deciding where to spend its money in Zambia, runs a representative survey which asks:

    Rank these three outcomes:
    20 new public schools OR 5 new hospitals OR 3,000 new jobs

Comments are closed.