Aid Thoughts

Digesting the difficult decisions of development

Archive for category Worst practices

In which Malawi gives Madonna a spinning roundhouse kick

Apr 10

Posted by Matt in Africa, Worst practices | 1 Comment

norris_malawi

So Malawi was graced by another visit from Madonna recently. Somewhat miffed that she hadn’t received an invitation to go meet with President Joyce Banda, she wrote an overly-personal message to Banda (“Dear Joyce”) to ask if they could meet. To slightly complicate things further, the head representative of Madonna’s charity went after the President’s sister (who used to work for the Raising Malawi) and complained that the Material Girl wasn’t getting the right treatment from the government:

Madonna can continue her work here [even] if the politicians don’t want to welcome her because her work is all about the children who are here. The politicians can stay. Even donors are also surprised that government is treating Madonna like this when she is the biggest private donor in the country

In response, the Malawian government released an 11-point passive-aggressive smackdown. You can read the whole thing here, but one particular point stood out as being awesome and seriously bad ass:

7. If the argument is that because she is an internationally renowned star, and, therefore, Madonna believes she deserved to be treated differently from other visiting foreigners, it is worth making her aware that Malawi has hosted many international stars, including Chuck Norris, Bono, David James, Rio Ferdinand and Gary Neville who have never demanded state attention or decorum despite their equally dazzling stature. [Emphasis added]

Boom.

Hat tip to Kim Yi Dionne at haba na haba, who has covered both Madonna’s PR gaffs and the government response.

norris_poverty

 

Tags: , , ,

Awkward Dar es Salaam signs, cross-cutting transport infrastructure and global health edition

Apr 8

Posted by Matt in Africa, Worst practices | No Comments

bus_station

On causality and the returns to late marriage

Mar 25

Posted by Matt in Research, Worst practices | 1 Comment

Miss Havisham's vast wealth is undoubtedly due to her failure to marry.

Miss Havisham’s vast wealth is undoubtedly due to her failure to marry.

Over the last week or two there has been a fair bit of chatter about a report released by the University of Virgina’s National Marriage Project. While the results pertain primarily to marriage outcomes in the United States, the general interpretation of those results is a textbook case of making strongly-causal statements using methods which are insufficient for making these claims. These two graphs, documenting the earnings of women and men by age of marriage, provide a starting point for the discussion:

knotyet_income

knotyet_incomemen

 

These descriptive stats paint a pretty clear picture, allowing us to make the following statements:

  • On average, women who have married at a later age also tend to have higher incomes. 
  • On average, men who have married at a later age mostly have lower incomes (there is a bit of an inverse relationship here, especially at higher levels of education)

These statements are not causal: I can easily say “women who have higher incomes tend to marry at a later age,” which is an equivalent point to the one above. It is just a descriptive statement. Contrast these statements with quotes from these articles on the study, including one from the chief author, Brad Wilcox:

These highly educated adults have embraced a “capstone” model of marriage that typically leads them to put off marriage until they have had a chance to establish themselves professionally, personally, and relationship-wise. This capstone model is paying big dividends to the college-educated: Their divorce rate is low, and their income is high. We find, for instance, that college-educated women who postpone marriage to their 30s earn about $10,000 more than their college-educated sisters who marry in their mid-20s

From Ross Douthat in the New York Times:

Upper-class women reap a large wage premium from delaying marriage — a college-educated woman who marries in her 30s earns over $15,000 more annually than a woman who marries in her early 20s, and when you look at household income, the premium for marrying later rises to more than $20,000. Women without 4-year degrees also enjoy a wage premium when they delay marriage, albeit a smaller one (and a very small one when you look at household income). Men, meanwhile, reap a wage premium from marrying earlier, so late marriage tends to hurt their economic prospects.

From Eleanor Barkhorn in the Atlantic:

Financially, college-educated women benefit the most from marrying later. Women who marry later make more money per year than women who marry young.

Using the above data as a basis for their arguments, all of these authors, are, to varying degrees, are making implicit or explicit causal statements: delaying marriage is good for women and bad for men. Yet, given that the Wilcox et al. study is strictly observational, with (as far as I can tell) little effort being made to discern a causal relationship between age of marriage and labour market outcomes, we’re really far more limited in what we can say. Take, for instance, a model of the marriage `market’ where women want to be picky and marry late and high income acts a bargaining chip in the matching process. Richer men will inevitably be able to secure a bride at a much earlier age and richer women will inevitably be able to stave off marriage and find a good husband at a later age. Suddenly, it’s income affecting the age of marriage, not the other way around.

I am not claiming that this model represents “the truth” and that the prevailing explanation doesn’t – far from it, but we can come up with a million different explanations for the correlation observed above which do not involve a direct causal relationship between delaying marriage and income. In general, be cautious when you’re presented with simple stories based on descriptive statistics, both in work like this as well as development research.

Tags: ,

On MPIs and MDGs

Mar 8

Posted by Matt in Development, Economics, Worst practices | 4 Comments

Conan, what is best in life? "It is 1/3 crushing your enemies, 1/3 seeing them driven before you, and 1/3 hearing the lamentation of their women."

Conan, what is best in life? “It is 1/3 crushing your enemies, 1/3 seeing them driven before you, and 1/3 hearing the lamentation of their women.”

Sabine Alkire and Andy Sumner have released a short paper suggesting that the Multidimensional Poverty Index (MPI) be used as a `headline indicator’ for the post-2015 Millennium Development Goals (MDGs). If you’re unfamiliar with the MPI, you can read up on it here. Alkire and Sumner are suggesting that whatever indicators emerge out of the inevitable post-2015 intellectual bloodbath be aggregated into a single index using the same method that is used for the current MPI. This has excited some people, including Duncan Green, who thinks it will be useful in inducing governments to take the post-2015 goals seriously:

That in turn would allow the post2015 process to generate more traction on national governments (the lack of which is the subject of my paper) through league tables. Imagine if every year, all countries (including the rich ones) are ranked on a comprehensive human development table that (unlike the Human Development Index and other similar efforts) has buy in and recognition from across the international community. Each annual report would pick out the countries that have risen/fallen relative to the others. Regional tables could compare India and Bangladesh, or Peru and Bolivia, to generate extra public interest and pressure on decision makers.

I’ll go out and say it: I think this is a really bad idea. It combines the two things that make  two things that make me uncomfortable about both the MPI and the MDGs – arbitrary weights on different indicators/goals and an inflexibility to local preferences.

I’ll use a very basic example: let’s say that the next set of MDGs focuses on two things: hunger and access to clean water. After what will bound to be a seriously convoluted process, someone will agree on internationally-agreed weights on these two things. Let’s say the weights are fifty-fifty, that the final index puts just as much weight on a person who is hungry as one who does not have access to clean water.

Now consider a fictional country, Bigmacistan, which has a culture that sees hunger as being the ultimate state of poverty, much more than clean water. If Bigmacistan were allowed to assign its own weights, it would prefer 3/4 of the total weight to go to hunger and 1/4 to clean water. In fact, given limited resources, Bigmacistan will choose to combat poverty in a way that is not only seen as sub-optimal by the post-MDG framework, but would result in a fall in its global rankings, even if every single person in Bigmacistan is in agreement with its national emphasis on hunger. So differences in MPI 2.0 rankings not only reflect aggregate differences in each country’s success in fighting poverty, but differences in the structure of national social welfare functions.

What one could do is let countries set their own weights (I’ve argued that this is the only way the MPI could even be useful for governments in the long run), but this would never appease the technocrats, because once weights start varying across countries, country rankings start making even less sense.

One could argue that, if there are some indicators that we can reach a reasonably broad consensus on, then imposing these preferences on other countries might be defensible. Unfortunately, this still doesn’t adequately justify the use of the MPI, especially if they are used for annual rankings. Imagine the Bigmacistan actually cares as much about clean water as it does about hunger, but realises that, given its own complex context, it needs to deal with its hunger problem before it will have the capacity to deal with its water access problem. It draws up a national plan which ends hunger by 2020 and then improves access to water by 2025. Yet, from 2015 onwards, Bigmacistan is hounded by donors, NGOs and the media for its poor performance on the MPI 2.0 due to its lack of concern for those living without water.

Finally, any time we want to say anything interesting about the MPI 2.0, we’ll still have to unpack it into its composite indicators, a point Claire Melamed makes on Duncan’s blog:

Say the MPI 2.0, or whatever you called it, went up, or down, in a given country. You’d need an extra layer of data analysis – always fatal as that’s the point you lose people’s attention – to know why. It could be that health outcomes got a lot better, but education outcomes got a bit worse, and so the overall MPI score went up a bit. This would neither be helpful for policy makers, nor tell you much about what people think is important, and it would all be much too complicated to generate any campaigning or political energy anyway.

I do think MPI has its uses, but could we please avoid creating another worldwide indicator that doesn’t tell us very much and imposes what will ultimately be imposing fairly arbitrary weights on individual countries?

Tags: , ,

Angry post about empirical methods and philosophical plumbing

Jan 24

Posted by Matt in Research, Worst practices | 6 Comments

hendel_thewire_post

“We wish to announce we will no longer be reporting annual murder rates, because they represent a world view in which there is only one conceptualisation of “murder.” Murder is actually a fairly complicated, complex process, and to simply “count” these murders only stands to hide the philosophical basis for considering a crime a murder and ignores theories of change as to how or why murders happen. Anyway, the stats are all juked anyway.”

Perhaps this is nitpicking, but there was brief moment while reading Rosalind Eyben and Chris Roche’s rebuttal post on evidence in policymaking (part of a must-read exchange with Chris Whitty and Stefan Dercon), that nearly resulted in an early-morning brain aneurysm:

Let’s start by insisting that a criterion for rigorous research is that it should be explicit about its assumptions or world-view. We suggest that a weakness in many studies is that they usually focus solely on the methodological and procedural and render invisible their ‘philosophical plumbing’. The evidence-based approaches that Stefan and Chris advocate are imposing a certain view of the world, just as our approaches do. Their claims to the contrary foreclose any possible discussion about the different intellectual traditions in interpreting reality.  Theory invites argument and debate.

This argument is made time and time again with those who are both unfamiliar and intimidated by empirical methods. Let me be clear here: a comparison of means does very little to “impose a certain view of the world.” It is just a comparison of means. If I have run a randomised control trial on fertilizer use, I am answering the question “Did this treatment increase fertilizer on use, on average?” To argue that measurement has some sort of inherent, insidious philosophical underpinning is a dangerous and backward way to approach life. A breathalyser test uses various assumptions to measure a person’s blood alcohol level, but I can’t very well go about rejecting its validity because it doesn’t take into account the power relationship between the cop and the driver.

Can the use of rigorous empirical research be used to support theory or ideology? Of course. Are empirics often insufficient to answer really difficult questions. Of course. It is also the case that economists tend to think about problems a certain way, and this might not always be the way a problem needs to be thought about. Are sociological, anthropological and political methods often just as useful for providing evidence? Of course. Should these results often be considered carefully, keeping in mind the context and the various complexities and confounding factors? Of course. 

But measuring poverty, or infant mortality, while rife with methodological assumptions, does not rely on a certain view of the world, unless you classify “I believe some things should be measured” as a world view. So please, stop rejecting simple statistics as a “different intellectual tradition in interpreting reality” – it is really a very silly thing to say and diverts the argument from what really matters: what tools are best for promoting development, and how best can we implement these tools? Rigorous empirical methods are just another tool in the toolbox. Your view of the world will determine which of these tools you rely on the most.

I swear, I think this blog spends half its time trying to put the die-hard randomistas in their place and the other half trying to put the die-hard qualitatives in their place. I need to have a lie down.

Tags:

Panel beating

Jan 8

Posted by Matt in Aid, Development, Worst practices | 2 Comments

panel

Spot the speaker from a developing country. Oh wait, you can’t.

 

In response to the under-representation of women on speaking panels in the tech industry, Rebecca Rosen suggested that speakers pledge to only participate when there will be at least one female speaker. This spurred a fair amount of self-reflection from the development crowd on Twitter, neatly summed up and subsequently critiqued by Duncan Green:

They may not be as extreme as geeky tech events, but lots of development gabfests do indeed feature men on the panel talking to women (and men) in the audience. That violates basic fairness, inhibits the profile and (possibly) career development of half of the potential talent pool, and is likely to distort the agenda and resulting discussion (less focus on care economy, women’s rights etc).

I won’t dwell on the mechanism design here – although whether or not pledges would work and what the resulting equilibrium would be is certainly worth further thought. However, I will point out a more glaring inequity in the typical composition of development panels. Let’s see if you can spot it: above is a photo of a recent IIES panel on using aid to reduce world poverty. Can you spot the missing party?  Perhaps our pledges need to begin with geography before gender.

The Twelve Days of Christmas (Aid Edition)™

Dec 24

Posted by Matt in Worst practices | Comments off

A repost from a few years back, but always worth it. Maybe next year I’ll hire a choir to sing it:

On the twelfth day of Christmas my donors gave to me

twelve delayed disbursements!

eleven sketchy studies

ten consultants calling

nine economists arguing

eight mission meetings

seven worthless workshops

six gender trainings

five RCTs!

four 4x4s

three acronyms

two empty schools

and a lecture on M&E!

–

Still on sale.

Of mice and men working for the BBC

Sep 3

Posted by Matt in Africa, Worst practices | 3 Comments

“I’m holding onto this mouse until food prices go up”

Over at the BBC, Ros Atkins describes my favourite (well – in theory) roadside snack in Malawi: mice-on-a-stick. Always fun to discuss, but unfortunately Atkins tries to paint it as some sort of response to rising world food prices:

Food prices around the world are rising. Drought in the farming states of the American midwest and poor crops in eastern Europe have pushed up the cost of corn and wheat.

The increase in food costs has forced some people in developing countries to make changes in their diet and food choices.

In Malawi, which has the highest poverty levels in africa, young men sell cooked mice to make money and provide cheap food.

Classic – three sentences strung together with heavily-implied causality.  This is pretty silly – mice-on-a stick have been around for as long as I can remember. While the market for skewered rodents might be restricted to countries familiar with food scarcity, they are not a response to the recent food price increases. Maybe the demand for mice has gone up, but it would be nice to see something more than by-the-hip roadside anecdote. What is it about mice-on-a-stick that brings out the worst in journalists?

Randomised monkey trials

Aug 30

Posted by Matt in Economics, Research, Worst practices | Comments off

“What, you humans never tried calorie restriction?”

Following a series of animal studies showing benefits to health, cognition and longevity, the practice of calorie restriction has been getting a lot of attention over the past few years. While the mechanisms were never well-understood, limiting caloric intake by around 30% was seen as a shortcut to a longer lifespan. I’ve always been a bit skeptical and hesitant to embrace the calorie restriction camp, in part because a life in which my hummus intake falls by a third is not a life I would find worth living, but also because the most pertinant results have been based on a single randomised controlled trial of rhesus monkeys, conducted by the University of Wisconsin.

Those results were challenged recently by a similar trial conducted with rhesus monkeys by the National Institute for Aging, which found that treatment monkeys were healthier, but didn’t actually live any longer than those without the restricted diet.

Why am I writing about monkey trials on a development blog? The results of the two trials offer some important lessons for interpreting and relying on RCTs, which are quickly becoming the standard method of identifying development impacts.

The first thing to take away is that while RCTs can allow us to accurately identify treatment effects, we need to carefully consider what treatment we are measuring. Ideally, a control group should look identical to how the treatment group would have looked if they hadn’t been treated. In the University of Wisconsin study, while the treatment group was subject to calorie restriction, monkeys in the control group were allowed to eat as much as they wanted. While we’d really like to know the “impact of restriction above and beyond a normal diet,” the treatment effect measured in the Wisconsin study was something closer to “the impact of restriction above and beyond an all-you-can eat buffet at the Golden Corral.” It is hardly surprising then that the treatment group fared better and a good reason to be suspicious of the results. We cannot always be certain that a study has no effect on the control group – imagine a job-training programme which allows treated individuals to access jobs at the expense of untreated individuals – and so we should pay extra careful attention to what happens to controls groups, not just those who receive the treatment.

The second thing to note is that restricting your analysis to a particular subgroup of individuals or a limited set of outcomes can be tricky. The University of Wisconsin study limited its measure of mortality to “age-related deaths.” According to the New York Times, there was no difference in total mortality between the two groups, meaning that the reduction in age-related mortality (if it is to be believed) might have been offset by an increase in other types of mortality. Be wary of studies which subdivide outcomes like this without reporting aggregates, as rises in one indicator of success might easily be offset by another. In general, be skeptical when results don’t hold for the aggregate, but do for some magically-defined subgroup.

Finally, we need to keep going back to check up on our treatment and control groups for as long as possible. Both these monkey trials went on for over two decades, and while positive health results have been apparent for quite some time, it is only recently that the mortality rates have been high enough to detect (or reject) a difference. It’s possible that we’re measuring a lot of positive impacts today which aren’t going to amount to much in the long run.

Now, if you’ll excuse me, I’m going to break for elevenses.

Economics 101 and the wrath of Oxfam

Aug 1

Posted by Matt in Economics, Worst practices | 7 Comments

“If only you knew the power of the dark side! In my day it was discussed in section 4.2 of the textbook.”

In the morning, before I get up and have breakfast or exercise, I like to stay in bed a little longer and read over the morning’s news and latest blog posts. This puts me at great risk, because it increases the chance that I’ll come across something that will really annoy me and thus spoil my pleasant morning. This is indeed a first world problem, but nevertheless one which has led to this blog post.

Over at From Poverty to Power, Duncan has linked to this blog post by Oxfamer Kate Raworth, who claims that economics textbooks lack the sophistication and tools necessary to deal with the issues that come about from human interaction with the environmental space. She asserts that this diagram represents the current way of economic thinking:

She then points out that this simplistic diagram (which I have yet to come across in a textbook – readers could you help me out? turns out it is from the Wikipedia page for circular flow of income, thanks @brettkeller) ignores three main things:

  1. Environmental degradation (use of natural resources, pollution, climate change).
  2. Unmeasured/non-monetary sectors of the economy (mainly work at home)
  3. Inequality

Let’s deal with her first point, shall we? Raworth claims:

First, the economy does not float freely against a white background. It is embedded within the planet’s environment, drawing on its natural resources and dumping pollutants back out into it. Mention that and an economist will say – ah yes, environmental externalities, we’ll come to those later. But calling nature’s resources ‘externalities’ and leaving them till later has led us to this crisis of climate change. How can it make sense to treat the fundamental resource on which all life depends as a factor external to the system?

First: “externality” doesn’t mean “external to the system,” it indicates a positive or negative impact which a given agent (person, firm, etc) doesn’t normally factor into their decision making. Economists label externalities as such because we think they are important, not because we are trying to shunt them under the rug.

Secondly, climate change hasn’t happened because people picked up Econ 101, didn’t see “the environment” mentioned as anything but an externality (or – admittedly – at all) and decided that it wasn’t important. Climate changed has happened because agents (citizens and firms and governments) don’t internalize the environmental impact of their actions. Not enough people think carefully about their energy usage. Not enough firms are given incentives to consider the environmental impact of their output. Economics has quite a lot to say about fixing externalities for a very, very long time (for example, the idea of Pigovian taxes has been around for more than 90 years), and to claim that climate chance is somehow the result of our way of thinking is a little silly.

Raworth also laments the lack of concern for the “unpaid care economy,” worrying that it will lead us to misunderstand the working lives of many of the world’s women. It is true that this is often left out of the discussion of the economy, but mainly because it’s pretty difficult to measure, not because no one cares about it. It’s also true that Econ 101 doesn’t have a lot to say about household chores or child care, but economics as a whole certainly does – Ms. Raworth would do well to have a look around the JEL classifications codes, especially in D and J. Her assault completely ignores decades of research on the household, the family and interactions between husbands and wives.

Similar things could be said about inequality – yes, Econ 101 is mum about it, often retreating to pareto efficiency without making any normative statements about initial allocations. In fact, much of Raworth’s objection seems to be that economics as a discipline isn’t normative enough, but I see this as a strength. It is up to us as a society, not economists, to determine how much we care about inequality. Yet, public policy requires a complex calculus of weighing different things that we care about; what economics and other social sciences try to do is make that calculus easier, by coming up with many different ways to measure income inequality and poverty, the impact of both on myriad indicators of well-being, including economic growth, health, (increasingly) the environment, and so on.

There is a missed opportunity here: I completely understand Raworth’s concerns and agree with her that we should be urgently focused on internalising the environment and inequality, and do more to measure what is currently unmeasured. Yet these concerns don’t validate her argument that economics is fundamentally lacking. Quite the contrary – we have been talking about externalities for a long time. We’ve been talking about inequality for a long time. We’ve been talking about the household economy for a long time. We have quite a large toolbox – some of the tools are better than others, and some of them are quite flawed, but perhaps a discussion as to which of them are more useful and which need refinement would be more fruitful than a casual dismissal of the way economics is taught.

Should more of these thing be making it into Econ 101? Some of them – like externalities – certainly do. It was one of the first things I learned in my undergrad economics courses. Then most of my lecturers -wrongly – dismissed externalities as something that would be dealt with by the Coase theorem. This is because many of them were libertarians who hated the idea of government intervention, not because economics told them to believe it to be true. Similarly, Ms. Raworth and I share very similar educational backgrounds: we both took the MSc in Economics for Development at Oxford and we both ended up as ODI Fellows in Africa. That our views diverge has less to do with the type of economics we were taught and more to do with who we are. This shouldn’t stop useful discourse – Ranil was taught quite a different style of economics during his masters, yet we managed to write a blog together.

I don’t believe I have the “wrong model of the world” stuck in the back of my head. Perhaps my tool set is weird, imperfect, and overly-mathematical, but that doesn’t stop me from caring about inequality and climate change.

Addition: this post could also be titled “In which Matt defends microeconomics, but has little energy for macro”

Tags: ,