Aid Thoughts

Digesting the difficult decisions of development

Almost as awesome as the title suggests

Apr 30

Posted by Matt in Research | 1 Comment

remington

Charles Remington *is* the treatment

A new working paper, titled “Household Vulnerability to Wild Animal Attacks in Developing Countries: Experimental Evidence from Rural Pakistan.” Alas, this does not involve crazy academics running around unleashing wild animals on unsuspecting villages. The abstract:

Based on a three-year panel dataset of households collected in rural Pakistan, we first quantify the extent to which farmers are vulnerable to attacks by wild boars; we then examine the impact of an intervention on households’ capacity to reduce related income losses. A local nongovernmental organization implemented the intervention as a randomized controlled trial at the beginning of the second survey year. This experimental design enabled us to cleanly identify the impact of the intervention. We find that the intervention was highly effective in eliminating the crop-income loss of treated households in the second year, but that effects were not discernible in the third year. The finding from the third year could be due to the high implicit cost incurred by the households in implementing the treatment. Regarding the impact of the intervention on a number of consumption measures, the difference-in-difference estimate for the impact on consumption was insignificant in the second year, but highly positive in the third year when estimated without other controls. A part of this consumption increase was because of changes in remittance inflows. The overall results indicate the possibility that treatment in the absence of subsidies was costly for households due to hidden costs, and hence, the income gain owing to the initial treatment was transient.

So instead of randomising boar attacks, they randomised what I will dub a boar counter-insurgency strategy:

With the help of the district’s agriculture and livestock departments, PHKN designed a pilot version of the Anti-WBA Program (AWBAP). The main objective of this program was to prevent WBAs and subsequent crop-income losses. The program comprises HRD training that focuses on the awareness and prevention of WBAs. The prevention component of the program imparts information on basic techniques for scaring or trapping animals and for curtailing boar-population growth. Moreover, under the program, some basic equipment and animal drugs were provided free of charge to the treated households, upon the successful completion of training.

Drugs? From the footnote:

Drugs are used in the long term to control the boar population. It is claimed that female boars lose their fertility after consuming the drugs; however, the efficacy of the drugs has not yet been established.

So, using The Ghost and the Darkness as an analytical framework (which, frankly, I do for most things in life), they aren’t randomising the lions, they’re randomising Michael Douglas.

Hat tip to Ranil for finding this one.

Tags: ,

In which your lab experiment might not be entirely representative

Apr 22

Posted by Matt in Research | No Comments

life-of-brian1_poster

Self-selection can be a bitch when you want a representative sample

A new paper by Elaine Liu, Paul Frijters and Tao Kong:

We compare the characteristics and regression coefficients between the participants in a field experiment in China and the survey population from which they were recruited. The experimental participants were more educated, younger, more likely to be male, more risk-loving and work fewer hours than the more general population. The estimates of their regression coefficients in the standard analyses of wages, happiness and entrepreneurship differed significantly from non-participants, indicating that inferences drawn from experimental samples may not hold for more representative groups of the population.

Lab experiments have always caught flak for non-representative participant groups, which more often than not still comprise Western university students. The rise of lab experiments in the field, where populations more suitable for the context of the study are targeted, led to somewhat robust claims of external validity. Maybe these claims were hasty.

Hat tip to Andres Marroquin.

Tags:

You can never win

Apr 13

Posted by Matt in Africa, Development, Economics | 2 Comments

paula

“I am Paul Atreides, the Muad’Dib, leader of the Fremen of Arrakis, the *only* source of Spice Melange, which gives me rights as Galactic Emperor! ……. oh I see, you can create it synthetically now. This is awkward.”

It appears that it is now possible to synthetically create artemisinin, used primarily (solely?) as a treatment against malaria. This is the stuff of first-pumping and cheering, especially if the synthetic version turns out to be significantly easy and cheaper to obtain than the natural version. This is a real win for global health.

Not so fast, says Jim Thomas in the Guardian, a newspaper which somehow manages to make us feel guilty about any good development-related news by identifying some poor sod who has lost out (a wonderful example is their silly, excessive worrying over Western consumption of quinoa). According to Thomas, the new synthetic version of the drug will put farmers of sweet wormwood (the plant from which artemisinin is usually derived) out of business:

Now it turns out that artemesia farmers are dismissed as entirely expendable. The rejoinder of “let them plant potatoes” seems dismissive of farmer knowledge: farmers understand markets well and those now growing artemisia annua do so because it helps them bring in income. As for the argument that synbio is necessary to eradicate malaria, the botanical approach was already producing more than enough artemisinin to address malaria.

“There is simply no rationale to have a synthetic product on the market when farmers could produce enough raw material to produce the tablets from pulverised high quality plants,” said Professor Hans Herren, World Food Prize winner, who has worked extensively with east African artemesia farmers.

Now, it remains to be seen how much cheaper the synthetic version will be, but let’s assume that it’s significantly cheaper than the natural one (although it turns out the price of the latter is insanely volatile). How much hand-wringing should we actually do if welfare gains for fighting malaria outweigh the welfare losses of having farmers switch to another type of crop?

If you still feel outraged by all this, would you feel differently if we switched out the words “sweet wormwood” with “biofuel crops”? Would you feel differently if someone had discovered a working vaccine for malaria?

Tags: , ,

In which Malawi gives Madonna a spinning roundhouse kick

Apr 10

Posted by Matt in Africa, Worst practices | 1 Comment

norris_malawi

So Malawi was graced by another visit from Madonna recently. Somewhat miffed that she hadn’t received an invitation to go meet with President Joyce Banda, she wrote an overly-personal message to Banda (“Dear Joyce”) to ask if they could meet. To slightly complicate things further, the head representative of Madonna’s charity went after the President’s sister (who used to work for the Raising Malawi) and complained that the Material Girl wasn’t getting the right treatment from the government:

Madonna can continue her work here [even] if the politicians don’t want to welcome her because her work is all about the children who are here. The politicians can stay. Even donors are also surprised that government is treating Madonna like this when she is the biggest private donor in the country

In response, the Malawian government released an 11-point passive-aggressive smackdown. You can read the whole thing here, but one particular point stood out as being awesome and seriously bad ass:

7. If the argument is that because she is an internationally renowned star, and, therefore, Madonna believes she deserved to be treated differently from other visiting foreigners, it is worth making her aware that Malawi has hosted many international stars, including Chuck Norris, Bono, David James, Rio Ferdinand and Gary Neville who have never demanded state attention or decorum despite their equally dazzling stature. [Emphasis added]

Boom.

Hat tip to Kim Yi Dionne at haba na haba, who has covered both Madonna’s PR gaffs and the government response.

norris_poverty

 

Tags: , , ,

Awkward Dar es Salaam signs, cross-cutting transport infrastructure and global health edition

Apr 8

Posted by Matt in Africa, Worst practices | No Comments

bus_station

Super aid worker protection bracelet, activate!

Apr 5

Posted by Matt in Aid, Development | 1 Comment

craig

From the BBC:

A hi-tech bracelet could soon be helping civil rights and aid workers at risk of being kidnapped or killed. When triggered, the personal alarm uses phone and sat-nav technology to warn that its wearer is in danger. Warnings are sent in the form of messages to Facebook and Twitter to rally support and ensure people do not disappear without trace.

A few quick thoughts/questions:

  1. The objective here is to dissuade both murder and kidnapping of aid workers/advocates. What if murder and kidnapping are substitutes, not complements? Won’t this make it relatively less costly for attackers to just shoot their targets, even if it had some general deterrent effects?
  2. If you are wearing this bracelet, isn’t this a signal that you are worth kidnapping?
  3. If the bracelet is successful in deterring kidnappings and murders, what externalities are we imposing on non-bracelet holders? What’s the equilibrium here?
  4. As a friend pointed out on Facebook, “that sounds like a good way to get your arm chopped off”

Tags: ,

On causality and the returns to late marriage

Mar 25

Posted by Matt in Research, Worst practices | 1 Comment

Miss Havisham's vast wealth is undoubtedly due to her failure to marry.

Miss Havisham’s vast wealth is undoubtedly due to her failure to marry.

Over the last week or two there has been a fair bit of chatter about a report released by the University of Virgina’s National Marriage Project. While the results pertain primarily to marriage outcomes in the United States, the general interpretation of those results is a textbook case of making strongly-causal statements using methods which are insufficient for making these claims. These two graphs, documenting the earnings of women and men by age of marriage, provide a starting point for the discussion:

knotyet_income

knotyet_incomemen

 

These descriptive stats paint a pretty clear picture, allowing us to make the following statements:

  • On average, women who have married at a later age also tend to have higher incomes. 
  • On average, men who have married at a later age mostly have lower incomes (there is a bit of an inverse relationship here, especially at higher levels of education)

These statements are not causal: I can easily say “women who have higher incomes tend to marry at a later age,” which is an equivalent point to the one above. It is just a descriptive statement. Contrast these statements with quotes from these articles on the study, including one from the chief author, Brad Wilcox:

These highly educated adults have embraced a “capstone” model of marriage that typically leads them to put off marriage until they have had a chance to establish themselves professionally, personally, and relationship-wise. This capstone model is paying big dividends to the college-educated: Their divorce rate is low, and their income is high. We find, for instance, that college-educated women who postpone marriage to their 30s earn about $10,000 more than their college-educated sisters who marry in their mid-20s

From Ross Douthat in the New York Times:

Upper-class women reap a large wage premium from delaying marriage — a college-educated woman who marries in her 30s earns over $15,000 more annually than a woman who marries in her early 20s, and when you look at household income, the premium for marrying later rises to more than $20,000. Women without 4-year degrees also enjoy a wage premium when they delay marriage, albeit a smaller one (and a very small one when you look at household income). Men, meanwhile, reap a wage premium from marrying earlier, so late marriage tends to hurt their economic prospects.

From Eleanor Barkhorn in the Atlantic:

Financially, college-educated women benefit the most from marrying later. Women who marry later make more money per year than women who marry young.

Using the above data as a basis for their arguments, all of these authors, are, to varying degrees, are making implicit or explicit causal statements: delaying marriage is good for women and bad for men. Yet, given that the Wilcox et al. study is strictly observational, with (as far as I can tell) little effort being made to discern a causal relationship between age of marriage and labour market outcomes, we’re really far more limited in what we can say. Take, for instance, a model of the marriage `market’ where women want to be picky and marry late and high income acts a bargaining chip in the matching process. Richer men will inevitably be able to secure a bride at a much earlier age and richer women will inevitably be able to stave off marriage and find a good husband at a later age. Suddenly, it’s income affecting the age of marriage, not the other way around.

I am not claiming that this model represents “the truth” and that the prevailing explanation doesn’t – far from it, but we can come up with a million different explanations for the correlation observed above which do not involve a direct causal relationship between delaying marriage and income. In general, be cautious when you’re presented with simple stories based on descriptive statistics, both in work like this as well as development research.

Tags: ,

On MPIs and MDGs

Mar 8

Posted by Matt in Development, Economics, Worst practices | 4 Comments

Conan, what is best in life? "It is 1/3 crushing your enemies, 1/3 seeing them driven before you, and 1/3 hearing the lamentation of their women."

Conan, what is best in life? “It is 1/3 crushing your enemies, 1/3 seeing them driven before you, and 1/3 hearing the lamentation of their women.”

Sabine Alkire and Andy Sumner have released a short paper suggesting that the Multidimensional Poverty Index (MPI) be used as a `headline indicator’ for the post-2015 Millennium Development Goals (MDGs). If you’re unfamiliar with the MPI, you can read up on it here. Alkire and Sumner are suggesting that whatever indicators emerge out of the inevitable post-2015 intellectual bloodbath be aggregated into a single index using the same method that is used for the current MPI. This has excited some people, including Duncan Green, who thinks it will be useful in inducing governments to take the post-2015 goals seriously:

That in turn would allow the post2015 process to generate more traction on national governments (the lack of which is the subject of my paper) through league tables. Imagine if every year, all countries (including the rich ones) are ranked on a comprehensive human development table that (unlike the Human Development Index and other similar efforts) has buy in and recognition from across the international community. Each annual report would pick out the countries that have risen/fallen relative to the others. Regional tables could compare India and Bangladesh, or Peru and Bolivia, to generate extra public interest and pressure on decision makers.

I’ll go out and say it: I think this is a really bad idea. It combines the two things that make  two things that make me uncomfortable about both the MPI and the MDGs – arbitrary weights on different indicators/goals and an inflexibility to local preferences.

I’ll use a very basic example: let’s say that the next set of MDGs focuses on two things: hunger and access to clean water. After what will bound to be a seriously convoluted process, someone will agree on internationally-agreed weights on these two things. Let’s say the weights are fifty-fifty, that the final index puts just as much weight on a person who is hungry as one who does not have access to clean water.

Now consider a fictional country, Bigmacistan, which has a culture that sees hunger as being the ultimate state of poverty, much more than clean water. If Bigmacistan were allowed to assign its own weights, it would prefer 3/4 of the total weight to go to hunger and 1/4 to clean water. In fact, given limited resources, Bigmacistan will choose to combat poverty in a way that is not only seen as sub-optimal by the post-MDG framework, but would result in a fall in its global rankings, even if every single person in Bigmacistan is in agreement with its national emphasis on hunger. So differences in MPI 2.0 rankings not only reflect aggregate differences in each country’s success in fighting poverty, but differences in the structure of national social welfare functions.

What one could do is let countries set their own weights (I’ve argued that this is the only way the MPI could even be useful for governments in the long run), but this would never appease the technocrats, because once weights start varying across countries, country rankings start making even less sense.

One could argue that, if there are some indicators that we can reach a reasonably broad consensus on, then imposing these preferences on other countries might be defensible. Unfortunately, this still doesn’t adequately justify the use of the MPI, especially if they are used for annual rankings. Imagine the Bigmacistan actually cares as much about clean water as it does about hunger, but realises that, given its own complex context, it needs to deal with its hunger problem before it will have the capacity to deal with its water access problem. It draws up a national plan which ends hunger by 2020 and then improves access to water by 2025. Yet, from 2015 onwards, Bigmacistan is hounded by donors, NGOs and the media for its poor performance on the MPI 2.0 due to its lack of concern for those living without water.

Finally, any time we want to say anything interesting about the MPI 2.0, we’ll still have to unpack it into its composite indicators, a point Claire Melamed makes on Duncan’s blog:

Say the MPI 2.0, or whatever you called it, went up, or down, in a given country. You’d need an extra layer of data analysis – always fatal as that’s the point you lose people’s attention – to know why. It could be that health outcomes got a lot better, but education outcomes got a bit worse, and so the overall MPI score went up a bit. This would neither be helpful for policy makers, nor tell you much about what people think is important, and it would all be much too complicated to generate any campaigning or political energy anyway.

I do think MPI has its uses, but could we please avoid creating another worldwide indicator that doesn’t tell us very much and imposes what will ultimately be imposing fairly arbitrary weights on individual countries?

Tags: , ,

Some more thoughts on land grabs and tricky statistics

Feb 21

Posted by Matt in Economics, Research | 3 Comments

I suppose you could label this post as my response to Ricardo and Marloes’s response to my post on their recent media brief on the correlation between governance and land grabs. First, I should say that this is all very exciting – it’s nice to have an actual debate about this. NGOs frequently ignore substantive criticism of their analytical work (to be fair, so do a lot of academics), so I must commend Ricardo and Marloes for their enthusiasm and willingness to get in touch and have a reasonable argument about all this.

I think we’ll likely to continue to disagree about when results should be presented (or at least how they should be presented), so I’ll turn my attention to their three main technical points:

 

1) It’s not realistic to assume that investors target poor countries

True, but poor countries themselves might be more likely to put land up for sale. Discerning the difference between targeting and supply-side effects will always be difficult because we only observe actual land deals (in essence, the quantity `consumed’). But this is beside the point – spend fifteen minutes in an economics seminar and you’ll learn that a common way of challenging identifying assumptions is to come up with an equally-credible alternate story. I’ve shown that, at least in this very basic setup, income is a better predictor of a country having a land deal than governance. While my alternate story might be considered implausible (even if it does fit the data better), I really only put it up to point out how equally-flimsy the assumption of investor targeting is.

 

2) My last table is badly specified and then I forget to estimate a hurdle model.

Before delving into the technicalities of this argument, let’s briefly talk about burden of proof. It is Oxfam’s job here to convince us all that investors are targeting countries with poor governance, or at least that there is some consistent correlation between the two. By this very basic metric, I assert that the current analysis falls short, as it doesn’t provide enough evidence to reject the null of no relationship. One doesn’t always need to present and prove an alternate hypothesis, complete with fancy, well-specified econometrics, in order to disprove the one being asserted.

As far as the specification of the first two columns in Table 4 – sure, this is pretty much atheoretic wandering. I’m not going to assert that I’m cleanly identifying any individual channels, but seeing if Oxfam’s relationship stands up (I’m actually trying to help you here guys) once we start controlling for all these things. Multicollinearity doesn’t seem to be preventing some results from shining through. But yes, this is playtime with Stata, although I admit as much up front. See my point about burden of proof here.

Their final point is a technical one – I’m interested in whether, conditional on a country selling any land, governance is correlated with the number of land deals. Technically, this specification is subject to a form of bias due to selection on unobservables: for example, if hotter countries are more likely to sell land, and there is a correlation between temperature and the governance indicators, then estimates in columns (3) and (4) of Table 4 will be biased.  [OK this is not what their point was - see Paul's comment below.] Ricardo and Marloes would be happier if I estimated a model which took this selection into account.

But the problem is: as I point out at the end of my piece, I don’t really buy the selection equation in the first place, and this factors into their third point:

 

3). They take issue with my worry about “bias” in how land deals are reported. 

I’m worried the Land Matrix is a better measure of “number of reports on land deals” than “number of land deals” and that the measure of “have there been any substantial land deals” in the past ten years is really just a measure of “has anyone bothered to submit a news report to the land matrix on your country in the past ten years.” Ricardo and Marloes make a purely theoretical argument that reporting in the UK should be better than reporting in developing countries. If we were talking about general media reporting, I would be inclined to agree, but I’d be surprised if anyone is scanning British newspapers for land deals and submitting the data to the Land matrix.

Furthermore, consider the  final hurdle a land deal must clear to get into the Land Matrix: “entail the conversion of land from local community use or important ecosystem service provision to commercial production.” This seems like it should only be possible in societies where a significant percentage of the population is involved in agriculture and where large scale commercialisation is yet to happen. Sure, the quality of the British government is one of the reasons there aren’t many dodgy deals going on, but we have to remember that Great Britain has already gone through the long process of moving from smallholder farming to relatively large-scale commercial production. Yet Ricardo and Marloes want to code Great Britain  a zero and include it in the selection equation – I’m just not convinced.

How do we move forward? I’m happy that both Marloes and Ricardo want to continue working on this. This is definitely the best outcome – I can think of several ways that one could try and take it a little bit further

  • Let’s start exploiting the time dimension: we have a panel – let’s use it – although I do have a fear that, as several have pointed out, there won’t be enough meaningful variation in the WDI indicators across time to actually identify anything.
  • Number of deals and size - these are, save for my kitchen sink regressions, currently unexploited. As is information on whether or not deals are international or national.
  • Let’s get more data! I’m hesitant to throw a stake into the ground and say it’s time to make a call, especially when the data is as limited as it is. If we could get our hands on district level data (or, in my wildest dreams, GIS data) on land deals, we could start to say so much  more about what’s going on.

Finally, a word to idle academics out there – I implore you to pay more attention to this stuff. We have a hard enough time encouraging replication of our own studies, but I think the world would be a much better place if we sat down from time to time and just tried to recreate “killer facts” that otherwise dominate the discourse. I didn’t start my analysis with any intention to go after Oxfam’s results, but it only took a little while with the data before I realised that the story was much more complex, and worth a second look.

Again, thanks to Ricardo and Marloes for a fun debate (I’ve offered them a second reply if they’d like).

 

Tags: ,

Response from Oxfam: Governance, land grabs and tricky statistics

Feb 18

Posted by Guest in Development, Economics, Research | 1 Comment

by Ricardo Fuentes-Nieva and Marloes Nicholls

It is encouraging to read the post from Aid Thoughts. We appreciate the time he put into Oxfam’s analysis on large scale land deals. Indeed, we were hoping that our blog would spark debate and bring more attention to this topic.

As a quick summary, we took two databases, the Land Matrix and the World Governance Indicators and found that land deals are more likely to occur in countries with lower levels across different governance indicators.  We specified that “This analysis is only the first step towards a more in depth research project. Next steps include a more in depth analysis on the determinants of the number and location of deals”

Aid Thoughts seems to take issue with the use of this kind of analysis when they are so preliminary. There are two things to say to this:  Firstly, and as Aid Thoughts acknowledges, there is other evidence in the development literature that points to the fact that land deals are concentrated in poorly governed countries. Our conclusions were not based only on our analysis and we used the best evidence at hand (both internal and external) to generate a better understanding of the problem (which Aid Thoughts actually helped with his critical review). So, we stand by our decision to publish the preliminary results.

Now, there are a couple of things to discuss on the technical front of his critique. Here are some:

 

1) Investors or governments?

AidThoughts replaces governance indicators with income per capita because they better explain the existence of land deals. This leads him to suggest that “Maybe investors aim for countries who are more willing to sell off land, not because they are poorly governed, but just because they are poor.” This is an interesting idea but if we put aside the regression tables and reflect for a moment, is it sensible to think that land investors are attracted to countries for being poor? Why would investors be attracted to the characteristics of poverty, such as poor infrastructure, limited public services and low levels of education and health? A more interesting hypothesis that AidThoughts raises, and which we think is worth exploring too, is that it might not  be investors who target countries, but bad governments who sell the land of their citizens.

 

2) Truncated sample bias.

AidThoughts recognizes that running OLS with two control variables, as reported in Table 3, is not serious analysis (and yet he managed to muddle the significance of the estimators in his table). But what’s really puzzling is that, in order to prove his point, he then goes on to throw the entire kitchen sink of governance indicators into the next table (Table 4). These indicators are highly correlated amongst them, and it is difficult to find a sensible explanation to specify the model that way.

He then goes on to say of this table:

“In column (1), prior controlling for income, only one of the relationships we expected to see has returned: countries rated low on the rule of law index are more likely to have land deals. Political stability/violence is also associated with land deals, but unfortunately that wasn’t part of Oxfam’s theoretical model. Now, voice and accountability is positively correlated with land deals! Of course, most of these relationships vanish when we toss in income, although it is worth noting that the rule of law measure keeps its significance and sign. So the relationship between governance and land sales seems to be a lot more complex than the Oxfam brief is suggesting.”

That’s a lot of explanation for a badly specified model that includes highly correlated regressors. But that’s not even the most puzzling part of that table. AidThoughts then tries to explain the number of land deals with the same variables but he does not correct for the truncated sample (look how his sample drops from 212 and 183 in the first two columns to just over 50 in the last two). Ignoring the bias in the observed sample is a mistake and something we had identified as a problem, and that’s why we suggested exploring  a double hurdle estimation to understand the issue better.

 

3) Reported land deals bias.

Aid Thoughts briefly mentions the potential problem of bias in the Land Matrix, but we don’t agree that he identified the right direction of bias. He argues that land deals are more likely to be reported in developing countries by diligent activists than in developed countries like the UK. On the contrary, we argue that land deals are much less likely to be ignored in richer countries with freer press, more access to information and better organized civil societies. Does Aid Thoughts seriously believe that a land deal can be more easily concealed in the UK than in the DRC?

Overall, we are very encouraged by Aid Thoughts’ response. He mentions that he can be convinced of the problem with more data and more, better data is on its way according to conversations we’ve had with the people managing the Land Matrix. So here’s our proposal for Matt: let’s work together – rigorously and objectively – on this issue in the next few months to try to better understand what’s driving the land rush. The problem deserves as much attention as we can give to it.

Tags: , , ,