Aid Thoughts

Digesting the difficult decisions of development

Archive for category Research

Almost as awesome as the title suggests

Apr 30

Posted by Matt in Research | 1 Comment

remington

Charles Remington *is* the treatment

A new working paper, titled “Household Vulnerability to Wild Animal Attacks in Developing Countries: Experimental Evidence from Rural Pakistan.” Alas, this does not involve crazy academics running around unleashing wild animals on unsuspecting villages. The abstract:

Based on a three-year panel dataset of households collected in rural Pakistan, we first quantify the extent to which farmers are vulnerable to attacks by wild boars; we then examine the impact of an intervention on households’ capacity to reduce related income losses. A local nongovernmental organization implemented the intervention as a randomized controlled trial at the beginning of the second survey year. This experimental design enabled us to cleanly identify the impact of the intervention. We find that the intervention was highly effective in eliminating the crop-income loss of treated households in the second year, but that effects were not discernible in the third year. The finding from the third year could be due to the high implicit cost incurred by the households in implementing the treatment. Regarding the impact of the intervention on a number of consumption measures, the difference-in-difference estimate for the impact on consumption was insignificant in the second year, but highly positive in the third year when estimated without other controls. A part of this consumption increase was because of changes in remittance inflows. The overall results indicate the possibility that treatment in the absence of subsidies was costly for households due to hidden costs, and hence, the income gain owing to the initial treatment was transient.

So instead of randomising boar attacks, they randomised what I will dub a boar counter-insurgency strategy:

With the help of the district’s agriculture and livestock departments, PHKN designed a pilot version of the Anti-WBA Program (AWBAP). The main objective of this program was to prevent WBAs and subsequent crop-income losses. The program comprises HRD training that focuses on the awareness and prevention of WBAs. The prevention component of the program imparts information on basic techniques for scaring or trapping animals and for curtailing boar-population growth. Moreover, under the program, some basic equipment and animal drugs were provided free of charge to the treated households, upon the successful completion of training.

Drugs? From the footnote:

Drugs are used in the long term to control the boar population. It is claimed that female boars lose their fertility after consuming the drugs; however, the efficacy of the drugs has not yet been established.

So, using The Ghost and the Darkness as an analytical framework (which, frankly, I do for most things in life), they aren’t randomising the lions, they’re randomising Michael Douglas.

Hat tip to Ranil for finding this one.

Tags: ,

In which your lab experiment might not be entirely representative

Apr 22

Posted by Matt in Research | No Comments

life-of-brian1_poster

Self-selection can be a bitch when you want a representative sample

A new paper by Elaine Liu, Paul Frijters and Tao Kong:

We compare the characteristics and regression coefficients between the participants in a field experiment in China and the survey population from which they were recruited. The experimental participants were more educated, younger, more likely to be male, more risk-loving and work fewer hours than the more general population. The estimates of their regression coefficients in the standard analyses of wages, happiness and entrepreneurship differed significantly from non-participants, indicating that inferences drawn from experimental samples may not hold for more representative groups of the population.

Lab experiments have always caught flak for non-representative participant groups, which more often than not still comprise Western university students. The rise of lab experiments in the field, where populations more suitable for the context of the study are targeted, led to somewhat robust claims of external validity. Maybe these claims were hasty.

Hat tip to Andres Marroquin.

Tags:

On causality and the returns to late marriage

Mar 25

Posted by Matt in Research, Worst practices | 1 Comment

Miss Havisham's vast wealth is undoubtedly due to her failure to marry.

Miss Havisham’s vast wealth is undoubtedly due to her failure to marry.

Over the last week or two there has been a fair bit of chatter about a report released by the University of Virgina’s National Marriage Project. While the results pertain primarily to marriage outcomes in the United States, the general interpretation of those results is a textbook case of making strongly-causal statements using methods which are insufficient for making these claims. These two graphs, documenting the earnings of women and men by age of marriage, provide a starting point for the discussion:

knotyet_income

knotyet_incomemen

 

These descriptive stats paint a pretty clear picture, allowing us to make the following statements:

  • On average, women who have married at a later age also tend to have higher incomes. 
  • On average, men who have married at a later age mostly have lower incomes (there is a bit of an inverse relationship here, especially at higher levels of education)

These statements are not causal: I can easily say “women who have higher incomes tend to marry at a later age,” which is an equivalent point to the one above. It is just a descriptive statement. Contrast these statements with quotes from these articles on the study, including one from the chief author, Brad Wilcox:

These highly educated adults have embraced a “capstone” model of marriage that typically leads them to put off marriage until they have had a chance to establish themselves professionally, personally, and relationship-wise. This capstone model is paying big dividends to the college-educated: Their divorce rate is low, and their income is high. We find, for instance, that college-educated women who postpone marriage to their 30s earn about $10,000 more than their college-educated sisters who marry in their mid-20s

From Ross Douthat in the New York Times:

Upper-class women reap a large wage premium from delaying marriage — a college-educated woman who marries in her 30s earns over $15,000 more annually than a woman who marries in her early 20s, and when you look at household income, the premium for marrying later rises to more than $20,000. Women without 4-year degrees also enjoy a wage premium when they delay marriage, albeit a smaller one (and a very small one when you look at household income). Men, meanwhile, reap a wage premium from marrying earlier, so late marriage tends to hurt their economic prospects.

From Eleanor Barkhorn in the Atlantic:

Financially, college-educated women benefit the most from marrying later. Women who marry later make more money per year than women who marry young.

Using the above data as a basis for their arguments, all of these authors, are, to varying degrees, are making implicit or explicit causal statements: delaying marriage is good for women and bad for men. Yet, given that the Wilcox et al. study is strictly observational, with (as far as I can tell) little effort being made to discern a causal relationship between age of marriage and labour market outcomes, we’re really far more limited in what we can say. Take, for instance, a model of the marriage `market’ where women want to be picky and marry late and high income acts a bargaining chip in the matching process. Richer men will inevitably be able to secure a bride at a much earlier age and richer women will inevitably be able to stave off marriage and find a good husband at a later age. Suddenly, it’s income affecting the age of marriage, not the other way around.

I am not claiming that this model represents “the truth” and that the prevailing explanation doesn’t – far from it, but we can come up with a million different explanations for the correlation observed above which do not involve a direct causal relationship between delaying marriage and income. In general, be cautious when you’re presented with simple stories based on descriptive statistics, both in work like this as well as development research.

Tags: ,

Some more thoughts on land grabs and tricky statistics

Feb 21

Posted by Matt in Economics, Research | 3 Comments

I suppose you could label this post as my response to Ricardo and Marloes’s response to my post on their recent media brief on the correlation between governance and land grabs. First, I should say that this is all very exciting – it’s nice to have an actual debate about this. NGOs frequently ignore substantive criticism of their analytical work (to be fair, so do a lot of academics), so I must commend Ricardo and Marloes for their enthusiasm and willingness to get in touch and have a reasonable argument about all this.

I think we’ll likely to continue to disagree about when results should be presented (or at least how they should be presented), so I’ll turn my attention to their three main technical points:

 

1) It’s not realistic to assume that investors target poor countries

True, but poor countries themselves might be more likely to put land up for sale. Discerning the difference between targeting and supply-side effects will always be difficult because we only observe actual land deals (in essence, the quantity `consumed’). But this is beside the point – spend fifteen minutes in an economics seminar and you’ll learn that a common way of challenging identifying assumptions is to come up with an equally-credible alternate story. I’ve shown that, at least in this very basic setup, income is a better predictor of a country having a land deal than governance. While my alternate story might be considered implausible (even if it does fit the data better), I really only put it up to point out how equally-flimsy the assumption of investor targeting is.

 

2) My last table is badly specified and then I forget to estimate a hurdle model.

Before delving into the technicalities of this argument, let’s briefly talk about burden of proof. It is Oxfam’s job here to convince us all that investors are targeting countries with poor governance, or at least that there is some consistent correlation between the two. By this very basic metric, I assert that the current analysis falls short, as it doesn’t provide enough evidence to reject the null of no relationship. One doesn’t always need to present and prove an alternate hypothesis, complete with fancy, well-specified econometrics, in order to disprove the one being asserted.

As far as the specification of the first two columns in Table 4 – sure, this is pretty much atheoretic wandering. I’m not going to assert that I’m cleanly identifying any individual channels, but seeing if Oxfam’s relationship stands up (I’m actually trying to help you here guys) once we start controlling for all these things. Multicollinearity doesn’t seem to be preventing some results from shining through. But yes, this is playtime with Stata, although I admit as much up front. See my point about burden of proof here.

Their final point is a technical one – I’m interested in whether, conditional on a country selling any land, governance is correlated with the number of land deals. Technically, this specification is subject to a form of bias due to selection on unobservables: for example, if hotter countries are more likely to sell land, and there is a correlation between temperature and the governance indicators, then estimates in columns (3) and (4) of Table 4 will be biased.  [OK this is not what their point was - see Paul's comment below.] Ricardo and Marloes would be happier if I estimated a model which took this selection into account.

But the problem is: as I point out at the end of my piece, I don’t really buy the selection equation in the first place, and this factors into their third point:

 

3). They take issue with my worry about “bias” in how land deals are reported. 

I’m worried the Land Matrix is a better measure of “number of reports on land deals” than “number of land deals” and that the measure of “have there been any substantial land deals” in the past ten years is really just a measure of “has anyone bothered to submit a news report to the land matrix on your country in the past ten years.” Ricardo and Marloes make a purely theoretical argument that reporting in the UK should be better than reporting in developing countries. If we were talking about general media reporting, I would be inclined to agree, but I’d be surprised if anyone is scanning British newspapers for land deals and submitting the data to the Land matrix.

Furthermore, consider the  final hurdle a land deal must clear to get into the Land Matrix: “entail the conversion of land from local community use or important ecosystem service provision to commercial production.” This seems like it should only be possible in societies where a significant percentage of the population is involved in agriculture and where large scale commercialisation is yet to happen. Sure, the quality of the British government is one of the reasons there aren’t many dodgy deals going on, but we have to remember that Great Britain has already gone through the long process of moving from smallholder farming to relatively large-scale commercial production. Yet Ricardo and Marloes want to code Great Britain  a zero and include it in the selection equation – I’m just not convinced.

How do we move forward? I’m happy that both Marloes and Ricardo want to continue working on this. This is definitely the best outcome – I can think of several ways that one could try and take it a little bit further

  • Let’s start exploiting the time dimension: we have a panel – let’s use it – although I do have a fear that, as several have pointed out, there won’t be enough meaningful variation in the WDI indicators across time to actually identify anything.
  • Number of deals and size - these are, save for my kitchen sink regressions, currently unexploited. As is information on whether or not deals are international or national.
  • Let’s get more data! I’m hesitant to throw a stake into the ground and say it’s time to make a call, especially when the data is as limited as it is. If we could get our hands on district level data (or, in my wildest dreams, GIS data) on land deals, we could start to say so much  more about what’s going on.

Finally, a word to idle academics out there – I implore you to pay more attention to this stuff. We have a hard enough time encouraging replication of our own studies, but I think the world would be a much better place if we sat down from time to time and just tried to recreate “killer facts” that otherwise dominate the discourse. I didn’t start my analysis with any intention to go after Oxfam’s results, but it only took a little while with the data before I realised that the story was much more complex, and worth a second look.

Again, thanks to Ricardo and Marloes for a fun debate (I’ve offered them a second reply if they’d like).

 

Tags: ,

Response from Oxfam: Governance, land grabs and tricky statistics

Feb 18

Posted by Guest in Development, Economics, Research | 1 Comment

by Ricardo Fuentes-Nieva and Marloes Nicholls

It is encouraging to read the post from Aid Thoughts. We appreciate the time he put into Oxfam’s analysis on large scale land deals. Indeed, we were hoping that our blog would spark debate and bring more attention to this topic.

As a quick summary, we took two databases, the Land Matrix and the World Governance Indicators and found that land deals are more likely to occur in countries with lower levels across different governance indicators.  We specified that “This analysis is only the first step towards a more in depth research project. Next steps include a more in depth analysis on the determinants of the number and location of deals”

Aid Thoughts seems to take issue with the use of this kind of analysis when they are so preliminary. There are two things to say to this:  Firstly, and as Aid Thoughts acknowledges, there is other evidence in the development literature that points to the fact that land deals are concentrated in poorly governed countries. Our conclusions were not based only on our analysis and we used the best evidence at hand (both internal and external) to generate a better understanding of the problem (which Aid Thoughts actually helped with his critical review). So, we stand by our decision to publish the preliminary results.

Now, there are a couple of things to discuss on the technical front of his critique. Here are some:

 

1) Investors or governments?

AidThoughts replaces governance indicators with income per capita because they better explain the existence of land deals. This leads him to suggest that “Maybe investors aim for countries who are more willing to sell off land, not because they are poorly governed, but just because they are poor.” This is an interesting idea but if we put aside the regression tables and reflect for a moment, is it sensible to think that land investors are attracted to countries for being poor? Why would investors be attracted to the characteristics of poverty, such as poor infrastructure, limited public services and low levels of education and health? A more interesting hypothesis that AidThoughts raises, and which we think is worth exploring too, is that it might not  be investors who target countries, but bad governments who sell the land of their citizens.

 

2) Truncated sample bias.

AidThoughts recognizes that running OLS with two control variables, as reported in Table 3, is not serious analysis (and yet he managed to muddle the significance of the estimators in his table). But what’s really puzzling is that, in order to prove his point, he then goes on to throw the entire kitchen sink of governance indicators into the next table (Table 4). These indicators are highly correlated amongst them, and it is difficult to find a sensible explanation to specify the model that way.

He then goes on to say of this table:

“In column (1), prior controlling for income, only one of the relationships we expected to see has returned: countries rated low on the rule of law index are more likely to have land deals. Political stability/violence is also associated with land deals, but unfortunately that wasn’t part of Oxfam’s theoretical model. Now, voice and accountability is positively correlated with land deals! Of course, most of these relationships vanish when we toss in income, although it is worth noting that the rule of law measure keeps its significance and sign. So the relationship between governance and land sales seems to be a lot more complex than the Oxfam brief is suggesting.”

That’s a lot of explanation for a badly specified model that includes highly correlated regressors. But that’s not even the most puzzling part of that table. AidThoughts then tries to explain the number of land deals with the same variables but he does not correct for the truncated sample (look how his sample drops from 212 and 183 in the first two columns to just over 50 in the last two). Ignoring the bias in the observed sample is a mistake and something we had identified as a problem, and that’s why we suggested exploring  a double hurdle estimation to understand the issue better.

 

3) Reported land deals bias.

Aid Thoughts briefly mentions the potential problem of bias in the Land Matrix, but we don’t agree that he identified the right direction of bias. He argues that land deals are more likely to be reported in developing countries by diligent activists than in developed countries like the UK. On the contrary, we argue that land deals are much less likely to be ignored in richer countries with freer press, more access to information and better organized civil societies. Does Aid Thoughts seriously believe that a land deal can be more easily concealed in the UK than in the DRC?

Overall, we are very encouraged by Aid Thoughts’ response. He mentions that he can be convinced of the problem with more data and more, better data is on its way according to conversations we’ve had with the people managing the Land Matrix. So here’s our proposal for Matt: let’s work together – rigorously and objectively – on this issue in the next few months to try to better understand what’s driving the land rush. The problem deserves as much attention as we can give to it.

Tags: , , ,

Governance, land grabs and tricky statistics

Feb 14

Posted by Matt in Development, Economics, Research | 3 Comments

there-will-be-blood

“I’m a family man- I run a family business. And I heard your town scored low on the World Bank’s governance indicators.”

The last decade has been marked by a sudden increase in large scale land purchases in developing countries, a `land rush’ which has purportedly been driven by concerns over food security, food prices and a growing market for biofuels. The speed, size and lack of transparency over many of these deals, as well as their implications for the welfare and food security of those already living on the land, has led many to dub these large scale purchases as “land grabs.” This is a rather loaded term, but has successfully (and unfortunately) framed the context as one where anonymous, uncaring investors are systematically snatching land away from the poor and needy.

News reports suggest that at least some of this is happening – following the excellent Let’s Talk Land Tanzania for just a few days reveals how problematic some of these purchases have been. Yet, despite the ruckus these deals are creating, we still know precious little about their size and scale, the motivations and expectations of investors, the welfare impact on those in “grabbed” countries and the welfare impact on those in “grabbing” countries.

This lack of knowledge should be alarming rather than disarming, but while this is the perfect time for careful, dispassionate analysis and data collection, many have chosen to instead reinforce the simple “good” vs “evil” story I highlighted above. Take, for instance, this media briefing which Oxfam released last week, based on preliminary research on the relationship between country governance and land deals.

The two Oxfam researchers, Ricardo Fuentes-Nieva and Marloes Nicholls, use data from the World Bank’s Worldwide Governance Indicators (WGI) and the Land Matrix, which gathers data on media reports of land deals, to show that countries that had any land deals between the years 2000 and 2011 had significantly lower WGI scores than those that hadn’t had any. Here is the figure which they use to make their case:

land-grabs-and-governance

Again, this figure reveals that, across the four governance indicators considered by Fuentes-Neiva and Nicholls, countries with land deals consistently score worse than those without. How does Oxfam interpret these results?

Oxfam believes that investors actively target countries with weak governance in order to maximise profits and minimise red tape. Weak governance might enable this because it helps investors to sidestep costly and time-consuming rules and regulations, which, for example, might require them to consult with affected communities. Furthermore in countries where people are denied a voice, where business regulations are weak or non-existent, or where corruption is out of control it might be easier for investors to design the rules of the game to suit themselves.

So we have a pretty clear story here, right? Well, maybe not. Let me give a bit more structure to the above results by showing them as a series of bivariate regressions of the probability of observing a land deal in any given country between 2000 and 2011 and the average governance indicators for this period (the same data used in the Oxfam briefing).

oxfam1

Each column shows the results from regressing the probability of the country having at least one land deal during this period on each measure separately: voice and accountability, regulatory quality, rule of law and corruption (note that higher is `better’ for each of these measures). So far so good: in isolation, each of these variables is significantly* and negatively correlated with the probability of a land deal (i.e. countries that score poorly on each of these indicators individually are more likely to sell off land).

Yet, it’s a little strange that each of these seems to have about the same magnitude of an effect. We might expect some indicators to matter more. Also, for some reason, the Oxfam brief has left out two other WDI measures: political violence/stability and government effectiveness. Here is what the authors say about this exclusion:

Two of the Worldwide Governance Indicators – political stability and the absence of violence and government effectiveness were excluded from the analysis since there is no evident mechanism that would lead these aspects of governance to improve prospects for  investors.

OK – so we have a somewhat solid theoretical reason for excluding these variables. Presumably, we should not see the same negative correlation between these two WDI measures and land grabs. Table 2 below includes two extra columns in which I re-run the above results, but including both of the excluded indicators (PS and GE).

Read the rest of this entry »

Tags:

Angry post about empirical methods and philosophical plumbing

Jan 24

Posted by Matt in Research, Worst practices | 6 Comments

hendel_thewire_post

“We wish to announce we will no longer be reporting annual murder rates, because they represent a world view in which there is only one conceptualisation of “murder.” Murder is actually a fairly complicated, complex process, and to simply “count” these murders only stands to hide the philosophical basis for considering a crime a murder and ignores theories of change as to how or why murders happen. Anyway, the stats are all juked anyway.”

Perhaps this is nitpicking, but there was brief moment while reading Rosalind Eyben and Chris Roche’s rebuttal post on evidence in policymaking (part of a must-read exchange with Chris Whitty and Stefan Dercon), that nearly resulted in an early-morning brain aneurysm:

Let’s start by insisting that a criterion for rigorous research is that it should be explicit about its assumptions or world-view. We suggest that a weakness in many studies is that they usually focus solely on the methodological and procedural and render invisible their ‘philosophical plumbing’. The evidence-based approaches that Stefan and Chris advocate are imposing a certain view of the world, just as our approaches do. Their claims to the contrary foreclose any possible discussion about the different intellectual traditions in interpreting reality.  Theory invites argument and debate.

This argument is made time and time again with those who are both unfamiliar and intimidated by empirical methods. Let me be clear here: a comparison of means does very little to “impose a certain view of the world.” It is just a comparison of means. If I have run a randomised control trial on fertilizer use, I am answering the question “Did this treatment increase fertilizer on use, on average?” To argue that measurement has some sort of inherent, insidious philosophical underpinning is a dangerous and backward way to approach life. A breathalyser test uses various assumptions to measure a person’s blood alcohol level, but I can’t very well go about rejecting its validity because it doesn’t take into account the power relationship between the cop and the driver.

Can the use of rigorous empirical research be used to support theory or ideology? Of course. Are empirics often insufficient to answer really difficult questions. Of course. It is also the case that economists tend to think about problems a certain way, and this might not always be the way a problem needs to be thought about. Are sociological, anthropological and political methods often just as useful for providing evidence? Of course. Should these results often be considered carefully, keeping in mind the context and the various complexities and confounding factors? Of course. 

But measuring poverty, or infant mortality, while rife with methodological assumptions, does not rely on a certain view of the world, unless you classify “I believe some things should be measured” as a world view. So please, stop rejecting simple statistics as a “different intellectual tradition in interpreting reality” – it is really a very silly thing to say and diverts the argument from what really matters: what tools are best for promoting development, and how best can we implement these tools? Rigorous empirical methods are just another tool in the toolbox. Your view of the world will determine which of these tools you rely on the most.

I swear, I think this blog spends half its time trying to put the die-hard randomistas in their place and the other half trying to put the die-hard qualitatives in their place. I need to have a lie down.

Tags:

Aid as policy

Jan 14

Posted by Matt in Aid, Development, Economics, Research | 2 Comments

empire

“”The Empire is evil”? Don’t you realise what a silly statement that is? The Empire is actually a complex system of carefully targeted programmes and government departments. You should be asking specific questions like: “Is the Empire’s Death Star project effective at curbing population growth?”"

Lee Crawfurd over at The Roving Bandit recently wrote a compelling post about why the question “Does aid work?” is fundamentally flawed.

The question “does policy work” is jarring, because we immediately realise that it makes little sense. Governments have about 20-30 different Ministries, which immediately implies at least 20-30 different areas of policy. Does which one work? We have health and education policy, infrastructure policy (roads, water, energy), trade policy, monetary policy, public financial management, employment policy, disaster response, financial sector policy, climate and environment policy, to name just a few. It makes very little sense to ask if they all collectively “work” or are “effective”. Foreign aid is similar. Aid supports all of these different areas of policy. My colleagues and I at OPM work on aid-financed projects that support most of these different policy areas in different developing countries.

Lee, as he admits himself, is taking his cue from the combined work of Esther Duflo, Abhijeet Banerjee and Dean Karlan. Some policy questions are becoming more and more answerable: does X work in a given context is something that can be tested and applied. Lee is asserting that since aid is just used to fund policies, thus the question of whether aid works just boils down to the complex task of determining whether or not individual policies work.

Yet, when you examine this assertion a little further, it starts to fall apart. Let’s imagine we lived in a world where we, development economists from aid-giving countries, figured out all the good government policies. We knew exactly which actions developing country governments needed to take to save children’s lives, promote income and job creation, reduce hunger and conflict and so on. In the world which Lee has presented to us, policy works, because we knew what all the best policies were. Would aid work?

It certainly wouldn’t be guaranteed to. Aid isn’t just policy – it’s the transfer of financial resources and technical expertise from one country or entity to another. That transfer is inherently non-trivial: it can create huge differences in power, has the potential to distort the recipient’s decision-making, creates opportunities for rent-seeking and often is used for completely political purposes. We cannot, for instance, only judge a US-funded conditional cash-transfer programme in Afghanistan solely on its microeconomic impact – it has to be viewed within the context of the US’s ongoing military intervention in that country, and its likelihood of long-term success. The recent scrap between Rwanda and its donors over the security situation across the border in the DRC again shows that aid is vastly more complicated than simply choosing effective programmes.

Lee acknowledges some of these differences and potential problems, but then dismisses them as things which are hard to gather robust evidence on. This preference to stick to what we know is somewhat admirable and tempting, but ultimately dangerous: it is incredibly difficult to gather widespread, robust evidence on the effects of aid on the macroeconomy or on local political economy. This should make us more, rather than less, wary of possible deleterious effects.

I’m as equally horrified as Lee by the recent attention that right-wing attacks on aid have been getting in the UK, and I completely agree with him that aid cannot always be  judged as a whole. The question, “does aid work?” doesn’t get us very far in life, especially since we have little concept as to what metric we should be using. That said, we have to tread carefully around the argument that small, neat questions are sufficient for success. I agree that aid should be considered more carefully as a bundle of heterogeneous flows and relationships, but I also believe that “aid” is unique in several key ways, and it is only healthy if we continuously question whether the the things that make aid unique also undermine its effectiveness.

Tags: , ,

Ask not what your country can do for you

Jan 9

Posted by Matt in Africa, Development, Economics, Research | 5 Comments

bilbo

“Sorry Bilbo, I was going to take you on this *amazing adventure*, but then I checked your expenses from last year, and you seem to be spending your entire budget on food, not travelling.”

Economists can sometimes be a little sceptical of asking people what they want. If we’re trying to provide and finance a public good, for instance, we might be worried that beneficiaries will understate their value of that good to try and get away with paying less for it. Others – often those in the behavioural science camp – can be wary that people may not be reasonably informed as to what is good for them, or might let cognitive quirks and biases undermine their prioritisation.

Over at the African Can End Poverty blog, this scepticism seems to have been extended to Tanzanian businessmen, as Jacques Morisset argues that we should pay less attention to what local firms claim are policy priorities:

Allow me to illustrate. According to the entrepreneurs operating In Tanzania, electricity is their major constraint (85 per cent) followed by access to finance (52 per cent), taxes (37 per cent), and administrative red tape (25 per cent). Source: World Bank. Investment Climate Assessment, 2009. Surprisingly, labor and transports costs are only at the bottom of their concerns (less than 10 per cent). According to this ranking, the priority should be therefore given to reducing electricity costs, increasing access to finance and reducing taxation.

A closer look at the firms’ financial balance sheets provides a different picture. In reality, electricity counts for a marginal share of firms’ operating costs in Tanzania (see Figure). For example, it is equivalent to only 3 per cent for a standard firm operating in the apparel sector. In other words, a decline, say, of 50 per cent in electricity prices would only reduce its costs by 1.5 per cent – hardly a high number for such a big effort. By contrast, transport and labor costs are equivalent to 41 per cent and 38 per cent of its total operating costs. This means that reducing transport costs by only 4 per cent would achieve the same gains for the enterprise than cutting by half its energy costs.

I’m not entirely convinced by Morissets argument: he only presents data on the current breakdown of firm’s operating costs, but no evidence on how firm electricity usage might change if prices did come down. This is a little like arguing that that since poor, stunted children in a rural village only appear to consume maize, there’s little point in subsidising the cost of protein-rich foods.

Morisset admits that electricity access might be an issue, but then goes on to make his argumet based on the static view: that we should target inputs which are currently the most costly for Tanzanian firms. Perhaps this it the right course, but a difficult argument to make without more information on how firms change their behaviour when relative prices change.

Tags: , ,

Random thoughts left lying around

Dec 23

Posted by Matt in Economics, Research | Comments off

There has been much talk of economists starting up a trial registry for randomised interventions, or at least promoting the use of pre-analysis plans. One of the chief reasons for doing this is to curb data mining – if researchers make it clear up front which hypotheses they plan to test, this will reduce the incentive to report new results, discovered only after the researchers have had time to dig around.

While I think trial registries are worth a try, I have already discussed my worries their effects on the quantity of viable research (even if quality increases). These concerns aside, my question here is: why are trial registries primarily associated with randomised trials? Shouldn’t we also be moving to an equilibrium where all empirical research begins with a published pre-analysis plan?

I suppose the main hurdle is honesty here – for any dataset which already exists, it’s easy for me to download it, mine the data, then base my pre-analysis plan on empirical results I already know to exist. Furthermore, for any given dataset, the number of potential  hypotheses (and thus the number of pre-analysis plans which can be written by different researchers) is very large. This suggests that there is something special about writing a pre-analysis plan before the data is even collected, rather than before someone opens up Stata.

Tags: ,