A few weeks ago several researchers from UC Berkley, including the impressive development economist Ted Miguel, published a study connecting a rise in temperature with the incidence and onset of civil war in Sub-Saharan Africa. You can find the paper here.
Last week the Times, the BBC, many other news outlets, dozens of blogs and climate change activists have been reporting the link and touting the paper’s claim that climate change will lead to a “54% increase in armed conflict incidence by 2030.”
I looked over the paper behind the result, and found that, while the study is interesting and well thought out, there are several reasons that the conclusions should be taken with a grain of salt.
1. The link between temperature and civil war is not rock solid.
Brad DeLong, quoting Richard Freeman, once mentioned three good rules of thumb for judging econometric results:
- It had better be there in the ordinary-least-squares regression.
- It had better still be there in the econometrically-sophisticated high-tech instrument procedures.
- It had better still be there for small technical tweaks to the econometrically-sophisticated procedures.
Of course, real analysis is more complex than this, but the rules are still a good starting point. We don’t actually see the basic OLS regression which is OK by today’s standards, but it would still be handy to see. For their main specifications, only one of the three models the authors present has statistical significance at the 5% level.
The choice of the 5% level, which roughly corresponds to the probability that the result is being generated by pure chance, as a measure of validity is (while a bit ad-hoc) the standard threshold from where you go “wow, these are interesting results” to “looks neat, shame that nothing’s significant.”
If you churn through the supporting tables (a separate document available here), you’ll find a mix of results, some of them significant at the 5% level, some at the 10% level (which is worse) and many not significant at all.
To make matters worse, the confidence intervals indicate that we’re 95% confident that the effect could be much larger (8%), much, much smaller (.01%), or anywhere in between. This may seem like nitpicking, but the true value of the coefficient (the effect of temperature changes on civil war) has huge implications for their resulting predictions. If it was closer to .01%, then we wouldn’t have quite so much to worry about. The authors use the whole distribution in their projection, but still use the median of their projected increase (which could be as low as 6.2%!).
2. There are a lot of if’s in the projections for 2030.
Projections like these are often made ceteris paribus (holding everything else constant). To their credit, the authors do use both optimistic and pessimistic projections on the growth of income and governance in these countries. However, they fails to account for the future in several ways – because their study fails to find any significant relationship between governance and/or income with civil war (a result I find very hard to accept), rising incomes and governance do very little to offset the probability of civil war in their model.
While we may not be able to make precise statements about exactly how much income it takes to avoid a civil war, I think it’s reasonable to assume that growth and good government do make civil war less likely. Furthermore, I’d expect that growth and good government might make also mitigate the effect of climate change on the probability of civil war (democracies are more likely to respond to weather-related disasters). There must be interaction effects going on, but they aren’t explored in the study.
3. If the projections are true, isn’t the answer still “civil war prevention” rather than “climate change” mitigation?
The paper and many of those citing it suggest that the answer to climate-fueled civil war is working at the country-level to mitigate the effects of rising temperatures. This might be so, but shouldn’t we also consider other ways of buffering countries against the incidence and duration of civil war? Most civil wars happen in an environment of extremely low governance, so how realistic is it to assume we can implement large-scale climate change mitigation in countries that are essentially failed states?
4. The long tail
I should note that I do believe that climate (obviously) plays a role in modern conflict – this isn’t the first study to suggest it. When your crops fail due to unusually warm temperatures, joining that marauding band chaps with AK-47s seem a little more reasonable. I think the study, which is open and honest about its approach, is certainly more reasonable than slapdash NGO reports which haphazardly total numbers from wildly different sources. It is a very interesting and worthwhile study, but I just think we should be hesitant to infer too much from it.
COP-15 is fast-approaching, and advocates are already adding “climate change will lead to a 50% increase in civil wars” to their list of arguments. I think climate change is very scary business, not because we know precisely what will happen, but because there are many known unknowns and unknown unknowns. Even if the expected damage turns out to be quite low, it is the long tail of the distribution that should be especially disquieting. This is what should prompt our efforts to offset the rise in global temperatures, not a barrage of unverifiable statistics.
#1 by Ranil Dissanayake on November 27, 2009 - 11:42 am
nice piece. There’s another big fault with most regression analyses of civil war: they don’t take into much account group relations and motivation. While, as you say, crop failure due to high temp might make one more likely to pick up the AK and join the thugs, conflict still needs motivation. Otherwise, you’re talking about “lawlessness”, which is not the same thing.
I think it’s far more likely that climatic factors may make it more difficult to mitigate conflicts or may ‘dry the touchpapers’ for conflict, but any conflict will ultimately be local in source, direction and defining protagonists. All of which is of central importance for understanding their impact, solution etc.
#2 by Andy on November 27, 2009 - 2:11 pm
Agree with both of you.
Distilling human motivation into a set of quantifiable proxies, or ignoring social, cultural, political and judicial concerns altogether, is an incredibly distortive process.
Furthermore, the concept of additional “battle deaths” betrays a startling degree of ignorance in regard to conflict in Africa, where the vast majority of mortality is both civilian and as a result of entitlement failure and forced migration, not involvement in “battles”.
#3 by Matt on November 27, 2009 - 2:46 pm
Ranil – I agree that context does matter, but I also believe there are some universal things that can lead to conflict in nearly-failed and low governance states, and that econometrics can pick these things up, and when it does, it should be taken reasonably seriously. For example, there is a significant a persistent connection between natural resources and civil conflict.
If you look at the actual conflicts around natural resources, they often have different “motivations,” for the conflicts (just listen for five minutes to the rampaging militias in the DRC, or the LRA), but often these motivations are just cover for fundamentals: low opportunity vs a high-risk change of extremely high payoffs.
If oil was discovered tomorrow in Malawi, a civil war might not happen (as you say, context matters), but the chance of one happening certainly would have gone up.
Andy – it’s only distortive if the ignored variables are correlated with what you’re measuring. It may not be a useful result, but on average it still holds, even if the effects are heterogeneous because of social, cultural, etc.
The point of using “battle deaths” wasn’t to get a good measurement on the actual damage from the civil war, but just to measure the *incidence* of civil war. If there’s just a brief skirmish or two, it might not be enough to qualify as a civil war.
#4 by Andy on November 28, 2009 - 12:46 am
I will attempt to respond through the post-pub haze!
From my perspective, econometric analysis of conflict is not an objective tool that can “pick things up” – it is a highly subjective, reductive methodology that regularly misrepresents human agency in its attempt to define it. There may be a connection between resource and conflict, but not one that has ever been convincingly, econometrically defined. In regard to this study, consider the importance of, for example:
- Ease of extraction (e.g. “easy” alluvial diamonds vs “hard” offshore oil)
- Geographic distribution
- Ethnic / cultural / class distribution of returns
- Socio-economic marginalisation
- Political exclusion
All of these things are not only relevant, they can change the outcome of resource discovery / conflict completely. These factors have a very direct correlation to what the study is measuring but are not taken into account.
In regard to “battle deaths”, I understood this term as defining the “actual damage” – it is used in this context in the abstract. As a term it is highly misrepresentative of what actually happens in civil war, implying that military confrontation is the primary cause of death. The qualification of civil war is a controversy in itself – war economies often, if not usually, encourage military actors to avoid direct confrontation. As such, skirmishes or direct engagement are a very poor way of gauging incidence.
#5 by Matt on November 28, 2009 - 1:24 am
Andy,
Econometrics doesn’t attempt “define human agency,” it’s a statistical toolset that can be adapted to many different specifications. All of the things you mention matter, but that doesn’t mean that you can’t use statistical methods to identify average effects. If you found, for example, that striking oil in a country increased the probability of civil war by 30%, that would be an average effect for the group you’re looking at.
Of course the country-specific effect will vary based on country-level contexts…. often econometric specifications try and use interaction terms to tease out heterogeneous effects, but this can be more data demanding. When doing country-level work, most good economists would admit that they are dealing with average effects, and that there might be particular reasons why some countries might have lower or higher probabilities.
But the average effects *are still meaningful* Yes, before you start bargining into a country yelling “I’ve done the regression you’re going to have a civil war!” you should be able to understand the local context, but average effects are good things to know when you start determining what matters on aggregate, even if you have to get more detailed than that when doing analysis on a country-by country basis.
I think battle deaths are a great way of measuring incidence. If people aren’t actually killing each other, then it’s highly unlikely you’ve got a civil war on your hands! Battle deaths includes civilians.
Anyone murdered in a civil war is a battle death – someone that dies of cholara because someone has blown up the local hospital isn’t – the latter isn’t discernable from deaths caused from having a crap government.
#6 by Andy on November 28, 2009 - 1:08 pm
Thanks for this, most illuminating. Generally agree, though with a couple of qualifications:
“Battle deaths” – I admit pedantry, but language is important in this regard. Massacres of civilians (the primary cause of violent death in African civil conflict) are not “battles” and to imply as such is highly misrepresentative. This becomes problematic when, as you mention, such reports are picked up by non-specialist organisations like the BBC.
Econometric analysis – The pressure to make statistical models less “data demanding” provides a highly distortive incentive to look at a) the lowest number of factors possible and b) look only at those that are most easily quantifiable. This leads to an abundance of econometric conflict analyses that identify average effects relating to temperature, economy and resource , but that ignore less stat-friendly social, political, psychological and cultural factors that may be more significant. This lack of balance is reflected in the policy response, which tends towards technocratic solutions (e.g. the various agricultural recommendations in this paper) and not, say, political reform or social inclusion. It may also serve to further marginalise the legitimate grievances of a potentially violent population.
#7 by Ranil Dissanayake on November 30, 2009 - 5:30 am
Matt –
I actually agree with your points in response to me for the most part. Two more things to add, though:
Firstly, econometrics can pick up unifying themes across examples, and that is very useful. So do other subjects – I think you’d struggle to find a historian who didn’t recognise the central importance of natural resources to African civil war. Most though, would then also go on to describe how the definition of group identity and historical evolution or bunching of these within the borders created by colonisation and decolonisation interact with the importance of natural resources.
still, both approaches are useful, particularly when good econometrics is done to pick up obscure relationships.
Second – I think the bigger value of econometrics for development is in identifying the relationship. Data isn’t often good enough or unambiguously defined and measured enough to make the actual coefficients that useful.