David Roodman over at the CGD recently posted a great article about the new aid-growth paper published by WIDER, of which he remains a skeptic. Scroll down to the comments, and you’ll see Owen Barder making some very good points in response.
I also posted a comment on the piece. My concern was rather different and probably not best suited for that blog. Among various ramblings, I wrote:
At what point did finding universal answers become the only legitimate basis of economic study? Development has looked and proceeded differently in different places…
Statistics is just one type of evidence. Why have economists apparently forgotten this?
My plea was for an approach to development that took in far more specific case study analysis. By this I mean not the Duflo/Banerjee approach of randomized trials, but a holistic approach to development that focuses more on the actual historical process of development in specific places over real time than abstractions with pretense to universality based on cross section data or studies dealing with specific interventions.
The rationale behind this is simple to me: firstly, a disinterested analysis of what we know about successful development processes emphasises their diversity more than their similarities, though these exist and are important. There is little reason to assume that imagined future development processes will have more uniformity. Secondly, understanding of real development successes and real development failures (however they are defined) demonstrate that they are typically the result of a range of complex interacting factors. In most cases, causal mechanisms have shown inconstancy, with the same phenomenon having markedly different effects depending on context and time, even within the same country.
Of course, this kind of method is messy; it won’t ever give us unambiguous answers as to what works and to what extent. It requires that we formulate policy based on our understanding of the historical circumstances that led to the current state in a country, bearing in mind that the interplay between factors cannot always be modeled as there are concurrent effects on multiple levels (individual interactions, social interactions between groups and state-subject/citizen relations for example) and different effects in different regions or time periods. It will mean that we have to rely on what we know from other countries and other times, trying to tease out the central relevant lessons.
It’s an approach that is anathema to modern economics. The majority of our current work tends towards universality of analysis and conclusion. It seeks to pose theoretical relationships that hold under specific assumptions (which are often implicitly further assumed to hold everywhere if they hold anywhere) and then test them. If the tests work (and for the cynical, even if they don’t), they seek to tell us of a ‘robust’ concrete relationship: each unit of factor X contributes 0.1 units of GDP growth. Paradoxically, this often leads to just as much messiness and uncertainty as a historical analysis. The aid regressions Roodman looks at are a classic example: the exact same data produces opposite results depending on model specification. Neither result is unambiguous even on its own terms.
Economics was not always like this though. Early economists were multidisciplinary creatures by nature. They studied history, social relationships and the economy as interlinked phenomena, using a holistic method that took in historical evidence, theoretical abstractions from this evidence and some further statistical evidence to support their ideas. Statistics was necessarily a smaller part of their work, for their data and the sophistry of their statistical techniques was still quite basic. This period still produced what to my mind remain the two greatest works of economic thought: The Wealth of Nations and Capital.
Since this time, economics has undergone two major structural changes. One is undoubtedly positive: we have developed far better statistical modeling techniques and begun collecting more reliable (though often still flawed) data on economic phenomena. My problem is emphatically not with greater use of statistics.
The second phenomenon is where my concerns arise. Economics has increasingly adopted a theoretical approach called Methological Individualism (MI). MI is an approach that has been tried and discarded in most disciplines, one that builds models on the basis of ‘representative’ individuals and relations. For some reason, however, it has stuck in economics, retained some support in sociology and perhaps in political science. Unlike in sociology and political science, it is now the only acceptable approach for mainstream economics – to the extent that few economics students have even heard the term, for they have never questioned its status as the basis of economics.
This is so much the case that very few economics students actually read Adam Smith or Marx in their original writings; nor even Keynes or Schumpeter. They are not ‘rigorous’ (read: based on individualist mathematical models) enough to be studied except insofar as they have been adapted by modern economists. When studying economics myself I found to my horror that some of the self-professed ‘New Keynesians’ who were taking the same course as I had never actually read The General Theory, though they all claimed to know what he had argued (of course, I am not accusing real New Keynesians like Greg Mankiw of this).
In the period of MI’s dominance, economics has developed its two biggest contemporary flaws: universality and reductionism. They are closely related phenomena. Reductionism is the more direct effect of MI. Through MI, economists seek to model representative relationships; the problem with this is that models quickly become unmanageable when we try and capture the multiple levels on which a relationship works. Individuals react to specific phenomena on an individual level, as part of a group (or several groups) and as subject of a state which has coercive power over them. These reactions may pull in different directions. In some cases, individuals identify with multiple groups that contribute to contradictory responses to a single stimulus.
Of course, there is value to theoretically and analytically ‘cleaning up’ these relationships. It’s crucial, in fact, if we are to try and understand anything. All disciplines of study do this to different extents. The problem is that economics has gone to extremes through its modeling approach and assumes relationships work at their simplest level for most modeling. Even this can have significant benefits, provided we engage with other disciplines and learn from the complexity therein.
However, the insularity of economics as a profession has led to a cessation of such interaction. Instead we seek to apply the same reductionism to other disciplines. A great example is given in this Aid Watch post: It’s entitled History Matters, but the history it presents is remarkably reductionist, as several commenters have pointed out. It is not history, but ‘path dependency’ a different concept that is also based on analysis of past events. History is very rarely about linear relationships. Far more often it is about joint causality, confluence of circumstances, chance and unforeseen events.
Universality stems from reductionism. Because we try and strip away too much of the complexity from our world view, we are left with individual effects that are assumed to hold in all contexts. Econometrics is the statistical tool we use to look for these relationships, and it has many values, done well and with sensitive interpretation. Unfortunately, these two caveats often don’t hold, and the results of a regression are used to advocate for policy directly. Good econometrics can account for context, but sometimes this leads to very complex models and I would argue that our capacity for defining complex relationships is vastly overestimated by many econometricians. Furthermore, data for developing countries isn’t often strong enough to support these models.
Economics needs to restore balance to our consideration of development. Statistics, simplification and modeling can be valuable; but so too can be context, true historical analysis and complexity of causality. Matt is a far better econometrician than I, and we have often had this discussion in the past. I think both of our opinions have been revised on the back of these arguments. Our most recent such exchange was via e-mail. I can’t improve on his final analysis, so I’ll leave the last word to him:
Fruitful research has to incorporate both some rigorous statistics and case studies. I think the issue is that case studies alone have a tendency to conflate different effects. If we look at places where we think aid has a positive effect, we’re already selecting on just that: we’re likely to find positive answers when it conforms to our prior. Sometimes, but not always, you really do have to compare subjects using a large sample to tease out indirect effects. I’ve seen a world based on case-studies, and it’s an equally unsatisfactory world as a one based on pure econometrics.
(For the record, I’m probably more of an economist than a historian. I just find that writing with a more historical perspective adds greater value to development debates than being another economist among many.)