Angry post about empirical methods and philosophical plumbing


“We wish to announce we will no longer be reporting annual murder rates, because they represent a world view in which there is only one¬†conceptualisation¬†of “murder.” Murder is actually a fairly complicated, complex process, and to simply “count” these murders only stands to hide the philosophical basis for considering a crime a murder and ignores theories of change as to how or why murders happen. Anyway, the stats are all juked anyway.”

Perhaps this is nitpicking, but there was brief moment while reading¬†Rosalind Eyben and Chris Roche’s rebuttal post on evidence in policymaking (part of a must-read exchange with Chris Whitty and Stefan Dercon), that nearly resulted in an early-morning brain¬†aneurysm:

Let‚Äôs start by insisting that a criterion for rigorous research is that it should be explicit about its assumptions or world-view. We suggest that a weakness in many studies is that they usually focus solely on the methodological and procedural and render invisible their ‚Äėphilosophical plumbing‚Äô. The evidence-based approaches that Stefan and Chris advocate¬†are¬†imposing a certain view of the world, just as our approaches do. Their claims to the contrary foreclose any possible discussion about the different intellectual traditions in interpreting reality.¬† Theory invites argument and debate.

This argument is made time and time again with those who are both unfamiliar and intimidated by empirical methods. Let me be clear here: a comparison of means does very little to “impose a¬†certain¬†view of the world.” It is just a comparison of means. If I have run a randomised control trial on fertilizer use, I am answering the question “Did this treatment increase¬†fertilizer¬†on use, on average?” To argue that measurement has some sort of inherent,¬†insidious¬†philosophical underpinning is a dangerous and backward way to approach life. A¬†breathalyser test uses various assumptions to measure a person’s blood alcohol level, but I can’t very well go about rejecting its validity because it doesn’t take into account the power relationship between the cop and the driver.

Can the use of rigorous empirical research be used to support theory or ideology? Of course. Are empirics often insufficient to answer really difficult questions. Of course. It is also the case that economists tend to think about problems a certain way, and this might not always be the way a problem needs to be thought about. Are sociological, anthropological and political methods often just as useful for providing evidence? Of course. Should these results often be considered carefully, keeping in mind the context and the various complexities and confounding factors? Of course. 

But measuring poverty, or infant mortality, while rife with¬†methodological¬†assumptions, does not rely on a certain view of the world, unless you classify “I believe some things should be measured” as a world view. So please, stop rejecting simple statistics as a “different intellectual tradition in interpreting reality” – it is really a very silly thing to say and diverts the argument from what really matters: what tools are best for promoting development, and how best can we implement these tools? Rigorous empirical methods are just another tool in the toolbox. Your view of the world will determine which of these tools you rely on the most.

I swear, I think this blog spends half its time trying to put the die-hard randomistas in their place and the other half trying to put the die-hard qualitatives in their place. I need to have a lie down.