Last week the medical journal Lancet released an article suggesting that, on average, governments that receive more health aid divert tend to shift domestic resources away from health.
The paper made some headlines and upset aid critics and much of the global health community. A part of this has to do with a misunderstanding of what the findings mean – a confusion which isn’t helped by those that propagate incorrect and sensationalist interpretations of the study.
However, a lot of the anger over the results comes from those that do understand the implications of the study, but are angered by an apparent divergence in priorities between the global health community and recipient governments. Both Ranil and Owen Barder talk about this in more detail, although I’ll go through some similar arguments.
These are my scattered thoughts on the whole issue.
First, a quick primer on ‘fungibility’
True fungibility is the ability to take funds intended for one purpose and spend them on something else. If someone gave me £40 and told me to go spend it on food at Tescos, I could easily go spend it on dvds at HMV, because the funds are fungible.
The variation of fungibility that we’re concerned about, which I’ll call indirect fungibility, is the idea that funds earmarked for a specific purpose allows the recipient to divert funds elsewhere. For example, say I need to spend £40 in Tescos anyway to food myself in a week, and someone gives me a £40 gift certificate to shop there. I could spend £80 in tescos (my own money + the gift certificate) but I now prefer to my own £40 on other things. As the gift certificate has “displaced” my expenditure at Tescos, you can say that it the gift is indirectly fungible.
The paper isn’t making the case that health aid is being diverted or misused, but solely that it is correlated with shifts in domestic resources away from health, the same way I would shift my own expenses away from Tescos if you gave me a voucher.
Owen Barder discussed the nuances of the concept of fungibility and indirect fungibility more here.
The narrative that the econometrics can’t pick up
The results in the Lancet Article are reached through heavy-duty statistical (econometric) analysis. Studies like these offer us a window into the long-term processes behind resource allocation in developing countries. Econometric methods can reveal trends that are otherwise invisible to the casual observer, but they are not always useful in explaining the reason behind those trends.
The mistaken narrative that is being wrenched from the article is that donors are being mislead, tricked into letting governments spend their cash elsewhere. In reality, donors pay very close attention to national budgets while they deliberate with governments over future disbursements. Donors, on a yearly basis, will usually know in advance what the recipient’s budget will look like. I spent two years working in the Ministry of Finance’s budget department in Malawi – if there’s anything the donors paid attention to, it was the domestic health budget. Ranil explains is better than I:
…when donors earmark, they also don’t generally expect to peg the Governments’ domestic expenditures. When they do, they make this explicit. In Malawi, for example, under the conditions of the Health Sector Wide Approach, the Government were compelled to maintain real funding levels to the health sector, or donor funding would be reduced. This is easy enough to apply.
This means that visible indirect fungibility is either planned or, at a minimum, condoned by the donor community. Yet many are assuming something truly nefarious is going on. I wonder why? Maybe because the paper’s authors say things like this like this:
“We don’t know what countries are doing with their own money once the donor money comes in,” said Christopher Murray, director of the Institute for Health Metrics and Evaluation at the University of Washington and one of the paper’s authors. Murray said health aid saves millions of lives, but governments need to be more transparent about what they’re spending on.
Reality check: many governments are transparent about what they are spending on, but not every expenditure category has such a massive research and advocacy interest as global health. No one is running around collecting detailed time series data on government expenditures on science and technology expenditure. Just because it isn’t accessible on an internet database, doesn’t mean it isn’t sitting on shelves in the recipient country.
Furthermore, while the econometric results are, in a complicated sense, ‘averages,’ many of those vexed by the results have been assuming that this state of indirect-fungibility holds in very specific cases. Andrew Green at Change.org argues:
Limited budgets force governments to make choices. But in countries facing severe public health epidemics, funding health programming should be at the top of that list.
Yet because the study doesn’t look at indirect fungibility across subgroups of countries with different levels of disease burden, there’s no evidence to suggest that this is indeed a problem for countries with severe public health epidemics (except the broad conclusion that fungibility is worse in sub-Saharan Africa).
In fact, these concerns are brought up in one of the two criticisms of the study which were published alongside it in Lancet. Yet neither the media nor the blogging community has bothered to pick up on these criticisms, like one put forward in the second paper:
Although Lu and colleagues show that crowding out is a real and widespread occurrence, it does not happen in all countries for which international aid for health increases. In countries in which it does take place, it seems to be the result of deliberate policy choices, which have to be reviewed on a case-by-case basis if alternative options are to be widely promoted.
The blogging community has skipped the most important step: critiquing the methods of the paper
Much of the blogging community seems keep on brandishing the results of statistical studies as truth, then arguing the consequences, rather than discussion whether or not the results might be accurate or not. Many of the development twitters (no names mentioned) will tweet “Study shows that X affects Y” without any sense of uncertainty. This is a shame, as it mirrors the way the media reports the results of statistical studies without much critical analysis.
UPDATE: David Roodman provides the much-needed double-fisted econometric analysis chokeslam here.
I have to admit that I’m not at my strongest when discussion complex dynamic panel data methods, but a couple of things jumped out after my first read through:
- A huge proportion of the data in the study was imputed from missing data ( an astonishing 44% for low-income countries!). Imputation, the practices of replacing a missing data point with an “average” of nearby points (a bit more complicated than that) involves making very heavy assumptions about why data is missing. If data from countries is missing for non-random reasons (especially reasons that are connected to fungibility) the results could be heavily bias. Most modern imputation methods can attempt to correct for this, but are still limited in what they can accomplish.
- The authors apply their econometric methods rather mechanistically, with no innate preferences on one approach over another, with large deviations in the results depending on these approaches (one method suggests that despite indirect fungibility, total health spending still goes up, where the other suggests that it might be going down!). These deviations suggest that there are biases here that aren’t very well understood. It doesn’t help that most of the methods used by the authors are used because they were “suggested,” rather than some prior belief about where bias might be coming from. Getting help and suggestions is a natural part of research, but you still have to understand why you follow those suggestions.
To be honest, standards for certain types of empirical research differ across fields of study. For all of our dismal failings, economists are intensely critical of each other’s statistical research. This makes us very hard for us to say anything definite about the world – which is why the debate over whether aid causes growth isn’t likely to be solved any time soon. Yet studies that would make a decent journal and raise a few eyebrows would still be taken with a large grain of salt (there are, of course, some exceptions).
Yet journals like the Lancet which, to be honest, don’t have the same grounding in non-experimental statistical methods are happy to publish interesting studies and let the people run wild with them. Please – when you read about empirical results, from any journal, don’t leap to conclusions. Read the papers – and don’t assume that results immediately represent “the true state of the world.”
Beware the dangers of tunnel-vision advocacy.
The negative reaction to the results of the article is coming mainly from the global health lobby. This is perfectly rational – those working in global health work, live, breath, and advocate it all day long. The news that a $1 in health aid may not results in an equivalent rise in health spending is understandably frustrating.
While health is an incredibly important part of aid, development and human welfare, it’s not the only component. Decisions about how much weight to put on different sectors is a difficult task – and may not be one that many recipient governments are particularly good at. However, I see absolutely no reason why global advocacy centred around different sectors could improve upon this – both Owen and I have written about this before – advocacy is too blunt an instrument for country-level planning.
To the health advocates: funding health is important in every country, but it’s not your call to make that it be the biggest spending priority. When I was in Malawi, the government decided that it would spend a lot of its domestic resources on an expensive agricultural subsidy programme. I didn’t agree with it – but it was their call. This limited their ability to put more money into the health sector. I don’t have the knowledge to say which of the two sectors needed the marginal dollar the most, but really, neither can you.