Health spending, suggestive studies and the IMF’s guilt-by-association

Larry Elliot of The Guardian reports this morning that “Oxford University-led research finds signs that tough loan conditions imposed by IMF has led to health aid being diverted for other uses”.

Wow? Research from Oxford University? This must be true.

Alas, the devil is still firmly seated in the details. I took a gander at the study this morning, titled International Monetary Fund and Aid Displacement (International Journal of Health Services, 41 1: 67-76). You can find a temporarily ungated version here.

It is an interesting paper, but rather limited in its ability to provide a causal story. The authors compare the growth in health spending in countries that have been borrowing from the IMF since the mid-1990s to those that haven’t, and conclude that there is less `divergence’ (what we used to call fungibility) in countries which don’t take IMF loans.

There is an urgent, painfully obvious problem with concluding anything causal here: countries which decide to accept IMF loans are likely to be very different than those that don’t, and these differences are going to have impacts on health spending, above and beyond the restrictions imposed by IMF conditions.

The authors do recognize this, but they don’t make a big deal about it (they do say included some more controls in alternate specifications, but they don’t bother reporting them!).

A final important point is that countries seeking IMF support are likely to differ from countries that are not, and a request for an IMF loan is often associated with severe economic problems. Nonetheless, even in such circumstances,it is reasonable to expect aid from donors to have at least some positive impact on health funding, especially given that health needs are often greatest at such times.

Unfortunately, I don’t think all of us would consider that a reasonable expectation. There are further issues with this study, which is similar to the study on fungibility that ruffled some feathers last year which David Roodman subsequently demolished.

That doesn’t stop both The Guardian from making a very roundabout conclusion that it might be the IMF’s fault that some countries aren’t meeting their MDG targets:

In an investigation of more than 100 low and middle-income countries, the report sought to explain why increased aid spending had left many countries well off track to hit the United Nations millennium development goals (MDGs) for health, which include a two-thirds reduction in infant mortality and a three-quarter decline in maternal mortality.

They said one likely explanation was that the curbs on public spending stipulated by the fund were encouraging governments in poor countries to use health aid for other needs. Countries that did not borrow from the IMF were found to have channelled 45 cents into health systems for every dollar of aid received.

What needs to happen? The Guardian economics editor needs to start reading articles in non-economic journals with the same skepticism he would apply to econometric studies in economic journals, especially before he begins drafting articles which go about the painstaking task of desperately trying to suggest to the reader that there is a causal story without actually saying it – articles with headlines like “IMF loans `Divert aid from public health’”.

What we also need to see is more push back from academic bloggers. Newspapers will forever keep making causal conclusions from studies which can’t make the same claims, we need to call them out when they do so.

A final important point is that countries seeking IMF support are likely to
differ from countries that are not, and a request for an IMF loan is often associated
with severe economic problems. Nonetheless, even in such circumstances,
it is reasonable to expect aid from donors to have at least some positive
impact on health funding, especially given that health needs are often greatest
at such times.

5 thoughts on “Health spending, suggestive studies and the IMF’s guilt-by-association

  1. Ranil Dissanayake

    January 17, 2011 at 1:22pm

    This was my first thought when I read this story – of course countries borrowing from the IMF will 1) be very different from countries not borrowing from the IMF and 2) They are going to be very different *pre* and *post* borrowing. Why?

    Because there’s a reason they had to borrow in the first place!

    As a COMPLETE aside, I recently read somewhere that the combined number of economics degrees between the editorial staff in the economics sections of the Telegraph, Guardian and one other paper (can’t remember which) is ONE.

    One.

    I’d say ‘would they do this for physics or any other technical discipline?’, but Ben Goldacre has made a career out of pointing out that they do on badscience.net.

    I weep. Really, I weep.

  2. Matt

    January 22, 2011 at 4:06pm

    Hi Amanda – thanks for the link and the extra thoughts. Yes, these things spread way too quickly….

  3. Peter Chowla

    February 3, 2011 at 12:54pm

    Ahh the old correlation versus causation debate. I haven’t yet read the Stuckler et al study in full yet, but at least they do caveat the selection bias. I was very unimpressed by a study I was reading from the IMF on low-income countries response to financial crisis. Look at these two lines right next to each other:

    “A negative relationship between labor market flexibility and the decline in growth: countries with more flexible labor markets saw growth decline by less than others. As
    expected, more flexible labor markets appear to make adjustment to external shocks easier.”
    http://www.imf.org/external/pubs/ft/wp/2011/wp1127.pdf, page 17

    Turning a regression correlation relationship into a causal claim from one sentence to the next, with no discussion of the theory even of the mechanism of causation. And the regressions are admitted earlier in the paper to not really be robust because of “owing to small sample size, in the regression analysis the additional variables could only
    be included one at a time. Hence, no single preferred and encompassing specification including several additional variables could be identified.”

    At least they added “appear to…”

Comments are closed.