The IMF, inequality and the trickle-down of empirical research

"It took so many assumptions to put you together!"

“It took so many assumptions to put you together!”

By Nicolas Van de Sijpe

recent IMF staff discussion note has received a lot of attention for claiming that a smaller income share of the poor lowers economic growth (see also here and here). This piece in the FT is fairly typical, arguing that the paper “establishes a direct link between how income is distributed and national growth.”

It quotes Nicolas Mombrial, head of Oxfam International’s office in Washington DC, saying that (my emphasis): “the IMF proves that making the rich richer does not work for growth, while focusing on the poor and the middle class does” and that “the IMF has shown that `trickle down’ economics is dead; you cannot rely on the spoils of the extremely wealthy to benefit the rest of us.”

The aim of this blog post is to clarify that the results in Table 1 of  the paper, which are based on system GMM estimation, rely on assumptions that are not spelled out explicitly and whose validity is therefore very difficult to assess. In not reporting this and other relevant information, the paper’s application of system GMM falls short of current best practices. As a result, without this additional information, I would be wary to update my prior on the effect of inequality on growth based on the new results reported in this paper.

The paper attempts to establish the causal effect of various income quintiles (the share of income accruing to the bottom 20%, the next 20% etc.) on economic growth. It finds that a country will grow faster if the share of income held by the bottom three quintiles increases. In contrast, a higher income share for the richest 20% reduces growth. As you can imagine, establishing such a causal effect is difficult: growth might affect how income is distributed, and numerous other variables (openness to trade, institutions, policy choices…) might affect both growth and the distribution of income. Clearly, this implies that any association found between the income distribution and growth might reflect things other than just the causal effect of the former on the latter.

To try to get around this problem, the authors use a system GMM estimator. This estimator consists of (i) differenced equations where the changes in the variables are instrumented by their lagged levels and (ii) equations in levels where the levels of variables are instrumented by their lagged differences (Bond, 2002, is an excellent introduction). Roughly speaking, the hope is that these lagged levels and differences isolate bits of variation in income share quintiles that are not affected by growth or any of the omitted variables. These bits of variation can then be used to identify the causal effect of the income distribution on growth. The problem with the IMF paper is that it does not tell you exactly which lagged levels and differences it uses as instruments, making it hard for readers to assess how plausible it is that the paper has identified a causal effects.

The paper also omits to report a number of routine tests (for serial correlation in the error term, and for the validity of overidentifying restrictions) that could help to shed some light on the likely validity of the assumptions that are made in order to identify a causal effect. It does not report an instrument count, nor does it mention efforts to reduce the number of instruments used (by restricting the number of lags used as instruments, by collapsing the instrument matrix, or by replacing candidate instruments by their principal components). Without such adjustments, system GMM tends to generate too many instruments, weakening the usefulness of the aforementioned tests and reducing the likelihood that the estimated coefficients are causal. It also does not mention which type of system GMM estimator (e.g. 1-step or 2-step) is used, nor how standard errors are obtained. This matters: it is, for instance, well-known that, without an adjustment, the 2-step GMM estimator yields standard errors that are much too low, giving a false sense of precision (Windmeijer, 2005).

Especially in samples of this size, system GMM can be very sensitive to the exact assumptions made. It is good practice to examine how results change when using slightly different assumptions and different sets of instruments. While on page 7 the paper mentions that the results “survive a variety of robustness checks”, no details are provided. Footnote 2 claims that similar results are obtained when controlling for standard growth determinants, such as human and physical capital, but this is rather puzzling: the theory discussed in later pages suggests that human and physical capital accumulation are two of the primary channels through which inequality has an effect on growth. If that is true, then controlling for them should weaken the impact of inequality on growth.

To sum up, when applying system GMM, researchers are required to make a range of choices, in part reflecting the assumptions that they are willing to make about how the different variables are generated in the real world. These choices often matter a great deal in practice, and a rich literature is available that offers guidance on how to make these choices (see e.g. Roodman, 2009a,b). Even a judicious use of system GMM to identify the causal effect of the income distribution on growth is likely to leave at least some unconvinced, as the assumptions involved are not trivial. By not being transparent about these assumptions, which could have been described and motivated in a few pages of appendix, this paper falls short of current best practice in applying system GMM estimation.

It is therefore surprising to see how quickly some readers have treated these results as a breakthrough in establishing the deleterious effects of inequality (in the section on the drivers of inequality, though not in the section discussed in this blog post, the paper itself advises caution about “drawing definitive policy implications from cross-country regression analysis”). Inequality may be an important growth deterrent and this paper’s results may well stand up to further scrutiny but, in the absence of further information, at this point in time I would hesitate to update my priors on the effect on inequality on the basis of the new results that are reported in this paper.

 

References:

BOND, S. R. (2002): “Dynamic panel data models: a guide to micro data methods and practice,” Portuguese Economic Journal, 1(2), 141–162.

ROODMAN , D. (2009a): “How to do xtabond2: an introduction to difference and system GMM in Stata,” Stata Journal, 9(1), 86–136.
——(2009b): “A note on the theme of too many instruments,” Oxford Bulletin of Economics and Statistics, 71(1), 135–158.

WINDMEIJER , F. (2005): “A finite sample correction for the variance of linear efficient two-step GMM estimators,” Journal of Econometrics, 126(1), 25–51.

 

Comments are closed.