The rise of empirical econ, in one chart


Goddammit I need more memory

I apologize for the click-baiting title, but this is pretty cool. John C. McCallum has assembled a (rough) estimate of the price of computer memory (mainly RAM) over time. I’ve adjusted the prices for inflation and graphed it over time. The results are pretty amazing (keep in mind the y-axis is log-scale).


Without cheap memory, you can say goodbye to big data sets and complex calculations which really enabled empirical econ to take off. Sure, CPU speeds matters as well and the RCT folks were always a little less reliant on large data sets, but can you imagine having to bootstrap those standard errors with 2mb of RAM? File this under “things are getting better.” Hat tip to Data is Plural, a newsletter you really should subscribe to if you like random data sets.

I’m upset that my football team lost, so I’m going to have to ask you to leave the country


I AM THE LAW! Except when the Broncos lose. Then I just turn to jelly.

In the NYT, immigration judges contemplate how biases might creep into the decisions they make:

In all, 336 people from 13 countries and even more ethnic backgrounds appeared in San Francisco’s immigration court recently over three days. All of them were facing possible deportation, because they either were in the United States illegally or had committed crimes serious enough to jeopardize their legal presence as noncitizens. One challenge facing Judge Marks was deciding whether to deport some of them immediately after they had testified. Another challenge was her own biases.

“You have to go through some hypotheticals in your brain,” said Judge Marks, wrestling with the weighty decisions she must make, the little time she has to make them and all the impressions she and her judicial colleagues form from the bench about the immigrants before them.

“Would I treat a young person the same way I’m treating this old person?” she said. “Would I treat a black person the same way I’m treating this white person? This situation of rush, rush, rush as fast as we can go, it’s not conducive to doing that.”

The solution? Anti-bias training:

Now, as the country struggles with how these instinctive judgments shape our lives, the Justice Department is trying to minimize the role of bias in law enforcement and the courts. More than 250 federal immigration judges attended a mandatory anti-bias training session in August, and this summer the Justice Department announced that 28,000 more employees would go through a similar exercise.

This seems reasonable, but what about factors that influence decisions that go beyond the characteristics of the immigrant? Enter a recent (unpublished) paper by Daniel Chen:

I detect intra-judge variation in judicial decisions driven by factors completely unrelated to the merits of the case, or to any case characteristic for that matter. Concretely, I show that asylum grant rates in U.S. immigration courts differ by the success of the court city’s NFL team on the night before, and by the city’s weather on the day of, the decision. My data including half a million decisions spanning two decades allows me to exclude confounding factors, such as scheduling and seasonal effects. Most importantly, my design holds the identity of the judge constant. On average, U.S. immigration judges grant an additional 1.5% of asylum petitions on the day after their city’s NFL team won, relative to days after the team lost. Bad weather on the day of the decision has approximately the opposite effect. By way of comparison, the average grant rate is 39%. In contrast, I do not find comparable effects in sentencing decisions of U.S. District Courts, and speculate that this may be due to higher quality of the federal judges, more time for deliberation, or the constraining effect of the federal sentencing guidelines.

Yikes. If it’s true, then there are all sorts of external factors which affect the fates of thousands of asylum seekers, some of whom are turned away because the judge is just having a bad day. This wouldn’t be the first paper to find that irrelevant, external factors influence judicial decisions. A recent paper by Ozkan Eren and Naci Mocan find similar effects (this time via college football – go figure) on decisions in juvenile courts. Others have found that judges are less likely to rule in the defendant’s favour when they are hangry.

Maybe judges should take some sort of mood test before they are allowed to review cases. Or maybe, despite what the folks at ProPublica think, it’s time to let the machines do the work for us.

Hat tip to Charles Kenny for the Chen paper.

The European Union wants to use aid to kill people. Seriously.


Here we go. Deep breaths. From the Guardian:

When international donors and the Afghan government convene in Brussels next week, the EU secretly plans to threaten Afghanistan with a reduction in aid if the war-torn country does not accept at least 80,000 deported asylum seekers.

According to a leaked restricted memo (pdf), the EU will make some of its aid “migration sensitive”, even while acknowledging that security in Afghanistan is worsening.

Let’s be absolutely clear here: deporting people from Europe to Afghanistan harms them. The evidence of the enormous individual benefits of migration is – at this point – pretty irrefutable. At the very least, sending people to a poor conflict ridden country condemns them to a lower lifetime income, fewer opportunities, worse health outcomes and shorter lives (the average life expectancy – unweighted – in Europe is 78, in Afghanistan it is 60). Even worse, deported people face persecution and violent deaths. If we send 80,000 people back to Afghanistan, some of them will die unnecessarily and many more will  suffer.

I understand that countries have to deport people sometimes. But when these nasty, cynical policies are tied to development aid, those aid policies are forever tainted. It is another worrying sign that EU aid is being used to harm and endanger the very people it should be helping.

When I saw Alfonso Cuarón’s adaptation of PD James’s “Children of Men” six years ago, I worried it might some day come true. I worry more now. Shame on these people. Shame on all of them.

The sum of parts

You’re doing it wrong

The IMF has a new paper out on gender budgeting efforts in sub-Saharan African countries:

Gender budgeting is an initiative to use fiscal policy and administration to address gender inequality and women’s advancement. A large number of sub-Saharan African countries have adopted gender budgeting. Two countries that have achieved notable success in their efforts are Uganda and Rwanda, both of which have integrated gender-oriented goals into budget policies, programs, and processes in fundamental ways. Other countries have made more limited progress in introducing gender budgeting into their budget-making. Leadership by the ministry of finance is critical for enduring effects, although nongovernmental organizations and parliamentary bodies in sub-Saharan Africa play an essential role in advocating for gender budgeting.

These sorts of efforts have certainly improved in both scope and sophistication. Back when I worked in the budget division of the Malawian Ministry of Finance, I was only asked once to perform any sort analysis of the gender focus of the budget. The request that landed on my desk had come from the Commonwealth, who wanted to know how many times the word “gender” had been used in any of the previous presentations of the national budget to parliament. After fishing out the transcripts from the Ministry’s library, I eventually discovered the answer was “zero.”

Crowdfunding lives

Crowds sometimes choose inefficient ways to save lives.

Crowds sometimes choose inefficient ways to save lives.

A week ago, two climbers from Utah disappeared in a Pakistan mountain range. The two had already attempted to climb that particular peak last year, but a nearly-fatal accident prevented them from reaching the summit.

Since their disappearance, the internet has successfully crowdsourced over $100,000 to mount a rescue mission.

According to the impact calculator at The Life You Can Save, 100 grand could at the very least save dozens of lives if donated to the right charity. But we haven’t seemed to figure out how to make these causes feel quite as urgent as two blokes stop on top of a mountain.

Economists aren’t really supposed to judge people’s preferences, and it’s unlikely that the plight of the mountaineers is displacing money that otherwise could have been used to de-worm people or give them cash transfers. In fact, there is some (hopefully soon to be released) evidence that even big charity appeals lead to a net increase in people’s propensity to give. But it would be nice if that urgent, empathetic urge to give could be activated in the deeply impersonal world of effective altruism.

Help me test a (very silly) hypothesis by answering a few questions


I’ve held a silly hypothesis in my head ever since I was a grad student, but never had the time/resources to test it. I just recently came across a publication which drastically reduced the costs to testing the idea. It will almost certainly result in a “jokey” paper, but a fun one nonetheless.

But I could use your help. I have constructed an online survey displaying photos of people, and I need respondents to tell me whether these people are smiling, frowning or have neutral expressions. There are over 170 questions, but they are randomized, so even if you only manage to answer a few (and then close the window), it still would help a lot!

I can’t tell you what the idea is just yet, because it might spoil how you answer the questions. More information to follow, once enough people have answered the survey.

Click through here to enter the survey.

On Hirschman, measurement and RCTs


Every intervention is unique, perhaps unintentionally so.

Recently, Dan Honig of Johns Hopkins forwarded Ranil and me some thoughts he had in reaction to an Albert Hirschman on development projects that he felt was pertinent to the discussion on the pros and cons of RCTs. What followed is a discussion (rant) between Dan, Ranil and me. I’ve edited out the e-maily bits for clarity:


Matt, just after I hit send on this I realized I should have included you on this – I generally think you’re right on RCTs and the stale-ness of the conversation (and discussed this with Ranil a few months back some 2 days after you had dinner with him, hence the cc to Ranil) but feel like I’ve never seen this Hirschmann frame and wondering if it struck you as interesting. And yes, basically I’m trying to catalyze you writing something cool on this so I can quote/reference it down the road

Reading Hirschmann’s Development Projects Observed for the first time, and as I read it he’s with [Lant Pritchett and Michael Woolcock] on RCTs and causal density in international development projects. The quote below is from page 186 of the 1967 edition; italics are his, brackets mine; just before this he suggests we may not be able to identify good indicators of effects ex-ante and thus presumably couldn’t be pre-specified in a trial, meaning presumably we would be ill served by an RCT on a particular intervention even if we ignored external validity concerns.

“The indirect effects [of development projects] are so varied as to escape detection by one or even several criteria uniformly applied to all projects. Upon inspection, each project turns out to represent a unique constellation of experiences and consequences, of direct and indirect effects.”



Hey Dan, that’s a really interesting quote by Hirschman. If my interpretation is correct, it seems to be more damning for empirical evaluation in general than for RCTs in particular.

I’m not sure how I feel about this. Even if you move away from a simple, reduced form causal framework, Hirschman’s critique seems like it would apply. Even if development is a messy, complex thing that can’t really be boiled down in an impact evaluation framework, we still rely on measurement when we talk about development, and any given set of measurements is going to leave out things which might matter which are unmeasured. We can point at improving test scores but leave out student stress, etc, and the set of things that we leave out that might be important will change depending on the context. I guess I see this as a problem of measurement rather than as a problem for RCTs.

I also wonder what this means for how an empirical researcher operates. Over the last few years, I have become incredibly suspicious of surprising, counter-intuitive results, where a researcher measures something outside of the standard set of outcomes and finds a result. In a world of multiple hypothesis tests, expanding the set of outcomes to include as much of Hirschman’s unique constellation as possible will open up the door to a lot of false positives which will end up getting written up and published.

So that was a rant. Um, what do you think Ranil?

Continue reading

Come work for me in London or Washington D.C.


Like development? Like statistics and coding? Come join me and Vij Ramachandran at CGD.

The Center for Global Development, an independent, non-partisan research organization in Washington, DC and London, UK seeks a Research Assistant (RA) to support the work of Matt Collin and Vijaya Ramachandran. The successful candidate will have experience with research in economics, public policy, political science or a related field and will be based either in London or in Washington DC.  Applicants must demonstrate strong quantitative, analytical and communication skills. The position is well-suited to those who are considering doctoral study in the future.



Universal Basic Income: The Next Generation

"Wait, you're saying that in the Federation you don't have to worry about money? You can just schlep around the galaxy seducing alien women?"

“Wait, you’re saying that in the Federation you don’t have to worry about money? You can just schlep around the galaxy seducing alien women?”

Over at Five Thirty Eight, there’s a nice piece by Daniel Flowers on the idea of a universal basic income (UBI). Proponents say it will allow people to choose their careers and live their lives without having to worry about ever being poor. A common criticism is that it will create a massive disincentive to work at all. Several experiments have already been run which have found small, but non-negligible effects on the willingness to work. It is one of the outcomes that a host of new experiments of giving people a long term guaranteed basic income will test.

I am a little worried that these new experiments won’t capture the long term, generational impact of a universal basic income. Let’s imagine I really wanted to be a filmmaker (*cough*), but decided to become an economist because filmmaking is more likely to leave me in poverty. If I’m half way through my career as an economist and I start receiving a basic income, it might be too late for me to really break into filmmaking. Even if I pull it off, adjustment costs will be high, and it’s likely I’d end up embarking on a career which would be less successful than if I had started at a much earlier age. It is these decisions that will largely be picked up when the targets of a UBI experiment are largely adult workers.

What is more interesting is the impact on the next generation. Let’s imagine the UBI is introduced and governments can credibility commit to providing it for one’s entire lifetime. Now, all those aspiring filmmakers can select into the job of their choice with lower adjustment costs and a higher likelihood of actually being accomplished at what they’d most like to do. Who knows if this would have a net positive or negative impact on the creation of value, but it certainly would lead to better sorting. But even the most ambitious UBI experiments which are being proposed are unlikely to pick up these effects. Instead what we need to do is find a group of high schoolers and offer some of them (randomly) a credible lifetime UBI, then sit back and see how it affects career decisions and labour market participation in the long run.

As a side note – how much does relative poverty in developing countries lead to sub-optimal career decisions?

The road out of hell

In the FT yesterday, Branko Milanovic suggested that we might be able to increase global migration by reducing the citizenship rights of migrants. This is not a new idea – Lant Pritchett brought it up ten years ago and it is widely practiced by many Gulf states.

Over at Crooked Timber, Chris Bertram made it clear he really doesn’t like this idea, comparing Milanovic’s suggestion to “recreating apartheid” (I suspect the word apartheid will eventually be subject to its own form of Godwin’s law):

Milanovic wants us explicitly to abandon the liberal and democratic principles of legitimacy that those who are subject to the laws of a society should (in time in the case of migrants) get to have the right to make those laws. In doing so, he goes far beyond similar proposals (for example from Martin Ruhs that have been explicitly temporary in nature and have largely focused on labour-market rights. Milanovic’s lack of commitment to the norms of liberal democracy also comes across in the fact that he holds up illegitimate and tyrannical states, such as the Gulf kleptocracies, as models for his proposed policy.

Part of what’s going on here is the economist’s perspective on policy, which just focuses on net improvements in well-being or utility, with income serving as a proxy, and which doesn’t, therefore, see human beings as possessed of basic rights which it is impermissible to violate. Rather, all and any rights can be sacrificed on the altar of income improvement, just in case someone is poor and desperate enough to make a deal (who are we, paternalistically, to stop them?). The road to hell is paved with Pareto improvements.

Let’s be absolutely clear: we are already in hell and we are trying to find a path out of it. International migration restrictions – as they stand – already enforce a global system of apartheid. Most of global inequality in income (and likely in health and happiness) is driven mainly by where  you are born. By preventing someone mired in poverty overseas from moving to a place where they can make a better life for themselves – even temporarily – we are implicitly denying that person the same rights that we enjoy every day (rights that most have us have inherited, not earned). These are also arguments that Pritchett made before.

Human beings have a proximity problem: inequalities in outcomes or rights which are proximate to us (on the right side of an arbitrary national boundary) are weighted much higher that massive, gaping inequalities which are harder to observe because the people bearing the brunt of that inequality happen to live overseas.

We would all agree that a migration system which allows for restricted freedom is a worse solution than a system which allows for the same amount of migration with no restrictions on freedom. But the latter system does not exist, nor has anyone managed to propagate a convincing way to get there. I don’t know if a Milanovic/Pritchett system would work, but I can think of two main reasons why we might not want to consider it:

  1. There is a lower cost path towards a system which does not limit freedom that we can implement sooner.
  2. Adopting a system based on limited citizenship now will somehow make it harder to move to a free system later on.

If Bertram really wants to make a convincing case against the Milanovics of the world, he needs to start by showing us a better road out of hell.