The problem with nudges is that sometimes they don’t move things very much

Have you ever prescribed azithromycin when you didn't have to? Know what I mean?

Have you ever prescribed azithromycin when you didn’t have to? Know what I mean?

Over-prescribing of antibiotics is a problem because it speeds up the rate at which bacteria develop resistance. In a new study was published in the Lancet yesterday, researchers attempted to use a simple `nudge’ to get doctors in the UK to prescribe less often:

In this randomised, 2 × 2 factorial trial, publicly available databases were used to identify GP practices whose prescribing rate for antibiotics was in the top 20% for their National Health Service (NHS) Local Area Team. Eligible practices were randomly assigned (1:1) into two groups by computer-generated allocation sequence, stratified by NHS Local Area Team. Participants, but not investigators, were blinded to group assignment. On Sept 29, 2014, every GP in the feedback intervention group was sent a letter from England’s Chief Medical Officer and a leaflet on antibiotics for use with patients. The letter stated that the practice was prescribing antibiotics at a higher rate than 80% of practices in its NHS Local Area Team. GPs in the control group received no communication. The sample was re-randomised into two groups, and in December, 2014, GP practices were either sent patient-focused information that promoted reduced use of antibiotics or received no communication. The primary outcome measure was the rate of antibiotic items dispensed per 1000 weighted population, controlling for past prescribing. Analysis was by intention to treat.

This is a fairly standard behavioural intervention – use information (or, less graciously, spam) to nudge people into behaving in a more optimal way. The behavioural insights/economics crowd loves these interventions because they are cheap, so the cost-effectiveness hurdle is easy to overcome. However, that cheapness sometimes overshadows a bigger problem, that frequently these interventions just don’t have very large effects. Here are the results from the Lancet study:

Between Sept 8 and Sept 26, 2014, we recruited and assigned 1581 GP practices to feedback intervention (n=791) or control (n=790) groups. Letters were sent to 3227 GPs in the intervention group. Between October, 2014, and March, 2015, the rate of antibiotic items dispensed per 1000 population was 126·98 (95% CI 125·68–128·27) in the feedback intervention group and 131·25 (130·33–132·16) in the control group, a difference of 4·27 (3·3%; incidence rate ratio [IRR] 0·967 [95% CI 0·957–0·977]; p<0·0001), representing an estimated 73 406 fewer antibiotic items dispensed. In December, 2014, GP practices were re-assigned to patient-focused intervention (n=777) or control (n=804) groups. The patient-focused intervention did not significantly affect the primary outcome measure between December, 2014, and March, 2015 (antibiotic items dispensed per 1000 population: 135·00 [95% CI 133·77–136·22] in the patient-focused intervention group and 133·98 [133·06–134·90] in the control group; IRR for difference between groups 1·01, 95% CI 1·00–1·02; p=0·105).

Let’s focus on the intervention that worked: the peer information treatment. There was a clear decline in antibiotic use for the treatment group, and so the study focuses on the sheer number of prescriptions that were prevented (73,406). However, in terms of relative impact, the study barely changed behaviour. The treatment group’s prescription rate was a mere 3% lower than the control group’s rate. 

So if this is about finding cost effective ways to reduce prescribing, then I’m on board. But clearly these sort of nudges are not going to win the war on antibacterial resistance any time soon.

The ultimate Monopoly strategy and the British housing shortage

monopoly

Over at imgur, someone has discovered a Monopoly strategy that will not only let you win, but will also make the other players hate you:

A little-known rule of Monopoly is that the game has exactly 32 houses and 12 hotels. Once you run out of houses, no more can be purchased until they re-enter the supply by being sold or upgraded to hotels. If there are more players who want to build houses than there are houses available, they are auctioned off to the highest bidder, one at a time. The core of this strategy is to buy up as many houses as possible before anyone realizes what you’re doing, and DO NOT UPGRADE TO HOTELS to prevent people from improving their own properties.

Basically, you need to create several monopolies very quick and just focus on buying up all the available houses the game provides. Once there are no more houses, your friends can’t advance and so you’ll win, slowly and painfully.

Everyone suffers in this scenario, except for the lucky person who rushed their way onto the housing ladder. But note that the reason this strategy works and the first-mover will eat up all the resulting rent is because Monopoly has a purely arbitrary rule on the supply of housing. This is not a million miles away from the unnecessary planning restrictions which make it harder to build or upgrade in the UK and thus make land and housing far more expensive than they need to be.

Hate tip to kottke.org

So how do you feel about not winning the lottery?

"Here's to exogenous shocks to our neighbour's wealth"

“Here’s to exogenous shocks to our neighbour’s wealth”

Happy New Year. So I’ve been thinking a lot about the charity GiveDirectly recently. They were my charity of choice a year ago and I am planning to make another donation soon. For those of you who are not in the know, GiveDirectly makes unconditional cash transfers to poor people in Kenya and Uganda. For every dollar I donate, roughly 91 cents of that ends up with a household, which is then free to do whatever they want with it.

The other day GiveDirectly sent me an e-mail which linked to a series of interviews with residents of a single village that had been on the receiving end of these unconditional transfers. What is particularly astonishing is that the charity not only asked recipients how they were faring (pretty good, thank you very much), but roughly half of the interviews are with households which were not deemed eligible.

What I might have expected was a degree of unhappiness or animosity over not being selected to receive a $1000 USD transfer. GiveDirectly uses its own methods of determining whether or not a household is classified as “poor” (in the village in question it was households without a metal roof on their primarily residence). Even though (I presume) the charity goes through great pains to make the selection criteria transparent, to people on the ground the whole endeavour might seem a bit, well, random. A bit like a manna from heaven.

Recently, three academics who have previously studied GiveDirectly released a paper suggesting that these transfers do have some sort of negative spillovers on households that didn’t receive the transfer. Johannes Haushofer, Jeremy Shapiro and James Reisinger found that non-recipients in villages which received GiveDirectly transfers reported substantially lower levels of life satisfaction. So if this negative spillover, which I will go ahead and call the Haushofer Effect (there – I just branded it – coming to a book store near you), really exists, then I would expect a substantial amount of lamentation in the GiveDirectly interviews of non-recipients.

To the contrary, most non-recipients said that, overall, they were happy that their village had received the transfers. I found this hard to believe, but after going through 50 interviews of non-recipients, most replied positively to the question “Are you, overall, happy that GiveDirectly came to your village?,” a handful replied neutrally, and only one was vocally unhappy about it. There was another question aimed more at the negative effects of not being selected, and even then only about 25% responded with identifiably-negative comments.

So what is going on here? Why is the Haushofer Effect not appearing in these qualitative interviews? As much as I would like to believe that people do feel happy about seeing their neighbours get a shitload of money, I think I am more likely to believe one of the following:

(1) People don’t want to appear selfish, especially in front of a charity which might might might might give them a ton of cash some day. One respondent actually spelled it out: “”I am happy with your coming with the hope that one day I will also benefit.”

(2) The more complicated answer is that there is something about the conditionality of the question that changes its meaning. These families might be honestly happy about the fact that their neighbours (who are poorer) got transfers. But all the negative externalities associated with that (envy, local prices, etc) still make them unhappy in aggregate. A great example of this appeared in a recent episode of This American Life, where Neil Drummond tried to reconcile the fact that he really was happy his old friend Ta-Nehisi Coates had found fame and fortune with the reality that their friendship was slowly dissolving as a result of it.

(3) This village is different than the average village in the study above in some unobservable way.

 

I have no sense as to which answer is the most likely. And none of it will stop me from donating to GiveDirectly again. That said, while the charity should be praised for putting these interviews up on their website, they could take a step further and link to the paper on negative spillovers.

 

Update: GiveDirectly’s Max Chapnick has a helpful reply/explanation in the comments below, rightly pointing out that the academic paper I cited relies on within-village randomization (rather than GD’s method of targeting poor households), so the Haushofer effect might be primarily driven by the unfairness inherent in that lottery mechanism. This is a pretty plausible reason for the differences between the empirical study and the informal interviews.

The Twelve Days of Christmas (Aid Edition)™

It’s an oldy but a goody, and the only thing I always  repost. Go on, sing along:

On the twelfth day of Christmas my donors gave to me

twelve delayed disbursements!

eleven sketchy studies

ten consultants calling

nine economists arguing

eight mission meetings

seven worthless workshops

six gender trainings

five RCTs!

four 4x4s

three acronyms

two empty schools

and a lecture on M&E!

Still on sale. And I promise this won’t be the only post I write for another six months.

The most trolling, self-aware Nigerian e-mail scam ever

I spotted this on Joe Wein’s extensive 419 scam repository. Spamming, in the name of Nigeria’s reputation:

Dear Sir/Madam,

Compliments of the day.

We hope this email would find you in the best of health and spirits.

You are reading an e-mail from JUDICIARY OF ENGLAND AND WALES, UK. On a daily basis, the Judicial Office coordinates and transmits requests for investigative and humanitarian assistance. The preponderance of economic and financial crimes like Advance Fee Fraud, Money Laundering and Terrorist activities all over the world, etc has had severe negative consequences all over the world, including decreased Foreign Direct Investments and tainting of world’s national image.

The menace of these crimes and the recognition of the magnitude and gravity of the situation led to the signing of Memorandum of Understanding on Friday 21st March, 2014 between British Government, United States Government, United Nations, Australian Government, Canada Government and Nigeria Government at the United Nations Headquarters located at New York City, USA. It was agreed that to retain the good image of Nigeria and the rest of Africa countries, all the scam victims who lost his/her hard earn money to these faceless thieves will be compensated with just US$250,000.00 (Two hundred and fifty thousands United Stated Dollars only) to avoid sanctioning Nigeria and some Africa Countries.

To that effect, we are sending you this e-mail because your contact details were given to us as one of the victims. You will receive your compensation payment through ATM SMART CARD which is the simplest way to transfer huge amount of money to avoid transfer charges or any further delay.

The delivery of your ATM SMART CARD to your provided address via CHRONOPOST COURIER will cost you US$100 only. We have signed a contract with CHRONOPOST COURIER Company for the delivery of all the ATM SMART CARD which should expired June 30th, 2014.

I WANT YOU TO READ BELOW CAREFULLY, THE NOTICE BELOW STAND AS CAUTION BEFORE IT IS TOO LATE.

MR. CHARLES WILLIAM ROME presented an Authorization Letter for change of your data that you are dead one month ago. After the investigations however,it was revealed that there are some dubius Banks and Government Officials in Nigeria and the rest of Africa countries who are collaborating with some Foreigners to make these changes illegally without the knowledge of the Bona-fide Benefactors and one traced to your own change is this MR. CHARLES WILLIAM ROME of United States of America, who said you are dead, He have also forwarded his Name and Address below as the new Address that will receive this money.

Name: MR. CHARLES WILLIAM ROME
Address: 139 Chelmsford DR Aurora, Ohio 44202, United States

But we wanted to confirm if actually this is true and hence decided to write to your email address which from now and there is no response from you, We will then know that you are dead indeed and the Compensation payment of US$250,000.00 will be transfer to him.

IF PROVED OTHERWISE BY YOU THAT YOU ARE NOT DEAD, PLEASE, FILL THE Claimant Form below and SEND IT TO HON. MINISTER OF FINANCE, NIGERIA (ATTENTION DR. NGOZI OKONJO-IWEALA) IMMEDIATELY. SHE IS THE PROCESSING AND PAYING OFFICER.

CLAIMANT FORM:-
1.YOUR FULL NAME
2. GENDER:
3. HOUSE OR OFFICE ADDRESS (P. O. Box not accepted)
4. YOUR PHONE NUMBER(S)
5. COUNTRY:

NAME: ATTN: DR. NGOZI OKONJO-IWEALA
E-mail: fedministryoffinance_department@yahoo.fr

Immediately you send the above required information to DR. NGOZI OKONJO-IWEALA, she will proceed with the Processing/Releasing of your ATM SMART CARD to you within 2-3 working days based on our agreement.

PLEASE NOTE: YOU WILL BE REQUIRED TO PAY US$100 ONLY TO CHRONOPOST COURIER WHICH IS THE COST OF THE DELIVERY OF YOUR ATM SMART CARD TO YOUR PROVIDED ADDRESS. (YOU SHOULD NOT PAY ANY OTHER MONEY FOR ANY REASON(S) WHATSOEVER).

IMPORTANT: UNDER PENALTY OF LAW, THE INFORMATION YOU SUBMITTED TO DR. NGOZI OKONJO-IWEALA CORRECTLY IDENTIFY YOU AS THE RECIPIENT OF THIS PAYMENT; NO OTHER LIVING PERSON OR ENTITY IS ENTITLED TO ANY PART OF THIS PAYMENT; IT IS A VIOLATION OF LAW FOR ANY PERSON TO INTENTIONALLY OR KNOWINGLY FILLING FOR DOUBLE CLAIMING OR AID ANOTHER PERSON IN CLAIMING THE SAME FUND, BY MEANS OF FRAUD OR DECEIT. BE WARNED

We await your urgent reply.

Yours faithfully,

The Lord John Thomas of Cwmgiedd,
Lord Chief Justice of England and Wales

A randomista for hire is a dangerous thing

Our research shows that the treated (caged) group was 30% more likely to return home than the non-caged group.

Our research shows that the treated (caged) group was 30% more likely to return home than the control (non-caged) group.

The Behavioural Insights Team is a research unit made up of randomistas who prefer to rely on behavioural economics and psychology to develop and test `nudges’ to achieve certain policy goals. They originally grew out of the Cabinet Office, but eventually went private (the CO has retained a stake in the BIT).

I was always excited by the mere existence of the Behavioural Insights Team – this was the first clear example of government investing in rigorous randomisation to test some of its policies.

That said, while the BIT likely comprises a group of people who want to make the world a better place, they are beholden to their clients. One of these clients is the Home Office, which is currently paying the BIT to find ways to convince illegal migrants to voluntarily leave the UK. From the BIT’s update report:

Increasing voluntary departures of illegal migrants

BIT has been working with the Home Office to consider new measures to help illegal migrants to voluntarily return home, focusing initially on engagement at reporting centres. Reporting centres are seen as an important but underutilised opportunity to prompt illegal migrants to consider whether leaving the UK voluntarily would be a preferable option in their circumstances.

Starting in December 2014, BIT undertook a short piece of ethnographic research at reporting centres across London, reviewing current procedures and interaction points to gain an understanding of the reporting centre experience from the perspective of a member of the reporting population and the reporting agent.

Informed by this, BIT developed several options for Home Office consideration to employ behaviourally informed trials in reporting centres that could encourage higher numbers of voluntary departures from the UK.

At this stage, the precise scope of a trial is still being finalised, with the aim to combine a number of behavioural elements to create a distinct reporting centre experience that encourages members of the reporting population to consider voluntary departure as an alternative to their current situation.

Note that many people who end up in reporting centres are asylum seekers, not just illegal `economic’ migrants. The BIT has another project in the pipeline aimed at targeting business who hire illegal migrants, with a similar end goal of convincing the migrants to voluntarily go home. The Home Office got a lot of push back from trying this before, in the not-too-subtle form of a van driving around telling migrants to go home:

tory-van-620_2628143b

So now the UK government has turned to more insidious methods, aided by a team of randomistas. It’s useful reminder that rigorous, evidence-based policy can be used for stupid, short-sighted policy as well.

 

*Disclaimer: I once applied to work at the BIT, but dropped out midway through the selection process to work on a project in Oxford.

How I Learned to Stop Worrying and Love the Bomb

bomb

Orbital Mechanics has created an incredibly creepy, yet riveting video showing every single nuclear detonation from 1945 onward.

 

You would be excused if your first reaction is “holy shit, we’re absolutely bonkers.” That was my first reaction.

My second thought was: “we were lucky that, aside from the obvious first two bombs, we were relatively lucky in that nothing ever went wrong.” But I was wrong to think that – nuclear testing led to long lasting, negative effects on the cognition of people who were exposed to the radiation while in utero. Sandra Black and co. have a really interesting paper examining the effect on Norwegians (apparently due to atmospheric conditions, a large hunk of radiation form nuclear testing landed in Norway):

Research increasingly shows that differences in endowments at birth need not be genetic but instead are influenced by environmental factors while the fetus is in the womb. In addition, these differences may persist well beyond childhood. In this paper, we study one such environmental factor – exposure to radiation –that affects individuals across the socio-economic spectrum. We use variation in radioactive exposure throughout Norway in the 1950s and early 60s, resulting from the abundance of nuclear weapon testing during that time period, to examine the effect of nuclear exposure in utero on outcomes such as IQ scores, education, earnings, and adult height. Importantly, we are able to examine the effects of exposure each month in utero to determine the periods when exposure is most harmful. We find that exposure to low-dose nuclear radiation, specifically during months 3 and 4 in utero, leads to a decline in IQ scores of men aged 18. Moreover, radiation exposure leads to declines in education attainment, high school completion, and earnings among men and women. We are also able to examine whether these effects persist across a second generation – we find that the children of persons affected in utero also have lower cognitive scores, suggesting a persistent effect of the shock to endowments. Given the lack of awareness about nuclear testing in Norway at this time, our estimates are likely to be unaffected by avoidance behavior or stress effects. These results are robust to the choice of specification and the inclusion of sibling fixed effects.

Hat tip to Kottke.

 

The IMF, inequality and the trickle-down of empirical research

"It took so many assumptions to put you together!"

“It took so many assumptions to put you together!”

By Nicolas Van de Sijpe

recent IMF staff discussion note has received a lot of attention for claiming that a smaller income share of the poor lowers economic growth (see also here and here). This piece in the FT is fairly typical, arguing that the paper “establishes a direct link between how income is distributed and national growth.”

It quotes Nicolas Mombrial, head of Oxfam International’s office in Washington DC, saying that (my emphasis): “the IMF proves that making the rich richer does not work for growth, while focusing on the poor and the middle class does” and that “the IMF has shown that `trickle down’ economics is dead; you cannot rely on the spoils of the extremely wealthy to benefit the rest of us.”

The aim of this blog post is to clarify that the results in Table 1 of  the paper, which are based on system GMM estimation, rely on assumptions that are not spelled out explicitly and whose validity is therefore very difficult to assess. In not reporting this and other relevant information, the paper’s application of system GMM falls short of current best practices. As a result, without this additional information, I would be wary to update my prior on the effect of inequality on growth based on the new results reported in this paper.

The paper attempts to establish the causal effect of various income quintiles (the share of income accruing to the bottom 20%, the next 20% etc.) on economic growth. It finds that a country will grow faster if the share of income held by the bottom three quintiles increases. In contrast, a higher income share for the richest 20% reduces growth. As you can imagine, establishing such a causal effect is difficult: growth might affect how income is distributed, and numerous other variables (openness to trade, institutions, policy choices…) might affect both growth and the distribution of income. Clearly, this implies that any association found between the income distribution and growth might reflect things other than just the causal effect of the former on the latter.

To try to get around this problem, the authors use a system GMM estimator. This estimator consists of (i) differenced equations where the changes in the variables are instrumented by their lagged levels and (ii) equations in levels where the levels of variables are instrumented by their lagged differences (Bond, 2002, is an excellent introduction). Roughly speaking, the hope is that these lagged levels and differences isolate bits of variation in income share quintiles that are not affected by growth or any of the omitted variables. These bits of variation can then be used to identify the causal effect of the income distribution on growth. The problem with the IMF paper is that it does not tell you exactly which lagged levels and differences it uses as instruments, making it hard for readers to assess how plausible it is that the paper has identified a causal effects.

Continue reading

Guns don’t kill people

commando

Think of all the cultural reasons I am wielding this rocket launcher.

Another shooting happens. This time in Charleston. I grew up about two hours away and often visited the town with my parents. It’s a lovely place, although like most places in South Carolina it has a difficult, disturbing past.

There are two opposing views which typically surface after  a mass shooting. The first is that gun violence is driven by gun ownership, and that effective gun control will reduce the number of people killed by firearms every year.  A simple mathematical way of describing this relationship would be to say that gun violence is a function of the number of guns in a country:

V = F(G)

The opposing view is that there are all sorts of other things that determine gun violence. Proponents look to countries with high levels of gun ownership but low levels of violence, such as Canada. Holders of this view assert a relationship that looks like this:

V = F(S)

Where S is “other stuff” which influences gun violence. This is somewhat consistent with the  “guns don’t kill people, people kill people” argument, which includes the unstated third statement: “and there are lots of things that determine whether people want to kill each other.”

Setting aside any preoccupations with the Second Amendment, the gun control debate can be characterized as a fight over whether V = F(G) or V = F(S). But this is a mischaracterization which gives more legitimacy to those opposed to gun control. In reality, gun violence is a function of both the number of guns in circulation and all the “other stuff,” and that, by construction, fewer guns makes it more difficult to commit gun violence, so that.

V = F(G,S) and V = 0 if G = 0 or S=0

That is: it doesn’t matter if Canada can have its cake and eat it. If there is some special ingredient to having guns without the violence (S = 0), we don’t know what it is, and won’t know any time soon. But that doesn’t mean that reducing G will not reduce violence. Whether it is a cost-effective way to reduce violence is another question, but unless someone identifies what goes into S, the best bet is for the US to focus on G.