Chris Blattman recently asked if Human Rights might be a morally dubious concept, following Adam Martin’s unearthing of an old interview with the libertarian economist F.A. Hayek. Hayek, who was making the argument that apartheid South Africa should have been left alone to run its own business as a ‘civilisation of a kind’, made the claim that intervention was being encouraged by the recent acceptance of human rights in America, just a few years prior to the video being recorded.

Leaving aside the irony of a libertarian regarding a situation in which 90% of humans had severely restricted liberty as unworthy of intervention, I really have to take issue with another point he makes: that human rights were a recent concept to the US when he was interviewed in the 1970s. Martin quotes:

I’m not sure whether it’s an invention of the present administration or whether it’s of an older date, but I suppose if you told an eighteen year old that human rights is a new discovery he wouldn’t believe it.  He would have thought the United States for 200 years has been committed to human rights, which of course would be absurd.

- An absurd statement itself, considering the contents of the Declaration of Independence. In the comments, Adam defends this statement, arguing that Hayek was referring to a new, modern conception of human rights:

… human rights are claims about how governments should act to produce particular patterns of outcomes

Adam contrasts this with the idea that classical human rights were statements about how individuals should treat each other. I’m dubious; a glance at the history of the ideas underlying statehood and rights suggests that the concept of rights was far more amorphous than this dichotomy would indicate, and spoke directly to the actions of the state as well as individuals. In fact the revolutions of America and France were specifically about the responsibilities of the state.

Human rights emerged as the basis of modern states (including the independent America) more than two centuries ago and their expansion from their original base began almost immediately – first incorporating new people and then new rights. The American and French Revolutions pioneered the ideas of ‘human rights’ as actionable concepts. The Declaration of Independence famously holds certain truths to be self-evident, notably that ‘all men are created equal’. France’s revolution was founded on the principles of liberté, egalité, fraternité – still the central elements of the French identity.

Nor were these the first time that universal rights had been invoked. The intellectual foundations for universal rights were laid by the very first Greek and Roman philosophers, and independently by Asian philosophers. These philosophers attempted to isolate the conditions for happiness, rightness and dignity that apply to all men: and thus laid the foundation for the idea of human rights as conditions that must be protected to allow the pursuit of these. As early as the third century BC, Mencius took these ideas further: he claimed that a ruler who tyrannises his subjects loses his divine right to rule and the people have the right to revolution – thus directly linking rights to the actions of states and leaders. The 1776 and 1789 revolutions made this explicit through the ideas they laid at the foundation of the new states they created.

What was amazing in the 1776 and 1789 revolutions was that they established as part of their basic justification the idea that Government had responsibilities and requirements that gave or denied it legitimacy, and that this was a truth that did not derive from exclusively from divine mandate (as with Kings who were either ‘chosen’ or descendent from Gods or the heavens) nor divine knowledge (i.e. from textual religion) – rather that these concepts were immutable, and derived from the condition of humanity. It was the failure to meet these responsibilities that motivated revolution.

The ideas of basic human rights established in these two revolutions have had an incredible enduring power. It is ahistorical to claim that the Sen-inspired approach to rights-based development is anything new. Rather, it’s a further modernization of a conceptual basis to the state that has existed for a great deal of time. The modification and expansion of rights is also not a new phenomenon. The original ideas were immediately passed into common currency and modified even in the few years following the revolutions. And these ideas have always been controversial and open to debate.

The first expansion was in who was covered by these rights. The concepts of the French Revolution were originally restricted to those white, free-born Frenchmen who powered and directed the revolutionary impulse in 1789. Yet within two years the first expansion of these emerging human rights occurred: in St Domingue, slaves took arm and rose under the leadership of Toussaint L’Ouverture, adopting the ideological framework of the French Revolution to their own struggle for freedom from the tyrannical plantation owners.

After an epic struggle, fighting off Napoleon’s armies, a British attempt to seize control of a new slave colony and a Spanish invasion, the free Republic of Haiti emerged. Within a few years, the British abolished slavery internationally and set about policing the newly expanded right of liberty for Africans. (Incidentally, Hayek’s justification for non-intervention in Apartheid South Africa would equally have applied to British attempts to police the end of the slave trade – the limits of libertarianism become quickly apparent in extreme circumstances). At the time, these expansions of human rights were profoundly controversial, shaking the very foundations of how the world was perceived in the West.

The second dimension of expansion was in the rights themselves. The original rights were limited to a basic legal equality and freedom; yet others argued that legal equality and nominal freedom meant very little when conditions of rule and economy denied vast swathes of humanity the ability to earn or harvest enough to avoid periodic starvation. This was considered to be a failure of equality of opportunity and a failure of freedom. This observation was taken in two different directions. On the one hand a new series of revolutions, starting in 1917 in Russia, created states with founding ideals that expanded on those of the French and American Revolutions. These new revolutions asserted that equality must be functional to eradicate the failures of the original rights in securing life. These attempts have either collapsed or been forced to re-examine their methods. The rights they sought to expand came at too great a cost in the other rights that had been established in 1776 and 1789. Yet for a while, these new rights were actively pursued through coercive state action.

A second approach to the same ends began to emerge a few decades later, alongside a development of a new vocabulary of rights. This is the new ‘creeping expansion’ of human rights which forms the basis of much current development discourse. Sen’s Entitlements Theory was essentially an attempt to expand the locus of human rights and give it a new vocabulary. In the modern world, we have been much more responsive to these ideas than we would have been in the 1700s. Back then, food insecurity was common for all but the most privileged, and any attempt to make material well-being a human right would have been utopianism. Now, though, many countries are almost completely free of food insecurity and in these places the attractiveness of labeling such things rights is massively increased: they are seen as an achievable basic condition of humanity.

These are arbitrary expansions and, like the previous expansions in the ideas of human rights, they’ve attracted controversy. Intellectual critics claim these rights are not immutable; but in a world conceived without a God, no concept is immutable because nothing is delivered to us from without the world. So immutable concepts are no longer feasible, unless we seek a religious world order. Other critiques are from the technical point of view – what do human rights achieve? What does it mean when we talk of them?

These arguments have been going on for 200 years; but what gives them such legs on the battleground of this newest expansion is that for the first time since the 18^th Century, rights are being coined without any basis in the coercive power of a state. When the French Revolutionary or Soviet states made something a right they would fight, discipline and punish until these rights were adhered to. There is no such possibility for these universal rights to a better life that development actors now champion. As a result, they are far weaker than the rights as first conceived. What it’s crucial to understand is that the rights themselves are not necessarily weak, but the context in which they are delivered is. To have power, they must be backed by coercion, not an organ of discussion like the United Nations.

Human Rights have only ever been political constructs; as such, they function when they are associated with powerful political entities. Debate about the usefulness of a rights-based approach to development needs to recognise this or risk redundancy.

Be careful of what you wish for. You might just get it.

Recently, Andrew Mwenda and five other prominent African intellectuals wrote to the Telegraph suggesting that Africa does not in fact need British development aid. Rather, they would be much happier if Britain contributed to the scrapping of the Common Agricultural Policy as a way of helping Africa.

Unfortunately, it seems like the Conservative-Liberal coalition Government might be giving them half of what they want – and not the good part. Yet another DfID-related leak has revealed that the British aid budget will from now on be allocated with a much stronger emphasis on UK security; in effect, moving aid away from being a stand-alone policy area and into a branch of a foreign policy drive aimed at ensuring the safety of the British public. Cynics will say this is nothing new, but it is surely more explicit and more closely felt than at any time since DfID’s formation.

Just to be clear: the leaked document does not suggest that Britain stop funding schools, or healthcare or even economic growth per se. DfID could continue to be a paragon of virtue in international development circles. What it does mean is that whenever DfID want to spend on these things, it will need to justify them on UK national security grounds. Since UK national security is best served by stable, prosperous, well-educated countries existing around the world this isn’t necessarily a recipe for disaster.

However, it’s another indication that the new Government want to make DfID, hitherto one of the best aid agencies to work with from a developing country point of view, more of a tool for an overall UK Government strategy founded in ideology and realpolitik. This is a real worry. Like the news from a few weeks back that DfID was dropping a number of commitments previously agreed, allegedly including the Paris Declaration, it is an indication that the Government wants to free up DfID to respond to its own priorities first and foremost.

Up til now, one of the reasons why DfID has developed such a good reputation was because it had a fairly high degree of operational independence from the rest of Government. This gave it the flexibility to pursue better aid allocations in the context of wider donor and Government spending, sometimes by taking on risk through budget support and other times by improving resource allocation procedures (budgeting, Parliamentary oversight and the like).

Giving DfID a requirement to justify what they do based on UK national security introduces an important restraint to them: it means that they cannot simply respond to country needs given the allocation of other resources, but needs to ensure it’s own resources pass a fitness test at home. What’s more, this all but rules out general budget support (from the recipient point of view, the best way of getting aid, if you care about building the ability of Government to allocate and account for funds), since there can be no guarantee on where this money will be spent.

All in all, this is a worrying sign though not a guarantee of catastrophe.

Despite the masses of negative publicity heaped on the continent by the famed Nigerian spam industry, Africa is actually one of the world’s safest places to go online in—featuring seven of the ten nations least attacked by malware.

Virus-checker company AVG surveyed 127 million computers in 144 countries and calculated the average rate of attacks—with the African nation of Sierra Leone emerging as the least assaulted, with only one virus event logged per 692 web users.

Really? My experience (admittedly primarily limited to the offices of Malawian government) is that you should treat any internet-capable computer south of the Sahara as a instant death.

Since internet and e-mail access on the continent tends to be a less reliable and more expensive, a lot of information transfer is done using memory sticks. Even if computers aren’t subjected to very many attacks from the outside,  it just takes one infected stick and a few marginally motivated employees to spread a virus to every other computer in the office. Many of these are the nasty, older viruses/trojans/worms which knock out the antivirus program’s ability to function, which means that AVG can’t see them.

This can happen astonishingly quickly. Tired of spending five minutes scanning my colleagues’ USB drives every time I wanted to get an Excel table from them, I once tried to quarantine and clear every computer in my department,  installing new (trial) antivirus on each cleared system. Unfortunately I missed a couple of computers, and within month when the trial software stopped working, the entire department had been reinfected.

It’s possible that AVG’s results are due to pretty extreme selection bias  on two fronts:

1. AVG users are probably a little more concerned and careful than those who don’t bother to update (as most don’t).
2. As I mentioned before, many attacks can knock out AVG, which means no reporting.
3. Many don’t bother to update AVG’s virus definitions, leaving the program incapable of detecting or reporting new viruses.

So yes, Africa might be a safe continent to go online by yourself in a locked room with tape over your USB drives, but any file-swapping outside the net should be handled with extreme caution.

Hat tip to Chris Blattman’s Google Reader shared items.

Where would people end up if there were no barriers to movement?

The folks at Gallup, who recently produced some interesting figures on the large number of people from developing countries who  would like to permanently emigrate, have followed up with new data on where people would like to move to.

Using their survey data to predict the proportion of the population who would move if all barriers were dropped, they constructed net migration indices, basically showing the increase/decrease in adult population which would result if everyone got their wish. For example, below we have the top gainers (in percentage terms):

The US and UK aren’t far behind with 62% and 60% predicted increases, respectively. The biggest losers aren’t terribly surprising:

The biggest weakness in these indices has to do with the assumptions about movement: each person responding to the survey is conditioning their choice on current conditions in the country they move to. So if I am one of the Haitians that has ‘voted’ to move to the US, my response is conditioned on the current US population, before anyone else moves there. My answer (or, in the aggregate, the probability of choosing the US) may change after a 60% foreign-born increase in the adult population .

While other research has shown that people are quite keen to move to countries with a larger co-national diaspora, it’s not clear that increased net migration from a third country will increase the chances I’ll choose to move somewhere (it’s more likely that my preferences would shift elsewhere).

Similarly, my decision to move at all might be affected, either positively or negatively, by the number of people who have left before me. Neither of these conditions is taken into account when Gallup constructs these estimates.

It would be very difficult to actually get at the equilibrium result – one would have to constantly condition the survey questions, and make some assumptions about strategic behaviour (i.e. where would I move if I thought X number of people from another country are moving there, and vice versa).

I’ve already written about how I think migration is a key part of improving welfare in developing countries. Ranil has rightly pointed out that it isn’t always analogous with development, as it isn’t clear that there are any direct benefits to the sender nation as a whole (although some work has shown that remittances can have some beneficial aggregate effects).

However, from an individual-welfare perspective it’s a possible win. The question we desperately need to be asking ourselves is what happens to welfare in the long run under our two scenarios: do we have a greater change at improving welfare by importing poverty, or exporting assistance? I’m not sure.

A Saudi judge has asked several hospitals whether they would punitively damage a man’s spinal cord after he was convicted of attacking another man with a cleaver and paralysing him, local newspapers reported today…

Abdul-Aziz al-Mutairi, 22, was left paralysed after a fight more than two years ago, and asked a judge to impose an equivalent punishment on his attacker under sharia law, reports said.

Fortunately:

King Faisal specialist hospital said that it would not do the operation. The article quoted a letter from the hospital saying “inflicting such harm is not possible”, apparently refusing on ethical grounds.

No doubt Wronging Rights will come up with something pithy and insightful to say about this, so I’ll sign off with this: gah.

Of all the Declarations, in all the world...

News from this weekend suggests that DfID will be reversing its hitherto strong backing to the Paris Declaration on Aid Effectiveness. My initial reactions were of shock and disappointment. Shock because DfID has been an ardent supporter of the Paris Declaration and Accra Agenda for Action. Disappointment because it was so unexpected: it has a strong, highly competent aid effectiveness department and has also used the Declaration to push Government reform.

I’ve noted after viewing the original leaked memo that the original advice was in favour of maintaining the Paris Declaration as a commitment by DfID. Most of the other commitments dropped simply serve to cut the amount of ringfencing of DfID’s budget and therefore increase its flexibility to meet the needs of different developing countries.

The decision to rescind their commitment to the PD is a much more problematic one, however. The issues essentially break down as follows:

What has DfID Reversed?

The Paris Declaration on Aid Effectiveness (PD) is an agreement signed by donor agencies and Governments and aid-recipient Governments in 2005. The Declaration establishes a number of best practices in aid management that all parties promise to adhere to, and twelve targets which all parties are to be assessed on. These targets and commitments were strengthened by the Accra Agenda for Action (AAA) in 2008.

The idea behind the PD and AAA is to make it easier for Governments to manage, use and report on aid by simplifying the way aid is contracted, disbursed and evaluated. It also seeks to maximise the benefit to the developing country by untying aid and ensuring that aid be channelled through the working local process of the aid-recipient Government. Thus aid is promised to be channelled through the local budget process, use the local accounting and audit procedures and be evaluated according to local processes. It further stressed the need to make aid as flexible as possible by using fungible General and Sector Budget Support.

Recipient Governments also made pledges to improve their own systems: of audit, budgeting and so on, and to be assessed independently on them.

The Paris Declaration has two very big positive points. The first is that it seeks to increase the ability of local actors to respond to their own problems flexibly and not be dictated to by a multitude of individual donors. It thus helps reduce the coordination problem of aid and encourages local solutions and visions of development.

The second major benefit, related to the first, is that it moves the lines of accountability of aid. Instead of aid money being handled by the donors, in which case the donors are accountable to their own taxpayers and no-one else, it creates dual accountability. First the donor gives money to the recipient Government to use. That Government is thus accountable to the donor, and must show that the money was used appropriately. But far more important than this, because aid money is now on budget and managed by local Governments a second line of accountability is created: of the recipient Government spending the money to the local electorate. Through the budget debates in Parliament, these people have the chance to contest the use of aid through their elected representatives; they also have the ability to vote a Government out of power if it doesn’t use aid money well. The Government now has to justify aid money in the same way it does tax money.

Additionally, the PD addresses lots of smaller, niggling issues that seriously hamper the capacity of Governments, for example setting a target for the reduction of cumbersome and time consuming donor missions by combining them.

The primary weakness of the PD is that it is actually not about the effectiveness of aid itself. At no point does the PD focus squarely on what aid achieves. The most it does is specify that each country *should* examine this. It focuses instead on processes of accountability which should stimulate effectiveness.

What DfID’s Reversal Does NOT Mean

Before going on to the criticisms that DfID opens itself up to by abandoning the Paris Declaration, it’s important to be clear about what DfID has not abandoned. It has not abandoned the impulse towards better, more effective aid. As I’ve pointed out above, the PD does not directly address or measure the effectiveness of aid. Thus, even without the PD, DfID is quite capable of pursuing the effectiveness of its aid portfolio.

DfID has also not abandoned any hope of coordination with other donors nor of making aid easier to manage. These can be addressed from without the framework the Paris and Accra agreements provide. Ideas like the marketization of aid are not covered in the PD.

What DfID’s Reversal Means for DfID

So why all of the hand-wringing? If DfID can continue to pursue more effective aid and coordinate with other donors in each country, why am I disappointed that it has rescinded its commitment to the Paris Declaration? DfID’s decision will cause itself a problem in its dealings with developing country Governments and will also hamper developing country Government efforts to get the most out of aid.

Firstly, there is the message it sends to developing countries. DfID made a multilateral commitment to take action and pursue targets for better aid provision and has unilaterally decided that this is no longer important. This is an appalling message to send to a plethora of developing countries which have made similar commitments to crucial reform processes that are painful, unpopular and against the interests of the ruling elite. Governments which rescind their backing to reform opaque budgeting and auditing systems can now point to DfID’s example and say they remain committed to ideals but no longer wish to tie themselves down on targets, deadlines or specific changes.

Closely allied to this problem is DfID’s loss of moral authority with developing countries. Up until now, DfID have been able to use their leadership on the PD agenda as a lever to push change in developing countries. They could push reforms when they themselves had reformed and were willing to take risks and use recipient-country systems. When local Governments show reluctance to improve their accounting or withdraw support for an independent Auditor General, DfID’s protests are now likely to be met by a response of: ‘What do you care how systems are reformed? Renew your commitment to using them before you start to tell us how to manage them.’ Even worse, the unspoken anger at donor hypocrisy, so tangible in every developing country Government I’ve worked in or advised, will only be strengthened.

More than a diffuse moral authority that comes with the commitment to use Government systems, DfID will also lose the ability to make legitimate threats or remind recipient Governments of their requirements under the PD. The PD actually makes it explicit that the amount of aid that the donor is required to funnel through a Government depends on how good their public financial management systems are, as independently assessed. This carrot and stick are now removed from DfID’s armoury.

What DfID’s Reversal Means for Developing Countries

On the recipient Government side, the defection from the aid coordination game by DfID has different potential problems. The PD is a very powerful document for developing countries because it ties all donors to a set of targets which are transparently and publicly assessed. Developing country Governments can use the PD to push reform through the targets and the commitments made. This is an important tool. Most donors wanted to do well on these targets, either due to central directives or sheer competitiveness. Governments have been able to use this impulse to drive them to greater use of Government systems, strengthening the budget process and hence creating contestability and local accountability in aid. These are significant gains where they have been made.

The reversal of DfID’s support to the PD has two major consequences. Firstly, it provides other donors with an ‘out’, particularly those donors who find it difficult to meet PD targets due to their own institutional strictures. Secondly, it removes a front-runner from the pack, leaving the rest with less of a catch-up distance when the PD is assessed.

What’s more DfID’s support in pushing other donors to improve their practices in line with the PD has been a major driver of change in many countries – one which they now lose some of their capacity for. Without adhering to the PD themselves, it’s difficult to see how they can continue to exert pressure on other donors without backlash.

Finally, technocrats in developing country Governments have been able to use the PD and donor pressure to fulfil the terms of it to push domestic reform processes by demonstrating to political leaders the importance of them to increase aid allocations and reorganise the types of aid received. DfID was again a powerful voice in this, but has lost some authority.

Of course, one donor defecting from the agreement hasn’t at a stroke reduced it to a mess. But it sets a bad precedent and removes on of the strongest proponents of it. It’s also true that the PD hasn’t resulted in gains everywhere. But where the domestic Government has made an effort with it, it has been useful, and this shouldn’t be overshadowed by other places where efforts have been weak.

Reform of aid and developing country processes is painful, slow and difficult. We don’t really need anything to make it more so.

DfID have gone all Anderson Silva on the fight for more effective aid management.

A leaked memo reveals that DfID will be dropping its commitment to the Paris Declaration on Aid Effectiveness.

This is a massive blow. The PD (as it’s known)  is very imperfect, and even the refinements we made in Accra in 2008 left plenty to be desired. But it’s the only real commitment the international community has made to improving donor systems for the management of aid – to making it easier to use, receive, negotiate. What’s worse, it’s one of the few places where recipient Governments are tied down to improvements in the way they themselves manage aid and their domestic resources.

DfID have been one of the biggest motors behind improving the PD and getting the simplification of access to and usage of aid money improved. This is not insignificant. Anyone who has spent time in a developing country Government can see how much of the recipient Government’s time is spent on managing, applying for and reporting on aid – not to mention following up on problems in its access, flow and predictability, all of which are covered by the PD. A conservative estimate for a heavily aid dependent country like Malawi is about 60% of Ministry of Finance time. Probably as much in the most aid dependent sectors, too. (To clarify – dropping the PD does not mean that DfID are abandoning the fight for better aid – but they are dropping their biggest weapon in the fight for better aid management.)

Dropping the PD means DfID have just lost a massive amount of moral authority in the fight to improve the way aid is used, and equally in the fight to improve the way Governments manage their own resources.

I’ll collect my thoughts for a more detailed post.

A paragon of transparency.

Today’s Guardian runs an eye-opening piece on expenditures made in the last year of the Labour administration. The Tories are, in their drive for transparency, publishing expenditures made by various Government departments online. The documents relating to the Department of Communities and Local Government show:

Among the expenses revealed was £1,673 to a company called Stress Angels, which offers massages, acupressure, Indian head massage and reflexology…

Then there was £626 on a trip to a nature reserve in Nottingham and £539 on an awayday to Blackpool Pleasure Beach. Accommodation at a hotel – the Rubens, opposite Buckingham Palace – cost £17,000. Another £3,670 went to Halfords cycle shop.

The litmus test for the Tories will, of course, be whether they maintain their drive for transparency when it is going to expose even their own Government.  We see this kind of thing happening all the time in Africa, relating to corruption. A new Government comes in promising change and a war against graft. For the first year they push hard to identify and punish culprits, making high profile arrests and prosecutions. These arrests and prosecutions damage the previous administration, normally the opposition party in Parliament.

Then as time goes on, the anti-corruption agency exhausts its ability to prosecute the opposition. It’s eyes turn towards current or recent corruption scandals – those that implicate the current regime. Suddenly, the political will dissipates – they’ve ‘done enough to show that corruption will not be tolerated’. Quietly, the support and direction of senior officials is withdrawn. The old bad habits reassert themselves and the Government continues to make merry with public funds.

Eventually they get voted out, and the whole cycle starts again. This happened in Kenya (though it all went a bit pear shaped when John Githongo showed the tenacity of a bull-terrier); it happened in Malawi and in almost identical circumstances, in Zambia.

It’s easy right now for the Tories to attack the culture of expenditure in Government, because the punches are landing on their opponents. The real win will be when they let the expenditures be published on a monthly or quarterly basis, and let the whip fall on themselves. After all, this is what we demand of developing country administrations. Why should the standards we hold for ourselves be different?

I was 16 when I first saw a slum tour. I was outside my 100-square-foot house washing dishes, looking at the utensils with longing because I hadn’t eaten in two days. Suddenly a white woman was taking my picture. I felt like a tiger in a cage. Before I could say anything, she had moved on.

On the educational value of these trips:

To be fair, many foreigners come to the slums wanting to understand poverty, and they leave with what they believe is a better grasp of our desperately poor conditions. The expectation, among the visitors and the tour organizers, is that the experience may lead the tourists to action once they get home.

But it’s just as likely that a tour will come to nothing. After all, looking at conditions like those in Kibera is overwhelming, and I imagine many visitors think that merely bearing witness to such poverty is enough.

A few months ago Ravi Kanbur wrote an interesting paper suggesting that development workers should have to go on routine ‘exposure’ trips, where they spend a few days staying in a rural village to get a better perspective on poverty. Several others thought this would be a good idea, but I remain concerned that this would be nothing more than a glorified poverty safari, akin to earning a merit badge in the Boy Scouts.

The very first post on this blog was on poverty safaris. What do you think of them?

Hat tip to Aid Watch for the link.

In fact, Peirce and others in the social sciences invented the RFT decades before the technique was widely used for therapeutics. By the 1930s, dozens of American universities offered courses in experimental sociology, and the English-speaking world soon saw a flowering of large-scale randomized social experiments and the widely expressed confidence that these experiments would resolve public policy debates. RFTs from the late 1960s through the early 1980s often attempted to evaluate entirely new programs or large-scale changes to existing ones, considering such topics as the negative income tax, employment programs, housing allowances, and health insurance.

So the randomistas aren’t so much as a “new wave” as the “next wave.” More interesting though, are Manzi’s thoughts on external validity:

By about a quarter-century ago, however, it had become obvious to sophisticated experimentalists that the idea that we could settle a given policy debate with a sufficiently robust experiment was naive. The reason had to do with generalization, which is the Achilles’ heel of any experiment, whether randomized or not. In medicine, for example, what we really know from a given clinical trial is that this particular list of patients who received this exact treatment delivered in these specific clinics on these dates by these doctors had these outcomes, as compared with a specific control group. But when we want to use the trial’s results to guide future action, we must generalize them into a reliable predictive rule for as-yet-unseen situations. Even if the experiment was correctly executed, how do we know that our generalization is correct?

One example he discusses the frequent experimentation used in crime-prevention, and how the (very few) subsequent attempts:

Criminologists at the University of Cambridge have done the yeoman’s work of cataloging all 122 known criminology RFTs with at least 100 test subjects executed between 1957 and 2004. By my count, about 20 percent of these demonstrated positive results—that is, a statistically significant reduction in crime for the test group versus the control group. That may sound reasonably encouraging at first. But only four of the programs that showed encouraging results in the initial RFT were then formally replicated by independent research groups. All failed to show consistent positive results.

My biggest fear about the current trend in social science RCT work is not only the failure to confirm positive results, but the failure to confirm negative results. While there is a small, but real incentive to repeat a ‘proven’ randomized study in a new setting, there isn’t much being done to confirm that a negligible treatment effect doesn’t improve elsewhere. While big RCT research groups do care about external validity, it is the initial findings that get seared into the mind of the policymakers. Flashy graphs which generalize without concern don’t help.

Here’s part of the closing to Manzi’s piece, which is a must-read if you’re interested or involved in this type of work:

It is tempting to argue that we are at the beginning of an experimental revolution in social science that will ultimately lead to unimaginable discoveries. But we should be skeptical of that argument. The experimental revolution is like a huge wave that has lost power as it has moved through topics of increasing complexity. Physics was entirely transformed. Therapeutic biology had higher causal density, but it could often rely on the assumption of uniform biological response to generalize findings reliably from randomized trials. The even higher causal densities in social sciences make generalization from even properly randomized experiments hazardous. It would likely require the reduction of social science to biology to accomplish a true revolution in our understanding of human society—and that remains, as yet, beyond the grasp of science.