When an RCT would have been really handy

The BBC reports on a study by two psychologists, purporting that staying hydrated can improve grades:

Students who bring water into the examination hall may improve their grades, a study of 447 people found.

Controlling for ability from previous coursework results, researchers found those with water scored an average of 5% higher than those without.

The study, from the universities of East London and Westminster, also noted that older students were more likely to bring in water to exam halls

I don’t believe an RCT is needed to answer every question out there, but it is a little silly in instances like this where a simple intervention could test the same hypothesis: just hand out water bottles to a random group of students before an exam, and see who performs better.

Surely, even controlling for ability (lagged dependent variable, anyone?) students who choose to bring water into exams might be different in some unobservable way. Of course, this doesn’t stop the researchers from making policy recommendations.

One thought on “When an RCT would have been really handy

  1. Ranil Dissanayake

    April 18, 2012 at 2:30pm

    Also, how big a difference is 5%? It’s not a 5 percentage point difference, but a 5% difference. What’s 5% of the average score? Not much, I’d imagine.

    And what’s the ultimate aim? Say we give everyone a bottle of water. If there’s a grade curve, then there’s no change in the grading.

    If there’s no grade curve, everyone does slightly better in the exams, because they’re better hydrated. I’m not sure why this is a good thing. It doesn’t mean anyone’s gotten any better at their subject. It doesn’t signal much to a university / employer (it may even have the opposite effect).

    If anything, I think we might want more variation in scores, especially in the top tier, because there’s not much to choose from any more. Policy should focus on 1) better teaching, better uptake of good teaching; and then 2) exams that better test the things we’re interested in.

    Exam results are not as important systematically as they are for individuals GIVEN a distribution of scores of other students.

Comments are closed.