Open Science

Over the last week, I’ve come across a couple new ‘science experiments’ - that is, experiments in improving science by encouraging the involvement of the online public. The first one, Petri Dish, is basically a Kickstarter for science projects:

Right now, it’s skewed pretty heavily towards ecology and animal behavior. Which makes me curious about how different scientific fields have embraced online innovation in different ways. For instance, there’s arXiv, which provides open access to articles in a handful of fields, including nearly all articles published in most subfields of mathematics and physics. However many other fields hide the bulk of their research away in closed access journals which require $10-20 per article.

Then there’s crowdsourced science projects like FoldIt and GalaxyZoo, which allow the public to participate in large scale protein synthesis and astronomical classification experiments, respectively. These may just be pioneers at the forefront of a wave of crowdsourced experiments in all fields, but so far it doesn’t look like it. So why these projects, in these fields?

My own field of psychology is in some ways ideal for small scale, independent research. It can be quite cheap to do a study, doesn’t require years of training or elaborate equipment just to get started, and understanding the published work in the field can be fairly intuitive compared to, say, physics. We’ve all got minds of some kind, after all. But I haven’t seen many efforts to involve the public in psychology as scientists. The field has embraced the internet as a source of subjects (see PRO, Project Implicit, the Moral Sense Test and of course Mechanical Turk for just a few examples) but I don’t recall ever seeing an effort to get the public involved in the research.

Until a few days ago. A friend linked me to the Reproducibility Project, an effort by the Open Science Framework, which is in turn the brainchild of University of Virginia psychologist Brian Nosek. The project is an effort to replicate all articles published in three major psychology journals in the year 2008. This seems largely in response to growing concern over the accuracy of published psychology findings (see here, here here and here for again just a few examples), although Nosek is quick to point out that a failure to replicate does not mean that the original article was fraudulent or even incorrect.

Although it appears that the actual replications are being done entirely within academic institutions, the meta-analysis is being conducted openly on the web, with the public able to review the full introduction and methods sections for each replications, as well as the “bare bones” result, participate in discussions of the project as a whole on the group mailing list, help with administrative work such as coding, and possibly much more.

I am planning on getting involved with the project, so I will report back. While the researcher in me would love to be able to run a replication on my own, independently, I understand the ethical problems (though they could be solved by a public IRB! I’m just sayin’) and I appreciate just how open the rest of this project promises to be.

At the same time, though, each of the projects are a far cry from, say, the Public Lab, which as far as I know does not require institutional affiliation for any part of the research process. They work primarily with aerial mapping. Is it possible to achieve that level of independence in other fields? If not, what’s stopping us?

There are so many roles that people can play in the research process, even without being academics. They can be subjects, funders, reviewers, data collectors, analysts - and, I am sure, visionaries. But I’m not sure how we get there.