Copyleft for experimental data

A quick note on an interesting point raised by Roger Peng:

But what’s the problem with the three follow-up scenarios described? The one thing that they have in common is that none of the three responding people were subjected to the same standards to which the original investigator (me) was subjected. I was required to register my trial and state the outcomes in advance. In an ideal world you might argue I should have stated my hypotheses in advance too. That’s fine, but the point is that the people analyzing the data subsequently were not required to do any of this. Why should they be held to a lower standard of scrutiny?

The first person analyzed a different outcome that was not a primary or secondary outcome. How many outcomes did they test before the came to that one negatively significant one? The second person examined a subset of the participants. Was the study designed (or powered) to look at this subset? Probably not. The third person claims fraud, but does not provide any details of what they did.

I think it’s easy to take care of the third person–just require that they make their work reproducible too. That way we can all see what they did and verify that there was in fact fraud. But the first two people are a little more difficult. If there are no barriers to obtaining the data, then they can just get the data and run a bunch of analyses. If the results don’t go their way, they can just move on and no one would be the wiser. If they did, they can try to publish something.

What I think a good reproducibility policy should have is a type of “viral” clause. For example, the GNU General Public License (GPL) is an open source software license that requires, among other things, that anyone who writes their own software, but links to or integrates software covered under the GPL, must publish their software under the GPL too. This “viral” requirement ensures that people cannot make use of the efforts of the open source community without also giving back to that community.

One can argue that a culture of responsible, reproducible research would simply not value those actions in the same way they’d value pre-registered resesarch.  I view the actions of the first and second people in Peng’s hypothetical as much like the critic passing by commenting that a different analysis might be better, or did you realize there’s a confound, or maybe we could try with a different population - part of the brainstorming process, not the actual research process.  (The third person in Peng’s hypothetical is just annoying.)

But maybe that’s giving the scientific community too much credit.  Unfortunately the large, recent survey of academics’ opinions re: open access licensing didn’t ask about “share-alike”, so it’s not clear what kind of support this would have.