Shauna's Blog

The perfect journal article

Originally at http://www.shaunagm.net/blog/2013/08/the-perfect-journal-article/

If I was the Supreme Dictator of Science, able to marshal all sorts of resources to support my decrees, I would require that every single journal article contain, in its supplemental materials:

  • Information on study pre-registration as well as changes made to the pre-registered study plan, and the reasoning behind them. This would be updated to include corrections and redactions as necessary.
  • Exhaustive information on all materials and procedures used to run the study, such as information on the sourcing of chemical compounds, digital copies of stimuli used in behavioral studies, or the make and settings of lab equipment.
  • Complete raw data, except where redacted for privacy. When redacted, encrypted versions would be maintained in a central secure repository. When data is processed, copies/accounting of tools (software and other kinds) used to process data.
  • An explanation of how analysis was planned with disclosures according to standards in the field. For example, see social psychology’s PsychDisclosure or the researcher degrees of freedom.
  • A complete description of the analysis, including copies of any software/scripts used for analysis. Results given according to standards in the field - for instance, when significance testing, including relevant descriptive statistics and using an exact p-value instead of p < .0X.
  • An accounting of each author’s contributions to the study, along with disclosure of potential conflict of interest.
  • Universally standard metadata which would allow people to easily do a thorough search not just by name or title but by materials, manipulation, population, and result.
  • Additional versions of the article written for educated laypeople and, optionally, a version for secondary education.
  • An interactive page where people can ask clarifying questions or make casual critique publicly.

The supplementary materials would, of course, be completely open access. Publishers could charge whatever they wanted for the traditional article, but you’d have to pay me to read that when all the above is at my fingertips.

When I’m reading journal articles I’m constantly struck by how much of a story they are. Authors make so many decisions about what to include, how to phrase their explanations and their results, which statistics to report, what graphs and images to illustrate. They create a narrative, which is not necessarily a bad thing. But when traditional articles are the only source of information, we readers become less like scientists and more like historians, trying to piece together the past through testimonies and artifacts.