Comprehensive review of US's clinical trials system needed
An editorial in Nature by Arthur Ammann points out a number of problems with our current clinical trial system:
Meta-science
69 posts
About Meta-science
A Tumblr collecting links, commentary, and notes on reproducibility, publication bias, research integrity, and the sociology of science. Originally at metascience.shaunagm.net (2012–2013).
- Read more → More from Meta-science →
-
Daisy-chained replications
Read more → More from Meta-science →This is a little old - I’m going through my bookmarks - but:
-
These problems are not new
Read more → More from Meta-science →From a paper published in 1945 by Vannevar Bush, head of the U.S. Office of Scientific Research and Development (OSRD) during World War II:
-
University of California opens one door and closes another
Read more → More from Meta-science →The recent announcement that the University of California - the world’s largest public university system - will be embracing open access has been meet with celebration. However, the new policy only encourages faculty to share their work - there are no requirements. Meanwhile, the UC Regents have voted to create a new corporate entity to manage research funding:
-
Retractions as political statement
Read more → More from Meta-science →From Retraction Watch, a story about a retraction is really a story about Nature’s failure to publish a refutation:
-
Pressure to publish and the prevalence of positive results
Read more → More from Meta-science →This PLoS One study attempted to look at how pressure to publish might influence the prevalence of positive results. The author, Daniele Fanelli, made an odd choice by using ‘papers per capita by state’ as the measure of pressure to publish. The state seems way too macro a level to look at. I would expect pressure to vary strongly between schools and between departments within schools. There’s no reasoning for this given in the paper - I suspect that it was just simpler to use this measure, already provided by the NSF, then to come up with a method for looking at individual institutions.
-
Solutions: Open Science Data Cloud
Read more → More from Meta-science →This article about Bionimbus, about an effort to efficiently and securely share cancer genomics data, mentions the Open Science Data Cloud. The OSDC is a platform for sharing large datasets, and although their current set of projects, including Bionimbus, are mostly genomics-focused, they seem to be open to datasets of all types.
-
Softer sciences publish more positive results
Read more → More from Meta-science →A colleague referenced this 2010 paper which measures proportion of positive results by scientific discipline.
-
Copyleft for experimental data
Read more → More from Meta-science →A quick note on an interesting point raised by Roger Peng:
-
Solutions: P-Curves
Read more → More from Meta-science →A paper and associated web-based tool have been proposed to detect publication bias and p-hacking in the literature. From the paper:
-
Methods reporting in the fMRI literature
Read more → More from Meta-science →I spent three years working in fMRI labs. To this day it’s not clear to me if the field has exceptionally ambiguous standards, or if it’s only one of many scientific subfields based around new technology struggling to define good practice. Whether it’s got company or not, neuroimaging certainly has issues.
-
Coding error influences public policy
Read more → More from Meta-science →There’s an op ed in the New York Times about how a coding error in an economics article may have had profoundly negative influence on economic policy:
-
Power failure
Read more → More from Meta-science →There’s a new article out in Nature Reviews Neuroscience about the failure of scientific studies in general (and neuroscience and fMRI studies in particular) to adequately power their studies. The NRN paper isn’t open access, but you can email the authors for a pre-print. There’s a good write-up at National Geographic.
-
Fraud on the rise, especially in the United States and in high impact journals
Read more → More from Meta-science →An article in the Washington Post details an alleged case of Fraud which led to an as-yet-uncorrected Nature paper and a suicide:
-
Avoidable waste in the production and reporting of research evidence
Read more → More from Meta-science →An opinion article from Iain Chalmers (of the very well regarded though not fully open access Cochrane Collaboration) and Paul Glasziou (of the Center for Evidence-based Medicine) discusses the high potential for waste in much research.
-
Obama administration enacts (qualified) open access
Read more → More from Meta-science →The Obama administration has announced that all research funded by federal agencies (with R&D budgets greater than $100 million) should be made open access within one year of publication:
-
The harm done by tests of significance
Read more → More from Meta-science →An interesting article from Accident Analysis and Prevention from 2004 goes over three case studies where Null Hypothesis Significance Testing may have cost lives.
-
Why paywalls have to go
Read more → More from Meta-science →On the PLoS blog, an article from Jack Andraka, winner of the 2012 Intel Science prize for the invention of a dip stick test for pancreatic cancer, on trying to reach behind paywalls as a 14 year old:
-
Correcting the Lies that Data Tell
Read more → More from Meta-science →Found this interesting paper from a couple years ago: Detecting and Correcting the Lies that Data Tellby F Schmidt.
-
Open Access and its consequences
Read more → More from Meta-science →This post at the Scholarly Kitchen touches on a number of arguments against mandatory CC-BY licensing. They’re all worth talking about, although none seem worth keeping science behind closed doors.
-
Trials with positive results published faster, more often than trials with negative results
Read more → More from Meta-science →A new PLoS one paper looked at the outcome of 1054 trials submitted to the ethics committee of a major Spanish hospital from 1997-2004. This flowchart captures the basic results:
-
Conflict of interest in PubMedCentral indexing
Read more → More from Meta-science →It’s been my experience that I wildly disagree with some of the fundamental values of the folks at Scholarly Kitchen, but I get a lot out of their posts and arguments. Their most recent post on a possible scandal at Pub Med Central is a case in point:
-
Ben Goldacre's petition to register all trials
Read more → More from Meta-science →A relevant follow up to the last post. Science critic Ben Goldacre has a new project: a petition to get all trials registered and to enforce reporting deadlines.
-
Most trials subject to mandatory reporting do not report within a year
Read more → More from Meta-science →An article in BMJ explores how well registered trials adhere to reporting deadlines.
-
No clear evidence on how pharmaceutical promotions influence prescribing
Read more → More from Meta-science →A new PLoS Medicine report investigated how pharmaceutical promotions and advertising influence prescriptions:
-
The Long Way From α-Error Control to Validity Proper
Read more → More from Meta-science →Back in November, Perspectives on Psychological Science put out a special issue on replicability. I’ll be attempting to summarize each of the articles in that issue for this tumblr. See this post for links to all article summaries, or use this tag to browse.
-
Misuses of models overstates significance
Read more → More from Meta-science →A brief comment points out a major flaw in a published study, in part by using simulations:
-
Most medical research: true or false?
Read more → More from Meta-science →A friend forwarded me this paper, “Empirical estimates suggest most published medical research is true”, perhaps in an attempt to challenge my cynicism. I like to believe I am open to being challenged, and I do recognize that purposefully cataloging problems with research leaves me biased. But I don’t think this paper is the best counterpoint.
-
Study on data compliance unearths incorrect analyses
Read more → More from Meta-science →A post at Retraction Watch summarizes the situation:
-
Bias in reporting efficacy and toxicity in breast cancer trials
Read more → More from Meta-science →A recent paper did an analysis of breast cancer studies published over the last 16 years. They evaluated 164 trials and looked at whether results re: the drug’s toxicity or overall survival rate was reported prominently in the abstract, within the article, or at all. They looked at who funded the work, the impact factor of the journal the work was published in, and most interestingly, whether the trial found positive or negative results.
-
Is the replicability crisis overblown? Three arguments explained.
Read more → More from Meta-science →Back in November, Perspectives on Psychological Science put outa special issue on replicability. I’ll be attempting to summarize each of the articles in that issue for this tumblr. See this post for links to all article summaries, or use this tag to browse.
-
Replicability In Psychology Research
Read more → More from Meta-science →Perspectives on Psychological Science recently came out with a special issue on replicability in psychology research and its implications for the reliability and health of the field as a whole. They cite as motivation the last two years, which have been filled with fraud, admissions of questionable research practices, a published paper in a major paper claiming evidence of ESP, reports that researchers cannot or will not share data or disclose the full extent of their analyses, and the failure to replicate several major findings in the field.
-
Replications in Psychology Research: How Often Do They Really Occur?
Read more → More from Meta-science →Back in November, Perspectives on Psychological Science put out a special issue on replicability. I’ll be attempting to summarize each of the articles in that issue for this tumblr. See this post for links to all article summaries, or use this tag to browse.
-
Science Fraud faces legal threats
Read more → More from Meta-science →Science Fraud faces legal threats
-
The academic cookie jar
Read more → More from Meta-science →This review of Diederik Stapel’s autobiography quotes this interesting passage:
-
Overfitting
Read more → More from Meta-science →I found this 2004 article through the footnotes of Nate Silver’s book. Here are the highlights (but I recommend reading it all the way through.)
-
Uncited research
Read more → More from Meta-science →This article raises, for me at least, the interesting question of whether the large amount of research that remains uncited in medical/scientific literature is a good thing, a bad thing, or a neutral thing. I did a little more reading, and found some other posts on the topic.
-
Scientists Propose Central Database for Disclosing Conflicts of Interest
Read more → More from Meta-science →Scientists Propose Central Database for Disclosing Conflicts of Interest
-
The Weirdest People In the World
Read more → More from Meta-science →This article (warning: pdf) talks in detail about how the psychology community frequently fails to account for the great diversity of cultures in our world.
-
Medical Journals Are an Extension of the Marketing Arm of Pharmaceutical Companies
Read more → More from Meta-science → -
Now is the time for transparency and access to clinical-trial data
Read more → More from Meta-science →From this post:
-
Article Level Metadata
Read more → More from Meta-science →Article Level Metadata
-
Citizen Science
Read more → More from Meta-science →This article focuses on making study results available to patients who are providing the medical data. But it touches on more general themes - on the right of patients not just to control their own data but to access the results of research into their illnesses, and to be empowered to do some of the research themselves.
-
Stapel Report
Read more → More from Meta-science →Stapel Report
-
Underrepresentation of Women In Science
Read more → More from Meta-science →I normally do not post about sexism, racism, and other forms of bigotry in the scientific community here. While I obviously think they exist and am against them, it is ambiguous how these “isms” impact/bias the scientific literature. (I am sure they do. But perhaps in a less concrete way than, say, the file-drawer phenomenon, or clear misuse of statistics.)
-
Figshare
Read more → More from Meta-science →From the copy on their website:
-
Retractions stigmatize scientific fields, study finds
Read more → More from Meta-science →Retractions stigmatize scientific fields, study finds
-
False-Positive Psychology
Read more → More from Meta-science →Quoting my own write up:
-
"I don’t really believe about 95% of what gets published."
Read more → More from Meta-science →Vaughn Bell of Mind Hacks interviews researchers about the problems with fMRI:
-
Skeletons & Standards
Read more → More from Meta-science →Dorothy Bishop critiques a 2003 neuroimaging paper:
-
A Sharp Rise In Retraction Prompts Calls For Reform
Read more → More from Meta-science →From the article:
-
How Many Scientists Fabricate and Falsify Research?
Read more → More from Meta-science →“A Systematic Review and Meta-Analysis of Survey Data”
-
Pew Research Center Survey on Science
More from Meta-science →(Lots of other interesting non-dismal-sciencey results here.)
-
Reforming Science
Read more → More from Meta-science →A pair of articles in Infection and Immunity. In addition to summarizing problems, they offer the following reforms (all greatly abridged in the excerpts below):
-
Solutions: Alternative Forms of Peer Review
Read more → More from Meta-science → -
Solutions: Crossref.org
Read more → More from Meta-science →As described in this Nature article:
-
The Trouble With Retractions
Read more → More from Meta-science →From the article:
-
Reproducibility Project
Read more → More from Meta-science →Reproducibility Project
-
Why Most Published Research Findings Are False
Read more → More from Meta-science →Why Most Published Research Findings Are False
-
Surely You're Joking... : The Oil Drop Experiment
Read more → More from Meta-science →Surely You're Joking... : The Oil Drop Experiment
-
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Read more → More from Meta-science →Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
-
402 Citations Questioning the Indiscriminate Use of Null Hypothesis…
Read more → More from Meta-science →402 Citations Questioning the Indiscriminate Use of Null Hypothesis Significance Tests in Observational Studies
-
The decline effect and the scientific method
Read more → More from Meta-science →The decline effect and the scientific method
-
Walking Fast and Slow
Read more → More from Meta-science →Walking Fast and Slow
-
We Must Stop The Avalanche of Low Quality Research
Read more → More from Meta-science →We Must Stop The Avalanche of Low Quality Research
-
If you want reproducible science, the software needs to be open source.
Read more → More from Meta-science →If you want reproducible science, the software needs to be open source.
-
Solutions: Dryad Data Repository
Read more → More from Meta-science →Solutions: Dryad Data Repository
-
Journal Disavows Study Touted By Abortion Foes
Read more → More from Meta-science →Journal Disavows Study Touted By Abortion Foes
-
How Corporations Corrupt Science at the Public's Expense
Read more → More from Meta-science →How Corporations Corrupt Science at the Public's Expense