Browse Prior Art Database

Process for Improving Reproducibility of Scientific Research

IP.com Disclosure Number: IPCOM000247064D
Publication Date: 2016-Jul-31
Document File: 2 page(s) / 71K

Publishing Venue

The IP.com Prior Art Database

Abstract

Reproducibility of experiments is essential to progress of science. It has recently been estimated that a low percentage of biomedical scientific research is reproducible. The consequences of this low rate extend far beyond the laboratory as people’s lives depend on this research. The disclosed process addresses reproducibility during research design phase rather than relying on later attempts to repeat experiments. The process also acts as a signaling mechanism to scientific communities that an experiment and its analysis are sound.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 36% of the total text.

Page 01 of 2

Process for Improving Reproducibility of Scientific Research


Reproducibility of experiments is essential to progress of science. It has recently been estimated that a low percentage of biomedical scientific research is reproducible. The consequences of this low rate extend far beyond the laboratory as people's lives depend on this research. The disclosed process addresses reproducibility during research design phase rather than relying on later attempts to repeat experiments. The process also acts as a signaling mechanism to scientific communities that an experiment and its analysis are sound.

The general purpose of the disclosed process is to signal to the scientific community in general and to individual journals specifically that regardless of a result of a scientific study, that the research process used to execute the study was sound and bias-free. Too often studies are selected for publication based on their results. Positive and surprising results are more often published than studies that fail to reject null hypotheses (a.k.a. negative results). This bias is understandable; as in journalism, scientific journals achieve status in part based on how groundbreaking their articles appear to be. The disclosed process seeks to recalibrate this bias by redirecting status assignment away from positive and surprising results, which are often the

least reliable, and towards reproducible results, positive or negative and surprising or not. Increasing the status of negative results is a primary goal of the described technique. It will make such results more acceptable for publication in top journals specifically, which often have

the lowest ratios of negative-to-positive results studies. The process aligns the incentives of researchers with that of the scientific community early on and result in more highly reproducible research.

The process for aligning these incentives is a modification to this scientific research process as described above. Rather than analyzing all of the collected data prior to publication, a subset of the data points will be set aside, not to be analyzed or even viewed until post publication. This data will have been collected with the exact same techniques and methods as the data used for the published study. If the published results fail to replicate, within a margin of error, on the new "post publication data" (PPD), there will be no question about whether the methods of the original study were followed precisely. There will also be no question about a difference in what sociologists refer to as "'tacit knowledge' craft skills and extemporisations that their possessors take for granted but can pass on only through example". The "tacit knowledge" is controlled for as well, in a way that no subsequent study could achieve.

The process estimates an acceptable margin of error by comparing results obtained from the data used to publish vs. the results obtained from the PPD against their respective sample sizes. Acc...