Skip to main content

Table 1 Reproducibility indicators

From: An analysis of key indicators of reproducibility in radiology

Reproducibility indicatorRole in producing transparent and reproducible scienceOriginal sample (n = 300)
Articles
Article accessibility: articles were assessed for open accessibility, paywall access, or unable to access full textEase of access to publications enables interdisciplinary research by removing access barriers. Full-text access allows for validation through reproduction.All (n = 300)
Funding
Funding statement: presence of funding sources of the studyFunding provides researchers the ability to create new experiments and tangibly investigate their ideas. However, funding sources can play a role in influencing how researchers conduct and report their study (e.g., scientific bias), which necessitates its disclosure.All included publications (n = 294)
Conflict of interest (COI)
COI statement: presence of conflict of interest statementConflict of interest conveys the authors’ potential associations that may affect the experimental design, methods, and analyses of the outcomes. Thus, full disclosure of possible conflicts allows for unbiased presentation of their study.All included publications (n = 294)
Data
Data statement: presence of a data availability statement, retrieval method, comprehensibility, and contentRaw data availability facilitates independent verification of research publications. It can improve accountability of outcomes reported and integrity of the research published.Empirical publications (n = 195)
Pre-registration
Pre-registration statement: presence of statement indicating registration, retrieval method, accessibility, and contents (hypothesis, methods, analysis plan)Pre-registration explicitly reports aspects of the study design prior to the commencement of the research. Pre-registration functions as a way to limit selective reporting of results and prevents publication biases and P-hacking.Empirical publications (n = 195)
Protocols
Protocol statement: assessed for statement indicating protocol availability, and if available, what aspects of the study are available (hypothesis, methods, analysis plan)Reproducibility of a study is dependent on the accessibility of the protocol. A protocol is a highly detailed document that contains all aspects of the experimental design which provides a step by step guide in conducting the study.Empirical publications (n = 195)
Analysis scripts
Analysis scripts statement: presence of analysis script availability statement, retrieval method, and accessibilityAnalysis scripts are used to analyze data obtained in a study through software programs such as R, Python, and MatLab. Analysis scripts provide step by step instructions to reproduce statistical results.Empirical publications (n = 195)
Replication
Replication statement: Presence of statement indication a replication study.Replication studies provide validation to previously done publications by determining whether similar outcomes can be acquired.Empirical studies (n = 195)
Materials
Materials statement: presence of a materials availability statement, retrieval method, and accessibilityMaterials are the tools used to conduct the study. Lack of materials specification impedes the ability to reproduce a study.Empirical publications (n = 191)
  1. The indicators measured for the articles varied depending on its study type. More details about extraction and coding are available here https://osf.io/x24n3/
  2. Excludes publications that have no empirical data
  3. Excludes case studies and case series
  4. Excludes meta-analyses and systematic reviews, which materials may not be relevant, in addition to ‡