Reproducible research practices, scientific transparency, and subgroup claims : a meta-research dissertation
Abstract/Contents
- Abstract
- There is a growing movement to encourage reproducibility and transparency practices across all scientific fields, including public access to raw data and protocols and the conduct of replication studies. In this dissertation, I evaluate: the status and trends of reproducibility and transparency across all biomedical research fields; the reproducibility of subgroup analyses in randomized controlled trials; and the frequency, validity, and relevance of sex-based subgroup analyses in Cochrane reviews. Among a random sample of 441 biomedical journal articles published in 2000-2014, only one study provided a full protocol and none made raw data directly available. In general, replication studies were rare (n = 4). In a sample of 169 randomized controlled trials (from two datasets containing randomized controlled trials with at least one subgroup analysis), there were a total of 117 subgroup claims made in the abstracts of 64 articles. Of these claims, only 46 (39%) had evidence of statistically significant heterogeneity (P < 0.05 from an interaction test). Five (11%) of the 46 subgroup findings had at least one subsequent pure corroboration attempt by a meta-analysis or a randomized controlled trial. None of the corroboration attempts had a statistically significant P-value from an interaction test. Among 41 reviews, with at least one sex-treatment subgroup analysis in at least one forest plot, published in the Cochrane Database of Systematic Reviews, there were 109 separate treatment-outcome analyses ("topics"). Eight (7.3%) had a statistically significant sex-treatment interaction and only three were discussed by the Cochrane reviewers for a potential impact on different clinical management for males versus females. Overall, the current published biomedical literature lacks transparency. A minority of subgroup claims made in the abstracts of randomized controlled trials are supported by their own data, i.e., a significant interaction effect. Corroboration attempts of statistically significant subgroup differences are rare, and when done, the initially observed subgroup differences are not reproduced. Lastly, statistically significant sex-treatment interactions are only slightly more frequent than what would be expected by chance and there is little evidence of subsequent corroboration or clinical relevance of sex-treatment interactions.
Description
| Type of resource | text |
|---|---|
| Form | electronic; electronic resource; remote |
| Extent | 1 online resource. |
| Publication date | 2016 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Associated with | Wallach, Joshua David |
|---|---|
| Associated with | Stanford University, Program in Epidemiology and Clinical Research. |
| Primary advisor | Ioannidis, John |
| Thesis advisor | Ioannidis, John |
| Thesis advisor | Basu, Sanjay, 1980- |
| Thesis advisor | Minor, Lloyd B |
| Thesis advisor | Nelson, Lorene M |
| Thesis advisor | Sainani, Kristin |
| Advisor | Basu, Sanjay, 1980- |
| Advisor | Minor, Lloyd B |
| Advisor | Nelson, Lorene M |
| Advisor | Sainani, Kristin |
Subjects
| Genre | Theses |
|---|
Bibliographic information
| Statement of responsibility | Joshua David Wallach. |
|---|---|
| Note | Submitted to the Program in Epidemiology and Clinical Research. |
| Thesis | Thesis (Ph.D.)--Stanford University, 2016. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2016 by Joshua David Wallach
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...