Open Access Repository
Assessing theoretical conclusions with blinded inference to investigate a potential inference crisis
Altmetric
AltmetricDownloads
Downloads per month over past year
ORCID: 0000-0003-4324-5537, Heck, DW, Hicks, JL, Huff, MJ, Kellen, D, Key, KN, Kilic, A, Klauer, KC, Kraemer, KR, Leite, FP, Lloyd, ME, Malejka, S, Mason, A, McAdoo, RM, McDonough, IM, Michael, RB, Mickes, L, Mizrack, E, Morgan, DP, Mueller, ST, Osth, A, Reynolds, A, Seale-Carlisle, TM, Singmann, H, Sloane, JF, Smith, AM, Tillman, G, van Ravenzwaaij, D, Weidemann, CT, Wells, L, White, CN and Wilson, J 2019
, 'Assessing theoretical conclusions with blinded inference to investigate a potential inference crisis'
, Advances in Methods and Practices in Psychological Science, vol. 2
, pp. 335-349
, doi: 10.1177/2515245919869583.
|
PDF
(Post-print)
Starns et al (I...pdf | Download (989kB) | Preview |
|
|
PDF
(Post-print supplement)
Starns et al (I...pdf | Download (325kB) | Preview |
Abstract
Scientific advances across a range of disciplines hinge on our ability to makeinferences about unobservable theoretical entities based on empirical data patterns. Accurateinferences rely on both a) discovering valid, replicable data patterns, and b) accurately25 interpreting those patterns in terms of their implications for theoretical constructs. Thereplication crisis in science has led to widespread efforts to improve the reliability of researchfindings, but comparatively little attention has been devoted to the validity of inferences basedon those findings. Using an example from cognitive psychology, we demonstrate a blindedinference paradigm for assessing the quality of theoretical inferences from data. Our results30 reveal substantial variability in expert judgements on the very same data, hinting at a possibleinference crisis.
| Item Type: | Article |
|---|---|
| Authors/Creators: | Starns, JJ and Cataldo, AM and Rotello, CM and Annis, J and Aschenbrenner, A and Broder, A and Cox, G and Criss, A and Curl, RA and Dobbins, IG and Dunn, J and Enam, T and Evans, NJ and Farrell, S and Fraundorf, SH and Gronlund, SD and Heathcote, A and Heck, DW and Hicks, JL and Huff, MJ and Kellen, D and Key, KN and Kilic, A and Klauer, KC and Kraemer, KR and Leite, FP and Lloyd, ME and Malejka, S and Mason, A and McAdoo, RM and McDonough, IM and Michael, RB and Mickes, L and Mizrack, E and Morgan, DP and Mueller, ST and Osth, A and Reynolds, A and Seale-Carlisle, TM and Singmann, H and Sloane, JF and Smith, AM and Tillman, G and van Ravenzwaaij, D and Weidemann, CT and Wells, L and White, CN and Wilson, J |
| Keywords: | memory, modeling, Bayesian methods, bootstrap, metascience, blinded inference, open data |
| Journal or Publication Title: | Advances in Methods and Practices in Psychological Science |
| Publisher: | Sage Publications, Inc. |
| ISSN: | 2515-2459 |
| DOI / ID Number: | 10.1177/2515245919869583 |
| Copyright Information: | © The Author(s) 2019 |
| Related URLs: | |
| Item Statistics: | View statistics for this item |
Actions (login required)
 |
Item Control Page |