Open Access Repository

Assessing theoretical conclusions with blinded inference to investigate a potential inference crisis


Downloads per month over past year

Starns, JJ, Cataldo, AM, Rotello, CM, Annis, J, Aschenbrenner, A, Broder, A, Cox, G, Criss, A, Curl, RA, Dobbins, IG, Dunn, J, Enam, T, Evans, NJ, Farrell, S, Fraundorf, SH, Gronlund, SD, Heathcote, A ORCID: 0000-0003-4324-5537, Heck, DW, Hicks, JL, Huff, MJ, Kellen, D, Key, KN, Kilic, A, Klauer, KC, Kraemer, KR, Leite, FP, Lloyd, ME, Malejka, S, Mason, A, McAdoo, RM, McDonough, IM, Michael, RB, Mickes, L, Mizrack, E, Morgan, DP, Mueller, ST, Osth, A, Reynolds, A, Seale-Carlisle, TM, Singmann, H, Sloane, JF, Smith, AM, Tillman, G, van Ravenzwaaij, D, Weidemann, CT, Wells, L, White, CN and Wilson, J 2019 , 'Assessing theoretical conclusions with blinded inference to investigate a potential inference crisis' , Advances in Methods and Practices in Psychological Science, vol. 2 , pp. 335-349 , doi: 10.1177/2515245919869583.

PDF (Post-print)
Starns et al (I...pdf | Download (989kB)

| Preview
PDF (Post-print supplement)
Starns et al (I...pdf | Download (325kB)

| Preview


Scientific advances across a range of disciplines hinge on our ability to makeinferences about unobservable theoretical entities based on empirical data patterns. Accurateinferences rely on both a) discovering valid, replicable data patterns, and b) accurately25 interpreting those patterns in terms of their implications for theoretical constructs. Thereplication crisis in science has led to widespread efforts to improve the reliability of researchfindings, but comparatively little attention has been devoted to the validity of inferences basedon those findings. Using an example from cognitive psychology, we demonstrate a blindedinference paradigm for assessing the quality of theoretical inferences from data. Our results30 reveal substantial variability in expert judgements on the very same data, hinting at a possibleinference crisis.

Item Type: Article
Authors/Creators:Starns, JJ and Cataldo, AM and Rotello, CM and Annis, J and Aschenbrenner, A and Broder, A and Cox, G and Criss, A and Curl, RA and Dobbins, IG and Dunn, J and Enam, T and Evans, NJ and Farrell, S and Fraundorf, SH and Gronlund, SD and Heathcote, A and Heck, DW and Hicks, JL and Huff, MJ and Kellen, D and Key, KN and Kilic, A and Klauer, KC and Kraemer, KR and Leite, FP and Lloyd, ME and Malejka, S and Mason, A and McAdoo, RM and McDonough, IM and Michael, RB and Mickes, L and Mizrack, E and Morgan, DP and Mueller, ST and Osth, A and Reynolds, A and Seale-Carlisle, TM and Singmann, H and Sloane, JF and Smith, AM and Tillman, G and van Ravenzwaaij, D and Weidemann, CT and Wells, L and White, CN and Wilson, J
Keywords: memory, modeling, Bayesian methods, bootstrap, metascience, blinded inference, open data
Journal or Publication Title: Advances in Methods and Practices in Psychological Science
Publisher: Sage Publications, Inc.
ISSN: 2515-2459
DOI / ID Number: 10.1177/2515245919869583
Copyright Information:

© The Author(s) 2019

Related URLs:
Item Statistics: View statistics for this item

Actions (login required)

Item Control Page Item Control Page