Skip to main content

Research Repository

Advanced Search

Examining the generalizability of research findings from archival data

Delios, Andrew; Clemente, Elena Giulia; Wu, Tao; Tan, Hongbin; Wang, Yong; Gordon, Michael; Viganola, Domenico; Chen, Zhaowei; Dreber, Anna; Johannesson, Magnus; Pfeiffer, Thomas; Generalizability Tests Forecasting Collaboration; Uhlmann, Eric Luis

Examining the generalizability of research findings from archival data Thumbnail


Authors

Andrew Delios

Elena Giulia Clemente

Tao Wu

Hongbin Tan

Yong Wang

Michael Gordon

Domenico Viganola

Zhaowei Chen

Anna Dreber

Magnus Johannesson

Thomas Pfeiffer

Generalizability Tests Forecasting Collaboration

Eric Luis Uhlmann



Contributors

Abstract

This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.

Journal Article Type Article
Acceptance Date Jun 8, 2022
Online Publication Date Jul 19, 2022
Publication Date Jul 26, 2022
Deposit Date Jul 20, 2022
Publicly Available Date Jul 21, 2022
Journal Proceedings of the National Academy of Sciences
Print ISSN 0027-8424
Electronic ISSN 1091-6490
Peer Reviewed Peer Reviewed
Volume 119
Issue 30
Article Number e2120377119
DOI https://doi.org/10.1073/pnas.2120377119
Keywords Multidisciplinary
Public URL https://nottingham-repository.worktribe.com/output/9089236
Publisher URL https://www.pnas.org/doi/full/10.1073/pnas.2120377119
Additional Information Gerardus Lucas was one of the co-authors who lent their time and expertise as contributors to the forecasting study and credited as part of the Generalized Tests Forecasting Collaboration.

Files






You might also like



Downloadable Citations