Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

A Bayesian bird's eye view of 'replications of important results in social psychology'

Marsman, Maarten, Shonbrodt, Felix D., Morey, Richard D. ORCID: https://orcid.org/0000-0001-9220-3179, Yao, Yuling, Gelman, Andrew and Wagenmakers, Eric-Jan 2017. A Bayesian bird's eye view of 'replications of important results in social psychology'. Royal Society Open Science 4 (1) , 160426. 10.1098/rsos.160426

[thumbnail of BayesianBirds_MarsmanEtAl_main.pdf]
Preview
PDF - Published Version
Available under License Creative Commons Attribution.

Download (500kB) | Preview

Abstract

We applied three Bayesian methods to reanalyze the preregistered contributions to the Social Psychology special issue “Replications of Important Results in Social Psychology' [1]. First, individual-experiment Bayesian parameter estimation revealed that for directed effect size measures, only 3 out of 44 central 95% credible intervals did not overlap with zero and fell in the expected direction. For undirected effect size measures, only 4 out of 59 credible intervals contained values greater than :10 (10% of variance explained), and only 19 intervals contained values larger than :05. Second, a Bayesian random-effects meta-analysis for all 38 t-tests showed that only one out of the 38 hierarchically estimated credible intervals did not overlap with zero and fell in the expected direction. Third, a Bayes factor hypothesis test was used to quantify the evidence for the null hypothesis against a default one-sided alternative. Only 7 out of 60 Bayes factors indicated non-anecdotal support in favor of the alternative hypothesis (BF10 > 3), whereas 51 Bayes factors indicated at least some support for the null hypothesis. We hope that future analyses of replication success will embrace a more inclusive statistical approach by adopting a wider range of complementary techniques.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Psychology
Uncontrolled Keywords: preregistration; evidence; reproducibility; credible interval; Bayes factor
Additional Information: Published by the Royal Society under the terms of the Creative Commons Attribution License.
Publisher: Royal Society, The
ISSN: 2054-5703
Date of First Compliant Deposit: 12 December 2016
Date of Acceptance: 12 December 2016
Last Modified: 05 May 2023 08:15
URI: https://orca.cardiff.ac.uk/id/eprint/96854

Citation Data

Cited 23 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics