Adrian K. Yee

The stereotype threat effect, once a darling of social psychology, goes down the drain in another large, pre-registered replication project. osf.io/preprints/psya… Stereotype threat refers to the fear of being judged based on negative stereotypes about the performance of a certain group one identifies with. Stereotype threat is widely studied and discussed in the psychological literature, covered in many introductory psychology textbooks, and featured in prominent court cases on the fairness of selection into academic institutions. This registered replication report describes the result of eight direct replications (total N = 1502) of a representative stereotype-threat study by Johns et al. (2005), who found that threatened women (but not men) underperform when they are confronted with a mathematics test that is presented to measure gender differences, and that this effect can be alleviated by altering test instructions.The seminal study by Johns et al. (2005) paper has been widely cited (855 times on Google Scholar and 283 times on Web of Science, as of April 25th 2024). With this registered replication report, we hope to demonstrate the value of large-scale stereotype-threat experiments. We were unable to replicate a stereotype-threat effect, neither as the interaction between the stereotype-threat conditions and gender, nor as a simple main effect among women. The average effect size we found among women was virtually null, and, thus, substantially smaller than the originally reported effect size. The current results fail to replicate the stereotype-threat effect by Johns et al. (2005), hence casting doubt on the generalizability of the effect of stereotype threat on women’s mathematics performance. Our null results are in line with the recent findings in preregistered (replication) studies. Though there are theoretical explanations imaginable, these results seems to be in line with previous notion that effects of stereotype threat as described in the published literature might have been inflated due to publication bias.