Rickard Carlsson

Join our conference on Open Science and Reproducibility. Keynote by @lakens and @annemscheel, with @nulliusverbapod live on stage at the record shop / “folkölscafé” (bar) Mono. Does the fake code make you cringe? Then you will love the replication games! URL below.

The Dunning-Kruger Effect is one of the most famous psychology findings. It's the idea that people of low competence/ability overestimate their ability. But is it REAL? At Clearer Thinking, we investigated (with surprising twists and turns): 🧵

Error Report #2: matthewbjane.quarto.pub/meta-analysis-… The meta-analysis by Stanyer et al. (2022) contains substantial data extraction errors (i.e., effect size calculation errors). Due to the severe data extraction errors and the meta-analytic results for memory retention scores are incorrect and hyper-inflated. The reported effects from the word-pair associate outcomes are 5x larger than the corrected effects (meta-analytic mean of corrected effects: d=0.19, mean of Stanyer et al. reported effects: d=0.98).

New comment in NHB by Farid Anvari, me and friends arguing that psychology is fragmented into the study of too many constructs and measures with too few links.

Very good start of the new Metaror journal as the public peer reviews are as critical of @jamesheathers recent claim that 1 in 7 papers are fraud as I want my peer metascientists to be of such badly supported bold claims. Read the reviews here: metaror.org/kotahi/article…

My commentary on Isager, van ‘t Veer, and Lakens’ “Replication value as a function of citation impact and sample size” is now accepted at Meta-Psychology. CC: @lakens @peder_isager Preprint: osf.io/preprints/meta…

Hattie’s Visible Learning is a good start!

@squig @lakens @mVranka Ok that is fair!

@RickCarlsson @lakens @mVranka Nobody is ignoring those rights. All studies include contact information for participants for more information. Seems a much more straightforward way of informing than publishing it behind a paywall somewhere.

We love to share new papers on here. But how many of the studies that scientists preregister on the Open Science Framework are never shared publicly? In a new paper in AMPPS we estimate 40% of preregistered studies are never shared. That’s a lot. 🧵⤵️ journals.sagepub.com/doi/10.1177/25…

@squig @lakens @mVranka What IRB is ok with the deliberate act of collecting human (however trivial) data and not publishing the results in any form? As someone who teaches research ethics, I am deeply intrigued!

@squig @lakens @mVranka There are, of course, situations where the benefits of research can outweigh some rights of participants, but a cavalier attitude towards it is quite schocking to me. Where I come from (Sweden) this type of deliberate behavior is probably even a punishable offense.

@lakens @Kim_De_Roover It is a real problem though. Many researchers shares the view that the journals are crappy and don’t review for it. Everyone loses on the prestige game.

@Kim_De_Roover I have many questions based on this tweet. Plosone is not a crappy open access journal, and you would need your budget to publish there, as much as any other open access journal with APC. Ans good diamond open access journals have access to the best reviewers (like me).

Why has my ERC-StG not yet resulted in publications? Because I cannot use the requested budget for open access fees for open access fees, & thus can only publish in crappy open access journals, which reject papers without reading them (PlosOne) or are unable to get reviewers.🙄

@lakens @squig @mVranka I would prefer A. You are contributing to a problematic culture in academia by keeping your information secret. You then talk about all these replications you have done but people can’t verify it. It adds confusion and the net problems for everyone else is worse than your gain.

@lakens @squig @mVranka I think that participation in research is a cost by itself regardless of monetary compensation. Participants have the right to learn about the results of the study they took part in

@squig @lakens @mVranka I can in principle agree that data can be used without being shared in such a way that it’s balanced with costs. Rare but possible. Also if data is eg simulations it’s different.

@squig @lakens @mVranka The idea is that involving non-scientists in research incurs an ethical cost benefit analysis and that involves actually using the data. How unethical depends on type of data. It’s always problematic to collect and throw away but level varies wildly.

@lakens @squig @mVranka @Meta_Psy I think that pre-registrations should have a preprepared reporting function and that everyone always upload data meta-data and main results. It can be pretty much automated. Embargo shouldn’t be on lifting the registration but on sharing the results!

@lakens @squig @mVranka I would be very happy working on how we can create a good format for this. We tried with file drawer reports at @Meta_Psy but not with great success. Hard to review, reproduce and little interest in submitting. Clearly a lot more work is needed on that format to work.