Daniele Fanelli

473 posts

Daniele Fanelli banner
Daniele Fanelli

Daniele Fanelli

@DFanDaBiasedMan

Meta-metascientist, methodical methodologist, integral integrity expert, conscientious consciousness student, ass Prof @ Heriot-Watt, everyone @comCensus.org

comCensus.org Katılım Haziran 2017
190 Takip Edilen437 Takipçiler
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
Last day to apply for our postdoc on complexity and integrity. LLM, NLP enthusiasts are especially encouraged to apply. PhD-equivalent experience is acceptable, jobs.ac.uk/job/DME208/res…
English
1
0
1
268
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
SHORT-TERM POSTDOC at TEMET lab: We are looking for an enthusiastic metascientist to help us develop methods to measure the complexity of studies. Less than a year left but Summer is the best (festival) period in Edinburgh! Please circulate. enzj.fa.em3.oraclecloud.com/hcmUI/Candidat…
English
0
2
4
614
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
Lee, sorry but I'll move on to more productive activities. I am sure you have better things to attend, too. All you needed to do to show no dogmatism was to acknowledge that, once I showed to you that Protzko et al (which you set as your best evidence) does not show high rep as you thought, then your belief in a rep problem was logically weakened. Instead, you seem to keep changing topic, eluding the point. My instinct is to be disappointed, but I am happy to call it a misunderstanding. Clearly X is not a good medium for intelligent scientific conversations. Cheerio.
English
1
0
0
41
The Dark Fiddling Pirate Jussim
The Dark Fiddling Pirate Jussim@PsychRabble·
I put aside whether anyone else is "dogmatic" -- I know my own views quite well, and they are articulated super clearly in the convo. They may be debatable but the last thing they are is dogmatic. Here is Daniele's accusation leveled at me (this is the current issue): x.com/DFanDaBiasedMa…
English
1
0
0
69
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
@PsychRabble @BrettButtliere Lee, I hold your work in great esteem. But it seems clear to me at this point that you'll do anything but answer my very simple and direct question. It makes me think that @BrettButtliere is right, unfortunately.
English
1
0
1
31
The Dark Fiddling Pirate Jussim
The Dark Fiddling Pirate Jussim@PsychRabble·
Replying to @DFanDaBiasedMan: The first topic is NOT an evaluation of the retracted Nature study. It is whether, as per your q to me, whether the standards I repeatedly articulated, and implemented on initial read constitue "dogmatism." You have yet to answer that. The quality of the retracted Nature study is a different issue entirely and I won't address that till you acknowledge that those standards are not the least bit "dogmatic."
English
2
0
0
31
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
@PsychRabble what evdence would convince you that the replication crisis is exaggerated? Your argument, as I understand it so far: anything below 70% (why 70??) reproducibility is too low, and yes there is a crisis. My argument: it really depends on the field; immature (low-codified) fields dealing with highly complex (variable in space and time) phenomena may have low reproducibility for perfectly fair reasons. Unless we know how much we should expect for a specific literature, claims of "crisis" and QRPs are unwarranted and unjustified. I would change my mind if I saw systematically low reproducibility in most literatures, regardless of phenomena and maturity. To the contrary, the numbers produced by replication studies show >>50%, and often >>80% in virtually all cases. The lowest figures are for social psychology, unsurprisingly (high complexity). This evidence, a long time ago, made me change my mind and figure that this crisis narrative is overblown. I am sure plenty of literature in psychology and elsewhere is rubbish, but even in social psych average repr. is much higher than you could pessimistically expect given the complexity involved. Over to you: what evidence makes you think that rep is dismally low (everywhere in science??) and what evidence would make you change your mind?
English
1
0
3
89
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
@PsychRabble It is not dogmatism if people are open to changing their minds faced with new evidence. Now, is the new evidence that you just learned about Protzko et al not showing high rep. changing your mind at least a bit?
English
1
0
0
22
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
@PsychRabble Ok, thanks. So, you say, if they did not succeed in showing 90%, this would change your mind. right? So, now that you know that they did NOT achieve that at all - and it is not my opinion but documented fact - shouldn't you start to change your mind?
English
1
0
0
29
The Dark Fiddling Pirate Jussim
The Dark Fiddling Pirate Jussim@PsychRabble·
Right. They reported a 90%+ success rate. Put aside that your opinion of it is different; and, instead, for the moment, stipulate that they succeeded as reported in the original paper. This is the only thing relevant to your OQ: "What could change your mind?" Stipulating that they succeeded, it completely convinced me that it was possible, using conventional methods in psych, to produce highly replicable work.
English
1
0
1
31
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
@PsychRabble Your response was "I've answered the "what could change my mind?" q several times now. x.com/PsychRabble/st……" . And that post cites Protzko. Would you be so kind to articulate?
The Dark Fiddling Pirate Jussim@PsychRabble

Let's get specific. Right about what? I've already given what it would take for me to conclude "this set of studies shows high replicability." And given one example: The Nature paper (and afaict, its Schroedinger's preregistration clusterf*ck does not change its high replic rate). Again, if you want to dismiss the Nature paper, that's fine, but its irrelevant to your q here, which is: "what evidence/analysis would conivnce you that you are wrong...?" Now, repeat that over and over and over and over, and I 'd say a field is fine. You say, "Well I just gave you that evidence." I say: "Well no. You just gave me evidence that you say meets your standards. When I checked it, it definitely did not meet my standards. You and I using different standards does not mean either one of us is particularly dogmatic."

English
1
0
0
28
The Dark Fiddling Pirate Jussim
The Dark Fiddling Pirate Jussim@PsychRabble·
@DFanDaBiasedMan Let's stick to one point. That last point was your q: "Do you have stds that would change your mind?" Let's start by you acknowledging that I articulated exactly such standards. Only after that is it worth discussing whether some particular set of studies meet those standards.
English
1
0
0
22
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
Ok, then back to Protzko et al. The real replication rate they observed, with ordinary analyses (and not the fanciful ones that were claimed to be pre-registered and weren't) was not 90%, as you think, but at best 70% (see nature.com/articles/s4156… and joebakcoleman.com/blog/2024/nose…). 70% was attained by plenty of replication studies before (see my thread, and Coleman's blog). Moreover, the Protzko study was not designed to prove causality. So - apart from its retraction - the study proves the contrary of what you are claiming . Because the rates they observed are in the same ballpark than those already observed in the literature by many replication studies. Do you see? And 70% is also your threshold for good science, right? Which was attained by plenty of replication studies already. Shouldn't all this count against your hypothesis?
English
1
0
0
23
The Dark Fiddling Pirate Jussim
The Dark Fiddling Pirate Jussim@PsychRabble·
@DFanDaBiasedMan . I've answered the "what could change my mind?" q several times now. x.com/PsychRabble/st… 2. I know psych. Seen lots of evidence that other social sci are (in general) mostly similar with some exceptions. 3. I'd guess its better in natural sci, but DK the data. 1/2
The Dark Fiddling Pirate Jussim@PsychRabble

Let's get specific. Right about what? I've already given what it would take for me to conclude "this set of studies shows high replicability." And given one example: The Nature paper (and afaict, its Schroedinger's preregistration clusterf*ck does not change its high replic rate). Again, if you want to dismiss the Nature paper, that's fine, but its irrelevant to your q here, which is: "what evidence/analysis would conivnce you that you are wrong...?" Now, repeat that over and over and over and over, and I 'd say a field is fine. You say, "Well I just gave you that evidence." I say: "Well no. You just gave me evidence that you say meets your standards. When I checked it, it definitely did not meet my standards. You and I using different standards does not mean either one of us is particularly dogmatic."

English
2
0
0
53
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
@PsychRabble @lakens LOL - you may be right, but surely you can see how I get the same impression from you. I'll ask here again the key question: what evidence/analysis would convince you that you are wrong and I am right?
English
1
0
0
46
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
@PsychRabble @lakens And there are more studies now, that show even higher rates. Let me ask you this: what evidence would convince you that you are wrong?
English
0
0
0
26
Daniele Fanelli
Daniele Fanelli@DFanDaBiasedMan·
@PsychRabble @lakens The point being that all of those studies are at least compatible with 70% or higher. and most suggest >50% minimum. These studies should be known to all - I didn't make them up.
English
1
0
0
30

Keşfet

@PsychRabble @BrettButtliere @elonmusk @BarackObama @taylorswift13 @cristiano @BillGates @NASA