Lau Holgersen

@cremieuxrecueil The most aggravating part is that he didn’t even understand what the problem was himself

This is from the original blogpost that gave away that the Wansink nutrition lab was engaging in p-hacking He posted this and had no idea he was admitting to p-hacking. He didn't see anything wrong with this The top comment asked if the post was a joke. He replied he's serious

@lakens @johuellm Do you know of any papers that study how often robustness checks support the main hypothesis? Might be interesting to look into

@johuellm Without reading, let me guess: None of the robustness checks change the main conclusion of the paper? I think if you would do meta-science research on this, 99% of robustness checks uphold the main claim. Like a small miracle.

These authors got worked: 21 robustness checks across four pages. pubsonline.informs.org/doi/10.1287/is…

I am hiring a PhD student on the meaningful interpretation of effect sizes as part of my VICI funded project. This is a 4 year paid position in a welcoming and collaborative environment. Find out more or apply at tue.nl/werken-bij-tue…

New paper: Blaming the Thermometer for the Fever: Separating Misapplication from Method in Null Hypothesis Significance Testing, with @laabho and @jesper_wulff . We - strongly - push back on criticisms against NHST, and argue for its proper application. papers.ssrn.com/sol3/papers.cf… >

@matloff @lakens @jesper_wulff I see where you are coming from, but the binary structure of NHST is a feature, not a bug. It does not ignore uncertainty... it manages it through predefined error control. CIs serve a different role in summarizing estimation uncertainty, not guiding decisions.

I skimmed through the paper last night. From a mathematical point of view, I found your point about choosing alpha to be very interesting, but again, problematic as binary. Re usurping, OK, the analyst can ignore the binary outcome of the test, but the deeper problem is that by being binary, it is discarding useful information. A CI retains that information.

New preprint with @lakens and @jesper_wulff: "Blaming the Thermometer for the Fever: Separating Misapplication from Method in NHST" NHST gets a lot of blame—mostly for issues it did not cause! Read it here: ssrn.com/abstract=52121…

@matloff @lakens @jesper_wulff (a) Measurement error affects all estimates. (b) A p-value is a probability statement, not a decision rule. It only “usurps” the analyst role if allowed. But you should read the paper; we do not claim NHST as the only useful methodology

This is fundamentally different from my views, matloff.github.io/No-P-Values. I contend that binary-outcome procedures, e.g. based minimal effect sizes, are inappropriate for two reasons: (a) they are subject to measurement error and (b) they usurp the role of the analyst--the decision should be made by the analyst, not by the algorithm.

As promised, the preprint is now out: "Blaming the Thermometer for the Fever: Separating Misapplication from Method in NHST" We push back against calls to ban dichotomous testing and defend error control—properly applied. Read here: ssrn.com/abstract=52121…

@Afinetheorem Good thread! No need to manually maintain a .bib file—use a reference manager like Zotero. Just add a DOI or ISBN, and it will fetch all the right details. You can share reference lists with coauthors, and it integrates seamlessly with Overleaf

For new PhD students in social science (and some old professors - I sure needed this a few years ago!): prepared some notes on what the "2024 PhD Tech Stack" should look like. Goal: your research is replicable, accessible, and efficient. 1/9

@intuitidbits Is this really what anyone would consider an acceptable DAG? I see absolutely no convincing argument here directly linking low RHR to aggression… I do see copious amounts of selective reporting though

Low resting heart rate (bpm < 70) is the best physiological predictor of violent/antisocial behavior. In a remarkable longitudinal study of over 700,000 men, those with the lowest RHR: -were the most likely to commit both violent and non-violent crime -had a 31% increased risk of getting into a car crash -were 41% more likely to be injured as a result of assault -demonstrated an earlier onset of criminality than those with a RHR > 82 bpm When adjusted for confounders (ex: socioeconomic status, physical activity), the relationships between low RHR and antisocial behavior became even stronger.

@ajordannafa It’s not entirely arbitrary though… but yes, using more stars for a larger alpha is the real crime

@RexDouglass But using smallest effect size of interest to test against clearly shows that it is not a made up concept to sell power analysis… clearly it serves other practical purposes

@RexDouglass To be fair, any concept in statistics is made up. Besides, smallest effect size of interest is useful for other things than power analysis… for example to test against instead of null thus providing a more severe test

@CharlesDriverAU @lakens @J_A_Quent But why would you want indirect error control when alpha offers a direct, and as you point out, standardized way of achieving this?

@lakens @J_A_Quent Do you think setting alpha levels is the most intuitive way to incorporate appropriate frequentist error control? Given alpha is often at least as arbitrary (though more standardised) as a prior, 'maybe' indirect error control via thoughtful prior setting would be... helpful?

I love it when Bayesians turn a Bayes factor into a frequentist error controlled test. I wonder if I should point out that for any given sample size and prior, we can recompute their Bayes factor threshold into an alpha level, and the observed Bayes factor into a p-value...

@RexDouglass Gelman aside, it seems to me that the three main points he makes are completely fair and not mischaracterized, shallow or creepy as you postulate. In fact this very blog is about dichotomous choices.

@andrewgwils @lukaemon Quite the conjecture… if used to point out flaws, discrepancies, grammar mistakes, or propose better structure, rather than simply creating content, wouldn’t it only make you a better writer in the long run?

@lukaemon I'm talking about communication. There are of course caveats, but I think when used casually it could atrophy vital cognitive functions and independent thought.

I don't use GPT or LLMs for writing and you shouldn't either. Don't offload cognitive functions.

@RexDouglass What does he say in the blog that makes him seem like a snake oil salesman?

@larsgade @P1radio @EsbenBjerre Hvis det kun er landmænd, der må udtale sig om landbruget, så sætter vi lidt ulven til at vogte fårene, gør vi ikke?

@schneider_jw Og derfor burde equivalence tests være meget mere udbredt... Desværre er det sjældent noget, man ser.

Men "ingen signifikant forskel" betyder ikke, at der ikke kan være en sammenhæng! "Abscence of evidence" betyder ikke "evidence of abscence".

@Perpetualmaniac The ability to extrapolate is a blessing and a curse

My god, this paper by that open ai engineer is terrifying. Everything is about to change. AI super intelligence by 2027.