Frank Harrell

@5_utr Reminds me of the time a well-respected machine learning expert in the computer science department at Stanford told me during my presentation that machine learning has no minimum sample size requirement. That's hogwash. fharrell.com/post/ml-sample…

@f2harrell Their analysis of binary response is a classic — as it takes n=96 just to estimate the intercept. Another “turn AI loose on the datasets”

“Interpretable deep learning” on thousands of CpG features. Validation n=10. AUC=0.97. The model has more parameters than patients. HT @f2harrell fharrell.com/post/ml-sample…

The ORBITA group continues to do groundbreaking, innovative cardiovascular research, and questioning commonly held beliefs in cardiovascular medicine. Proud to be associated with them.

50 years of oncogene theory. One broken premise: that we can identify the causal driver, that it’s singular, that inhibiting it collapses the network. We can’t. It isn’t. It doesn’t. The oncogene is not the disease. It’s the mutation we happened to sequence.

@kalashjain1 @5_utr @f2harrell @kalashjain1 Hello, you can read it here: threadreaderapp.com/thread/2052202… See you soon. 🤖

🧵 A new Nature paper on “spatial ecotypes” claims liquid biopsy can predict immunotherapy response. The tumor biology is cool. The statistics are a masterclass in how to do bad biomarker research. @f2harrell has a checklist for this. Let’s go. 1/

New presentation: hbiostat.org/talks/rcteff2.… #Statistics #clinicaltrial #rct #pharma

@djc795 @venkmurthy The main pitch is having timely data whether you stop or not. Correct, stopping early for inefficacy is not the dream but it highly advantageous to the sponsor to not keep investing in a drug with low P(benefit). Bayes also helps with stopping early for efficacy.

@f2harrell @venkmurthy I like the idea of stopping much earlier for lack of efficacy, but I don’t think that fits the narrative they are pitching.

Great to see this type of investment in modernization which will lead to faster, more efficient trials without compromising scientific merit or confidence!

@jeremyeorr @DrMakaryFDA @US_FDA This has been pretty well worked out in the past 30 years. For frequentist designs the p-value boundary is conservative at early looks. Bayesian operating characteristics of frequentist approach shows it to stop too late. Conservatism/w Bayes thru prior hbiostat.org/bayes/design

@DrMakaryFDA @US_FDA More information regarding the statistical implications needed here. Hopefully plan is not to just stop when the pvalue crosses 0.05? @f2harrell

A milestone day for clinical trial innovation. We’re announcing the first real-time clinical trials, where @US_FDA can see data signals and endpoints in real time. A quick explainer:

@djc795 @venkmurthy Some do. Bayes can protect one from overstating benefit if stop early, and can allow one to stop MUCH earlier for inefficacy. Frequentist group sequential methods stop too late. hbiostat.org/bayes/design #Statistics #clinicaltrial #rct

@venkmurthy Really? I sincerely doubt it. Very few interventions have a true effect size large enough to stop early. And most of the ones that do will be false positive signals.

Regression Modeling Strategies 4-day virtual course May 14-15, 18-19. More than regression:principled flexible modeling, accuracy, interpretation, selection, stability, validation, Bayes, contrast w/ML ... hbiostat.org/course @AmstatNews @InstatsS #Statistics @VUDataScience

1-day online course May 11: Intro to R/RStudio & an efficient comprehensive R workflow, rms package, enhancing regression skills to be ready for the RMS 4-day course. Details&registration at hbiostat.org/course @AmstatNews @InstatsS #Statistics #rstats @VUDataScience

@AprileBernardi @learnfromerror @PavlosMsaouel @elmir1omerovic @StatModeling Importantly, any disagreements between two observers on the posterior probabilities coming from their use of different priors pales in comparison to disagreements about what to do after computing a data probability computed conditional on an unobservable. #Statistics

@f2harrell @learnfromerror @PavlosMsaouel @elmir1omerovic @StatModeling So Bayesian error prob = 1-PPV? Valid in diagnostic screening. For hypothesis testing it falls apart. Predictive values require base rate assumptions. But there can’t be any rate of true Hs. A population where there are true *and* false Hs is impossible.

@f2harrell How would you respond to the criticism by Evans et al. in 10.1001/jama.2026.4175? Are the arguments made there against Bayesian methods in phase 3 RCTs valid in any meaningful methodological sense? In particular, what do you make of this claim: “Use of Bayesian methods in late-phase or confirmatory clinical trials has generally been limited to supplementary analyses, given recognition that their implementation can compromise evidentiary and integrity standards and the reliability of results through (1) concession of the benefits of randomization through their inclusion of external (prior) information; (2) loss of objectivity by incorporating sponsor- or investigator-specific priors; and (3) reduced robustness via reliance on strong and sometimes unverifiable assumptions.”

@AprileBernardi @learnfromerror @PavlosMsaouel @elmir1omerovic @StatModeling The analogy with diagnostic testing is useful but doesn't go that far. And "Base rate" (prevalence) is not a well-defined concept.

@rahatheart1 @CMichaelGibson @DavidWienerMD @DrJMieres @DocSavageTJU @GiuseppeGalati_ @FaisalBakaeen @ShrillaB @mirvatalasnag @venkmurthy What an incredible story from a true legend in cardiology!

A phenomenal mentor & mentee in conversation.... the humble interests in Cardiology which constituted a world legend, Dr. Eugene Braunwald by @CMichaelGibson! @DavidWienerMD @DrJMieres @DocSavageTJU @GiuseppeGalati_ @FaisalBakaeen @f2harrell @ShrillaB @mirvatalasnag @venkmurthy

@yudapearl Indeed; perhaps the most important point of all.

There is another question trialists often forget to ask: "What's the point?" or "What would we be able to do/conclude after the trial that we couldn't do/conclude before?"

@learnfromerror @AprileBernardi @PavlosMsaouel @elmir1omerovic @StatModeling It's OK to define terms as long as they accurately reflect what is going on. "Error probability" strongly violates that principle. A straightforward term is type I assertion probability, which exactly reflects the situation and does not oversell (type I = assume the null).

Some Bayesians refuse to grant frequentists the use of any term. Frequentist error probabilities are hypothetical claims associated with methods, (e.g., the prob test T would yield d(X) > d(x) computed under h'). If they were called X-probs, some Bayesians would still say, they're not X-probs, only posteriors are. Bayesians should allow frequentists to define their terms, reject their use if you don't like them, but don't insist they must be posteriors, when they're not.

Work like this misleads the clinician audience into dichotomania around “positive” vs “negative” and down the absence of evidence is evidence of absence fallacy and “zero” fallacy rabbit holes Efficacy is not a hypothesis, it’s a matter of degree @f2harrell

@learnfromerror @AprileBernardi @PavlosMsaouel @elmir1omerovic @StatModeling Definitely. You can't compute an error probability without a prior because you can't compute a posterior probability without a prior. Mislabeling type I assertion probability alpha as an error prob just adds confusion.

@f2harrell @AprileBernardi @PavlosMsaouel @elmir1omerovic @StatModeling That's exactly why a Bayesian posterior is not an error probability. But the last sentence makes no sense--a prior is the cost of an error prob???

@learnfromerror @AprileBernardi @PavlosMsaouel @elmir1omerovic @StatModeling To expand on one part, a man who rejects a hypothesis does this on the basis of the p-value without any knowledge about H0. Had knowledge about H0 been available, he would not have needed data.

@learnfromerror @AprileBernardi @PavlosMsaouel @elmir1omerovic @StatModeling Confuses "false positives" with error probabilities. "How often it does so erroneously" and "claims based on data" are unconditional probabilities that do not bring H0 into the equation. This was Fisher's glaring error