Lawrence Lynn@PatientStormDoc
There are two instantiations of the paradox which, unrecognized, have plagued critical care science for decades.
First “RCT-to-clinical protocol effect reversal”.
Suppose investigators conduct a RCT to test a treatment for patients with severe infection, which experts label “sepsis.” A consensus panel defines disease and cause agnostic triage criteria for sepsis, and patients meeting those criteria are enrolled in the trial.
The treatment protocol is tested in what appears to be a well-designed &! powered RCT that meets all methodological standards & reports strong evidence of benefit.
The treatment protocol is then converted into clinical guidelines. However, when implemented in practice, the protocol surprisingly results in patient harm, even when applied with the same treatment timing, the same hospital system, the same caregivers, & using the same clinical entry criteria as the trial.
This is Lynn’s paradox.
The paradox arises because the RCT was designed without causal modeling. Sepsis is not a single disease or causal entity it was initially derived as similarity heuristic & then migrated to causal disease equivalent status in the 80s. In practice, more than 50 different infectious diseases/causes can satisfy the consensus criteria used to define sepsis.
If the treatment benefits some of these diseases/causes but harms others, the overall result of the RCT reflects only the average effect across that specific tested mixture of diseases. The trial appears positive only because the particular mixture of diseases among enrolled participants happened, by chance, to favor those that benefited from treatment.
Unfortunately, this mixture is not stable. When the same criteria are applied in routine clinical practice, the proportions of diseases will shift. The result can be clinical guidelines derived from a “positive” RCT that produces net harm.
This phenomenon also appears as RCT reversal, in which sequential trials of the same intervention in the same setting and the same every criteria, equivalent covariates, and treatment timing produce opposite conclusions. This is Lynn’s Paradox of sequential RCTs.
The mechanism can be illustrated with a simple toy model. Imagine sequential RCTs enrolling patients who meet the latest consensus “sepsis criteria” but some participants have one infectious diseases (d1) and other participants have another infectious disease (d2). (both diseases meet the trial entry criteria).
The treatment effect benefits d1 to the same degree that it harms d2
If the first RCT happens to enroll more participants with d1, the trial will show benefit. If a subsequent RCT enrolls a higher proportion of d2 the result will reverse and show harm, even though the hospital, caregivers entry criteria, trial design, power, and covariate mix and treatment protocol remains equivalent
This is difficult to understand only because no one trained in SCM would think that disease & cause agnostic entry criteria for an RCT is acceptable. How could covariates be modeled as they may be different for each disease/cause and the disease/cause mix is not stable. Yet this is standard structure in critical care syndrome science
Lynn’s paradox was the reason the ventilator protocols failed during the pandemic. Severe COVID pneumonia met the disease agnostic criteria for adult respiratory distress syndrome (ARDS) so the guideline committee thought they had strong evidence for a particular ventilator protocol (high PEEP) and produced guidelines with “strong” evidence based recommendations for this ventilator protocol
But COVID pneumonia did not exist when the source RCT was performed it only met the criteria used in the trials. But meeting the syndrome criteria establishes causal equivalence in syndrome science so they thought they had high level RCT evidence for the protocol. It failed & has now been abandoned. That was Lynn’s Paradox
Requiring SCM interrogation as part of RCT design would prevent the paradox
