Adam Rodman

In case you were wondering why there hasn't been a new @BedsideRounds in a while ... magazine.hms.harvard.edu/articles/can-a…

@DanOduMD @rloganjonesmd @andrewparsonsMD @DavidWuMDPhD @UofH Email!

@AdamRodmanMD @rloganjonesmd @andrewparsonsMD @DavidWuMDPhD Fascinating research, the patient-facing side is the real gap. We built MyHealthDairy as a provider extension for this reason. My @UofH residents are piloting this and would love to discuss how our findings align! Best way to reach out?

Our study of AMIE at @BIDMC_Medicine is out! You can read about what we did in posts from my co-authors (including the Google post below). But I wanted to talk about some background for this study, and what I think are most interesting findings. 🧵⬇️ x.com/GoogleResearch…

@raj_mehta @AnilMakam EHR accuracy is a limit on is also fwiw. But I generally agree, even a so-called "perfect" agent for parsing and retrieving from EHRs would still have considerable limitations coming from the quality of data in the EHR itself

@AdamRodmanMD @AnilMakam The breakthroughs are amazing, but the march of nines is very tough in healthcare, especially given how EHR data has a ceiling on its accuracy. I imagine we are in for an exciting but long process of iterating.

Great thread worth your time And I know this study does not support this, but once AI gets good enough to do primary data collection without curation (conversation, EHR review, visual?) it will be better than majority of doctors for diagnosis, in part since status quo is awful (10-20% misdiagnosjs rate if not higher)

@anko_979 @rloganjonesmd @andrewparsonsMD @DavidWuMDPhD I'll be interested to see if this holds when other data sources are integrated. If there's any entropy signal at all, could be huge to help mitigate diagnostic error.

@anko_979 @rloganjonesmd @andrewparsonsMD @DavidWuMDPhD When I got the results back, I was actually excited -- this is a well-known feature (going back to Elstein in the 70s) of physician reasoning, and I almost couldn't believe that LLMs did the exact same thing.

@MedEdFlamingo @rloganjonesmd @andrewparsonsMD @DavidWuMDPhD @apalepu13 @BIDMC_Medicine @PeterBrodeurMD @BIDMChealth Ha, will do in my spare time :) The TL;DR -- findings of high empathy from LLMs in healthcare are probably mostly a mirage from the eval methods. When it's your own health, people don't find them particularly empathetic.

@AdamRodmanMD @rloganjonesmd @andrewparsonsMD @DavidWuMDPhD @apalepu13 @BIDMC_Medicine @PeterBrodeurMD @BIDMChealth Will be waiting for reading those follow-ups, especially on empathy and trust.

@AnilMakam ... from patients, but also look up and integrate information from the EHR (and eventually other sources of info like wearables). I've been quite surprised at how rapidly the tech is improving, even if there are still a few computational breakthroughs to get through yet.

@AnilMakam Thanks @AnilMakam! It's coming faster than we think -- EHRs are messy (and locked down), and we have significant long-context issues now, but we're seeing routine improvements in this space. In the not too distant future, the systems will not only collect info directly ...

@ajchaise @BIDMC_Medicine Deskilling is notoriously hard to measure, but some of my colleagues at @BIDMC_Medicine are running a multicenter step wedge trial on scribing in an attempt to quantify deskilling.

@ajchaise @BIDMC_Medicine Yes!!! We have lots more planned -- and my own lab is doing a ton of HCI work (mostly in experimental settings), including looks at how to best present findings to improve clinician behavior.

@eclecticherbal @BIDMC_Medicine Thanks!!! Just a little side project :)

@AdamRodmanMD @BIDMC_Medicine Amazing work!

@Gabe__MD @BIDMC_Medicine (I'm not aware that this has ever been tested in the experimental literature, but I would imagine the same things happen if you start giving humans statistical inferences from prediction models)

@Gabe__MD @BIDMC_Medicine ... to traditional ML classification technologies, and even good ol' fashioned Bayesian inference). My gut feeling is that we'll be able to start getting more entropy signals when we a) include different types of information and b) different types of prediction modeling

“picking poindexter” by richard sargent (1959)

There is a lot of chatter lately about how to evaluate LLMs in healthcare. Back in October 2023, right after our Med-PaLM @Nature paper showed LLMs performing well on licensing exam-style questions, and as we watched AMIE excelling at diagnostic dialogue in simulated OSCEs, @alan_karthi and I realized that this was the easy part. To actually matter in medicine, we needed to graduate from synthetic benchmarks and simulations to the gold standard. We needed a prospective clinical trial as part of an actual doctor-patienti visit workflow. The problem is trials are rigorous, hard, and they take time. We needed a collaborator who was visionary enough to see the future of AI in healthcare as we did, but grounded enough to respect the process. All roads pointed to @AdamRodmanMD at BIDMC. It took more than two years, a massive stellar cross-functional team spanning both @GoogleDeepMind @GoogleResearch and Beth Israel organizations, and serious patience to run a first-of-its-kind study like this. Very glad to have been able to share results this past week (March 2026). More in the thread below from Adam 👇 As @alan_karthi keeps saying, everything in healthcare moves at the speed of us trust. Not everyone has the patience for the grueling, multi-year reality of clinical research. There is still a long way to go, but I am incredibly proud of our teams for doing the quiet, hard work required to bring medical AI into the real world - rigorously assessing safety, demonstrating utility and building evidence.

@rloganjonesmd @andrewparsonsMD @DavidWuMDPhD @apalepu13 @BIDMC_Medicine @PeterBrodeurMD @BIDMChealth @GoogleResearch @GoogleDeepMind @alan_karthi @HardyShakerman @vivnat and @apalepu13 from GDM and Google Research, @robwfields for his vision on how technology can improve and reshape primary care. There were a LOT of other helpers along the way -- and I'm thankful to all of you!!!

@rloganjonesmd @andrewparsonsMD @DavidWuMDPhD @apalepu13 @BIDMC_Medicine @PeterBrodeurMD @BIDMChealth I want to thank all my @BIDMC_Medicine @GoogleResearch and @GoogleDeepMind colleagues for making this study possible -- @PeterBrodeurMD and Jacob Koshy, for pushing this study forward, Marc Cohen for his leadership, support, and generally putting up with me...