Synho Do, PhD

Accuracy ≠ safety. A new npj Digital Medicine paper introduces SA-ROC, a framework that asks the real question: when is it operationally safe to trust clinical AI? Instead of just reporting AUC, it defines: • “Safe zones” for autonomous action • A “gray zone” requiring human review • A metric to quantify the cost of indecision In one case study, the model with better AUC was actually less safe in practice. Maybe it’s time we stop equating performance with safety. nature.com/articles/s4174…

Microbiome research is a sea of data. We built a map. MINERVA—microbiome network research and visualization atlas: a scalable knowledge graph for mapping microbiome-disease associations academic.oup.com/bib/article/26… Platform: minervabio.org

Commentary on risk score thresholds to evaluate performance of a commercial #AI system for breast cancer detection and diagnosis doi.org/10.1148/ryai.2… @SynhoDo @MGHImaging #MammoRad #cancer #ML

Trade-offs between sensitivity and specificity when deploying a #BreastCancer AI system in clinical practice doi.org/10.1148/ryai.2… @SynhoDo @MGHImaging #cancer #DeepLearning #ML

Trade-offs between sensitivity and specificity when deploying a #BreastCancer AI system in clinical practice doi.org/10.1148/ryai.2… @SynhoDo @MGHImaging #MammoRad #DeepLearning #AI

Fuel your passion for healthcare innovation at Mass General Brigham MESH Core! Join us in Boston May 20-21 for two days of learning, networking, and key insights in our peer-reviewed innovation course. @MGBInnovation @MassGenBrigham Learn more: mesh2024.meshincubator.org

#AI needs to be safe for clinical care. @SynhoDo talks about novel approaches and challenges to making AI in medicine a reality now in @MGBInnovation Grand Rounds @MGHImaging @mgbmeshnetwork

Join us on Sept 13 for Mass General Brigham Innovation Grand Rounds. @SynhoDo will share his journey through research & commercial translation of several #ArtificialIntelligence-based products for clinical care @MassGenBrigham and beyond! Register: partners.zoom.us/webinar/regist…

@BcalvesNatlia @Radiology_AI Good question! With trustworthy AI that has UQ, we can automatically handle simple cases because we can rely on it. #RadAIchat

@SynhoDo @Radiology_AI But what is the performance of model B in the top 70-80% most "certain" cases? #RadAIchat

T5. How do you envision the impact of uncertainty quantification on #radiology practice and #research? #RadAIchat

@Radiology_AI It’s akin to an AI lie detector. #RadAIchat

@Radiology_AI AI with UQ and without UQ are very different. One is trustworthy because it's honest; the other isn't. #RadAIchat

@Radiology_AI It should be auditable. #RadAIchat, nature.com/articles/s4159…

T4. What are some challenges facing uncertainty quantification in #radiology #AI? How to address them? #RadAIchat

@Radiology_AI I'd prefer an honest AI (with uncertainty metrics) over a flashy AI (without uncertainty metrics) #RadAIchat

T3. How can uncertainty quantification optimize #AI interpretability, explainability, and trust? #RadAIchat

@Radiology_AI We want to make an AI that, like a curious child, honestly says 'I don't know' and asks questions instead of just guessing. #RadAIchat

@Radiology_AI We need to build AI that uses uncertainty metrics to show and admit its confidence. #RadAIchat

@Radiology_AI When AI's flaws are hidden, Uncertainty Metrics can reveal them. #RadAIchat

T2. Why is #AI uncertainty quantification important in #radiology? #RadAIchat

@Radiology_AI No algorithm is flawless, yet some excel in certain areas. We must identify their strengths and weaknesses, and uncertainty quantification helps achieve that. #RadAIChat

@MDohopolski @ShahriarFaghani @BcalvesNatlia Here is another one. #RadAIChat, nature.com/articles/s4155…

@ShahriarFaghani @BcalvesNatlia @SynhoDo Another good review as well. Maybe less comprehensive than the one above. review of uncertainty imerit.net/blog/a-compreh…. #RadAIChat

T1. Uncertainty quantification adds an extra layer to AI predictions where algorithms have the ability to communicate the confidence or trustworthiness associated with output likelihoods. This paper offers a great definition and insight nature.com/articles/s4174… #RadAIchat

@Radiology_AI AI uncertainty quantification is like the brakes on a fast-moving AI car. It helps us confirm the AI's results by comparing them to the prediction probability of its output. Uncertainty metrics should be independent of the prediction probabilities given by AI output. 😊#RadAIchat

Let's get started! T1. What is AI uncertainty quantification? #RadAIchat