Jia Ming (إحسان)
13.8K posts

Jia Ming (إحسان)
@__jiaming__
Monash PhD student - mobile projective augmented reality. hacker, builder, speculator: jack of all trades, master of none. #DeepSeaMining

🔥Today, we are releasing one of the first visual reasoning benchmarks for autonomous AI diagnosis in healthcare! 🚀Introducing Radiology’s Last Exam 2.0 (RadLE 2.0) from @CRASHLabAI, an uncertainty-aware benchmark for autonomous diagnosis in radiology! ✅In the last few days, the AI frontier has moved significantly. @OpenAI launched GPT-5.6 Sol. @Meta launched Muse Spark 1.1. @xAI dropped Grok 4.5. 🙌We’ve benchmarked all frontier, open-source and medical VLMs in RadLE2.0 and the leaderboard is now LIVE! 🚨 Before AI models are handed autonomy, one question matters more than any accuracy score: Do they know when to STOP and hand over to a human? ⚠️ A confident wrong diagnosis is far more dangerous than an honest “I don’t know.” Yet most models are bad at admitting the latter! 🚀 We release five RadLE 2.0 Scores: Confidence Weighted, Reliability, Accuracy, Safety and Handover Readiness and we find that models from @OpenAI @AnthropicAI @MetaAI @GoogleDeepMind @xAI @nvidia @Alibaba_Qwen @MistralAI @MiniMax_AI all score very differently as they optimize for different metrics! 🚨But most importantly, NONE of the Models have been able to reach the average human expert baseline! ⚡️A thread on what we found and which models aced our metrics! Link to the leaderboard and technical report at the end of the thread!


come for the best model, stay because we don’t treat you with contempt


I still think about this a lot

AI is a good way to judge your own IQ - if you cannot tell the difference between Fable and Opus, then you are not smarter than Opus. If 5.6 and Fable feel the same to you, then you are not intellectually on their level.

everyone who got oneshotted by an earlier llm than me is a gullible moron with bad taste everyone who got oneshotted by a later llm than me is a cynical moron with no ability to extrapolate


🚨 GPT-5.6 Sol's juice values (thinking budgets) have been severely degraded compared to release day If Sol now feels faster and more "efficient", this is probably why Terra and Luna juice values aren't affected, so their thinking budgets are now higher than Sol's


I beg you to write down a prediction for AI will be like 1 year out and set a reminder on your phone to check back how it aged. Exponentials completely break human brains. You can get caught off guard by progress over and over again and don't realize it's going to keep happening












