Richard Gao

MedGPT - A proposal on how we can super charge the healthcare industry with an AI diagnosis agent. It's a well known problem that doctors and nurses all around the world are in short supply. Education is long, pay and hours are bad. Yet, somehow hospitals are serving patients and the healthcare heroes are saving lives everyday. We can do better. I’ve recently come across this medium article. Where Dr Josh Tamayo-Sarver, MD, PhD struggles to get ChatGPT to help him in the ER. More so, he makes some rather uneducated statements that I feel are dangerous. This got me thinking. If we can train an LLM to be hyper specific on simply doing initial patient diagnosis based on as much information as possible, we could have one of the first true smart AI systems to remove pressure from the healthcare industry's bottlenecks. Let’s try to put this in layman's terms, as simple as possible. The LLM that we want to train, let’s call it MedGPT. Simple. Let’s imagine MedGPT trained to be as smart as GPT4, but fine-tuned and purposefully aligned to do medical diagnosis. We can do this, by importing and synthetically creating hundreds of millions of medical cases, and have the model train on that. Second layer, add RLHF (Reinforcement Learning from Human Feedback). The aligners should be doctors with a lot of expertise. This should give us a super aligned model for medical diagnosis purposes. We can assume the model to be truly multimodal (as GPT4) with the ability to interact using natural language as well as consume images or perhaps even videos as input and then have the ability to properly analyse what it sees. So let’s imagine that we now have a custom trained MedGPT, same base capacity as GPT4, but with the added context of being an absolute specialist on performing medical diagnosis. I can easily imagine a system, maybe based on the AutoGPT paradigm, where the MedGPT gets an objective to go figure out what's wrong with a patient. It’s given the patient's entire medical history by default as long term context. On top of that it’s getting the latest real-time information that has been gathered at the point of interaction, either at home in a proactive manner, or at the medical facility in a reactive/proactive scenario. This can include self-assessment, Apple Health data for a specific time period, or your entire recorded Apple Health data. It can include images, video and audio too. We can imagine the model being either multi-modal (as mentioned above), or having a huggingGPT like router, that can patch in any specific model to complete a goal/task. If it needs to analyse a chest x-ray, it uses a specific model for that. Or if it needs to listen to an audio recording of a heart, it uses another model. Maybe it needs to make an assessment on an ultrasound, it connects a different model, you get the point (all of this would be seamless and behind the scenes for the practitioner or patient). We can look at the initial model of MedGPT as the first line of defence. and then the subsequent models or layers acting as deeper knowledge specialists. The further out in the root system you go, the more specific the models and analysis can be made. At the centre we can see some sort of dispatcher that communicates back and forths between the different models. Much like huggingGPT. From the perspective of the medical professional the interface might be even simpler than ChatGPT. It might just be ONE additional button in an existing digital dashboard. The button initiates a complete cascading flow of events, similar to how AutoGPT takes just a few parameters and then starts executing to finish a desired goal/task. From a patient's perspective there might just be a simple self-check conversation similar to ChatGPT. Once the model feels confident that it has enough information to proceed, it provides the patient with a way to proceed like a button or a prompt asking for approval to start. The output of the combined efforts of the different models could then be as simple as “Everything is fine. You are healthy, and should not seek medical attention” to something like this: “this patient has a stroke and needs emergency treatment”. For example every year tens of thousands of patients are misdiagnosed for stroke and sepsis. I want to be clear, my level of understanding in the field of AI is limited, I’m not a PhD, I’m not a MD, I’m not a researcher. I’m just a person that likes connecting the dots, and looking at the big picture. Without AI, this proposed system would never work, or it would take too long to even start building something. But due to the innate ability for a LLM like GPT4 to take unstructured data and make sense of it, I can see how it could easily traverse different systems, data structures, and communicate back and forth between existing systems without much friction, if any at all. I’m currently not aware of anyone building something like this, and I’m not myself doing so. But I do understand that the implications of a system like this could be profound. Would love for someone that knows more than me to tell me this can or can’t be done. In my mind, it all seems very doable, and the incentives are clearly there! This was a note originally posted on my free newsletter - Inside my Head /Linus Ekenstam, March 15th 2023 * Disclaimer, MedGPT does not actually exist, it’s a made up idea, its up for grabs if you want it. 🧵