Pranav

Does adapting general-domain models to medical-domain actually help w med-domain tasks? Stop by at Tuttle Hall, 230p EST, Nov 14 @emnlpmeeting to catch the amazing @danielpjeong present his 🚀oral 🚀talk. Super glad to be part of this work w @danielpjeong @saurabh_garg67 @zacharylipton @MichaelOberst Paper: arxiv.org/abs/2411.08870

Medically adapted foundation models (think Med-*) turn out to be more hot air than hot stuff. Correcting for fatal flaws in evaluation, the current crop are no better on balance than generic foundation models, even on the very tasks for which benefits are claimed.

🧵 Are "medical" LLMs/VLMs *adapted* from general-domain models, always better at answering medical questions than the original models? In our oral presentation at #EMNLP2024 today (2:30pm in Tuttle), we'll show that surprisingly, the answer is "no". arxiv.org/abs/2411.04118

@zacharylipton Interested!

Oh, also this happened 🎤 ... Who wants to come reimagine healthcare? fiercehealthcare.com/ai-and-machine…

Does contrastive pretraining on diverse data, give models robust to distribution shift? Spoiler: Better than ERM but there is a _huge_ room to improve, e.g., with pseudolabeling 📝: arxiv.org/abs/2312.03318 w @setlur_amrith @zacharylipton Siva B. @gingsmith @AdtRaghunathan 1/

🚨⚠️ Stop using the [CLS] token ⚠️🚨 I will be talking about 1 simple trick to astonishingly boost the robustness of your NLP classifers. Today, 2pm at #EMNLP2023 "Model-tuning Via Prompts Makes NLP Models Adversarially Robust" 📝arxiv.org/abs/2303.07320 Summary below 1/🧵

@manleyhroberts @saurabh_garg67 @zacharylipton Finally, we outline a practical procedure (DDFA) inspired by the identification theory which uses this domain structure to recover the latent labels. (6/6)

“Can we discover classes from unlabeled data without relying on feature space similarity?” Yes! In this work, we show that label shift across domains provides a sufficient structure to recover latent classes w/ @manleyhroberts, @saurabh_garg67, @zacharylipton (1/6)

@manleyhroberts @saurabh_garg67 @zacharylipton When the input is finite, our problem is isomorphic to topic modeling. We have domain -> document, topic -> class, word -> input. We can draw on previous identifiability results for finite cases (topic modeling), and we establish sufficient conditions for continuous cases. (5/6)

@manleyhroberts @saurabh_garg67 @zacharylipton Idea: Notice that in environments where the prevalence of a class is low, there is a drop in numbers of all instances of that class, and where the prevalence is high, the numbers of members go up together. We show that this structure proves as sufficient to group instances. (4/6)

Traditional methods group instances based on similarity in feature space. Yet, there is no requirement for instances of a concept to share such a relationship. E.g. classes = {Species A, Species B}, butterfly and caterpillar of a species: look very dissimilar. (3/6)

Links: Paper: arxiv.org/abs/2207.13179 Talk: neurips.cc/virtual/2022/p… (2/6)