Chacha Chen

✨ New paper at #FAccT2023 next week ✨ Machine Explanations and Human Understanding arxiv.org/abs/2202.04092 Explanations are hypothesized to improve human understanding of machine learning models. However, empirical studies have found mixed and even negative results.

🚨Why can’t Transformers learn multiplication?🧮 Even with billions of params, models struggle with multi-digit multiplication. In our new work, we reverse-engineer two models: a standard fine-tuned (SFT), and an implicit chain-of-thought (ICoT) model to see why. Read on! 1/n 🧵

1/n You may know that large language models (LLMs) can be biased in their decision-making, but ever wondered how those biases are encoded internally and whether we can surgically remove them?

🚨 New paper alert 🚨 Ever asked an LLM-as-Marilyn Monroe who the US president was in 2000? 🤔 Should the LLM answer at all? We call these clashes Concept Incongruence. Read on! ⬇️ 1/n 🧵

I’m going to present at the poster session on Friday morning. Come find me! 😃😃

CaseSumm is publicly available on HuggingFace! We hope this dataset enables: - Better evaluation of long-context summarization - Research on legal language understanding - Development of more accurate & reliable legal AI tools Dataset: huggingface.co/datasets/Chica… I'll be presenting this work at 2pm and will be around until Sunday. Please reach out if you're interested in this line of work - would love to connect in person or virtually!

🚀🚀🚀Excited to share our latest work: HypoBench, a systematic benchmark for evaluating LLM-based hypothesis generation methods!

Just got accepted to NAACL!

Even ChatGPT knows. It is very hard to think of a positive spin for Rosalind here. A shocking display of ignorance and arrogance.

and I especially like the human evaluation we did. The ground truth reports are 100% clinically usable, although they slightly fall behind the fine-tuned Llama in terms of clarity and readability.

1/ ✨ Can Multimodal LLMs generate radiology reports from chest X-ray images? In our recent evaluation paper, we show: not yet.

the last row shows GPT-4v's gap with SOTA models

Joint work with @ChicagoHAI and @bmervakMD

We’re excited to present this work at @SymposiumML4H next month in Vancouver. Happy to chat and feel free to reach out 😊.

5/ We also conducted a human reader study to compare the quality and readability of AI-generated and human written reports. Paper: arxiv.org/abs/2407.12176 Code:github.com/ChicagoHAI/cxr… Altogether, our findings cast doubt on the viability of using GPT-4V in a radiology workflow.

4/ Our Evaluation Overview: We started with direct report generation using different prompting strategies, but found the model failed terribly. To understand the low performance, we decomposed the task into two steps: image reasoning and report synthesis.

I will be in Rio for #FAccT2024 next week. Hit me up if you will also be there in person and would like to chat!🏖️🍻😎

@yuwen_lu_ maybe not even fun 🥹

hci research, or most research, is just a fancy way of saying “fun stuff that almost nobody will pay you to do”

@manoelribeiro @PrincetonCS @PrincetonCITP HUUUUGE congrats! ㊗️

I'm thrilled to announce that I'll join @PrincetonCS/@PrincetonCITP as an assistant professor in Spring 2025 — can't wait to join this amazing group of researchers. I will continue to work on online platforms / the societal impact of AI. If you’d like to collaborate, reach out!