Sabitlenmiş Tweet
For the first paper, we did it under @fredsala's Foundational Model course. Took us a little under a month. We investigated Multimodal LLM's modality bias. Taking ideas from the LISTEN paper (arxiv.org/pdf/2510.10444). We asked ourselves: To what extent do multimodal LLMs bias towards text as opposed to other modalities? And what ablations or methods can we use to mitigate this, and in what cases?
We started out recreating the datasets, testing environments, and modality separations from the Mustard, PragCoT. and chain-of-thoughts papers. We used sarcasm as our north star to begin.
Halfway through we realized the limitations of our supporting papers, their datasets, and their reproducibility (as well as just bad PragCoT numbers) and switched to the AVUT dataset and did experiments with the @Alibaba_Qwen Qwen and @GoogleDeepMind Gemma Models.
We used all open-source models, the lowest cost compute we could get our hands on (local inference on our own compute, we're broke undergrads). And were able to get some really interesting results in the end.
Future work: Prompting is nice, but RL is where we really want to be (someone sponsor the compute please lol). And I'd personally love any opportunity to discuss this research with any AI/NLP serious researcher/PI who would be able to give good advice for next steps.
Link to the project page: pages.cs.wisc.edu/~samad/FMResea…
Link to the paper: pages.cs.wisc.edu/~samad/Researc…
English
