Andreas Stephan

📑 Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges Explore recent advances in text generation since 2017, focusing on five core sub-tasks and highlighting key research gaps. 🔗 Read the paper: arxiv.org/abs/2405.15604 🧵 1/8

Text-Guided Image Clustering This recent paper introduces a new way to cluster images using generated text descriptions which exceeds many state-of-the-art models using visual features directly from the pixel representation. The method is flexible, working with various image-to-text models like BLIP-2, Flamingo, and GIT. Key findings: 1. Generated captions often outperform image embeddings for clustering across 8 diverse datasets, including standard benchmarks and more challenging real-world scenarios. 2. Injecting task/domain knowledge via prompts to visual QA models further improves results, allowing for targeted clustering. Specifically, using text-guided image clustering, we can cluster images into different interpretable aspects (e.g., cluster based on the action in the scene or based on the objects in the scene) 3. The approach enables keyword-based cluster explanations, which enhanced interpretability. 4. Text representations can serve as a better proxy for "meaningful" image clusters than raw pixels, aligning with human-annotated groupings. 5. Performance improves with multiple caption samples per image, especially for TF-IDF representations. This research challenges traditional image clustering approaches and opens new avenues for leveraging multimodal models in unsupervised learning tasks. My question is: How deeply are language representations intertwined with our visual understanding? Visual features often have a corresponding language representation that we describe it with (words with cognitive meaning as well as their definitions). Psychological studies show there are close ties between visual and language understanding which are two key abilities to human intelligence. I am curious to see more research on that connection and how language and vision features can complement one another in machine learning research. Paper: aclanthology.org/2024.eacl-long… Code: github.com/AndSt/text_gui…

The research group Data Mining and Machine Learning at the University of Vienna is looking for a Postdoctoral Researcher in Natural Language Processing. For more details see: jobs.univie.ac.at/job/Postdoctor…

At 11AM I will talk at the “Factual Content in NLP” session about our work “Counterfactual Reasoning with Knowledge Graph Embeddings (aclanthology.org/2024.eacl-long…). Great work led by @lena_zellinger.

At 2PM I will talk at the “Multimodality” session about our work “Text-Guided Image Clustering” (aclanthology.org/2024.eacl-long…). Joint work with @lumik777 @kevin_sidak @jpwahle

Hello network! I'm thrilled to be attending EACL this week. Let’s have a chat! 😁 If you’re interested in the topics, I will give two oral presentations on Monday:

Thanks Collin and @janleike for your very interesting work. You might find our #ACL2023 paper (aclanthology.org/2023.acl-long.…) interesting as we critically question recent progress in weakly supervised learning in NLP. Some findings related to your work below. 1/N 🧵

Last week, Ben and @andst_link, @anasedova and @vassilikiKou had an invited tutorial about weak supervision at @Aalborg_uni in Copenhagen 🇩🇰 Big thanks to @johannesbjerva for the invitation!

Super happy to share that we won the Theme Paper Award at #ACL2023NLP! 🥳 Especially congrats to the project lead, the brain and muscles of the operation, @cs_dawei, for his determination and hard work! Unbelievable 😀

📢 Check out our new #ACL2023 paper! "Weaker Than You Think: A Critical Look at Weakly Supervised Learning" ⚠ Want to apply weak supervision to solve your real-world tasks? Wait a second! ⚠ arxiv.org/abs/2305.17442 🧵 1/N

Thrilled that the paper made by our #DoCSunivie student Andreas Stephan (@andst_link), #DoCSunivie student Vasiliki Kougia (@vassilikiKou) and Prof. Benjamin Roth, was presented at the @emnlpmeeting conference in Abu Dhabi by Andreas! 👏 arxiv.org/abs/2210.13898 📝 @csunivie

A throwback to our #ACL2022 visit, our first live conference since Covid started 🥳 So many great personal and research-related impressions! Here, @andst_link and Ben present the poster for WeaNF: Weak Supervision using Normalizing Flows at #Repl4NLP 📜 openreview.net/pdf?id=lfz6pM0…