Robert L. Logan IV

🚨Please repost🔃 Actively exploring new opportunities that align with my passions and skills, ideally a research scientist position with a spirit of open source/science DM me if you know such opportunities! About me yoshitomo-matsubara.net Long ver. linkedin.com/posts/yoshitom…

How diverse are the outputs of text-to-image models and how can we measure that? In our new work, we propose a measure based on LLMs and Visual-QA (VQA), and show NONE of the 12 models we experiment with are diverse. 🧵 1/11

Retiring the US debt would retire the US dollar: We don't tax billionaires to fight the national debt. We tax billionaires to fight BILLIONAIRES. twitter.com/doctorow/statu… 3/

Alignment is necessary for LLMs, but do we need to train aligned versions for all model sizes in every model family? 🧐 We introduce 🚀Nudging, a training-free approach that aligns any base model by injecting a few nudging tokens at inference time. 🌐fywalter.github.io/nudging/ 📜arxiv.org/pdf/2410.09300 1/7

prompt optimization is such a hard problem, and the algorithms are dumb. the fact that they work at all is baffling, and people don't talk about it enough for those who don't know, the prompt optimization problem is argmax_{x} ℓ(y | x; θ) where x is some prompt and ℓ is the loss for output y under language model θ. solving this problem exactly requires enumerating all possible prompts x... ...and the space of possible 10-token prompts in a 50k vocab is 50,000^10. yet people regularly find solutions to these problems with 100-token prompts. how can this be possible? the most popular algorithm (AutoPrompt/GCG) is basically "while true: rank a bunch of token-swaps, and then take the top one" and yet this works somehow. in fact, this process is "good enough" that everyone uses it and has been cited thousands of times. truly boggles the mind

Excited to share Just read twice: going beyond causal language modeling to close quality gaps between efficient recurrent models and attention-based models!! There’s so much recent progress on recurrent architectures, which are dramatically more memory efficient and asymptotically faster than attention 💨 But there’s no free lunch 🥪 these models can’t fit all the information from long contexts into the limited memory, degrading in-context learning quality. Is all lost?

Video: An attempt to explain Diffusion LMs in 15 minutes. Several people have noted that they have bounced off this topic before because the math felt a bit perplexing. Think it's a good time to revisit. youtu.be/WjAUX23vgfg

Lots of discourse around long-context language models (LCLMs) subsuming RAG and retrieval but how close are we to this paradigm shift? Introducing LOFT a 1 million token benchmark spanning 6 tasks & 35 datasets to test LCLMs’ ability to do in-context retrieval & reasoning [1/10]

@zouharvi 90% of the text GPT5 generates will be written at 1 am on nights of major conference deadlines.

GPT-5 will go to only a vaguely related talk somewhere in the department, motivated solely by the complimentary snacks.

LLMs can have a huge impact on gaming🎮, but not in the way you may think! Number 10 may surprise you!😱 I wrote up some of my thoughts and ideas about the use of LLMs in games. There's a lot more to it than just generative AI NPCs. 📖kolbynottingham.com/llms-and-games

Are Models Biased on Text without Gender-related Language? We've investigated the answer to this question in our recent ICLR 2024 paper! 🧵 Paper - arxiv.org/abs/2405.00588 🎥 (5 min) Video - youtube.com/watch?v=gmqBoB… 🌐 Landing page - ucinlp.github.io/unstereo-eval/

I'm happy to announce AI research internship opportunities at @Dataminr! Topics range from LLMs, #NLProc , CV, HCI, to #aiforgood , and more. Prior internships have resulted in papers at conferences such as @RealAAAI, @aclmeeting and @CVPRConf. dataminr.wd12.myworkdayjobs.com/Dataminr/job/U…

🔃Please RT🔃 My 1st TODO item this year is getting a new exciting research job! Please DM me if open source/science matters to your team e.g., open-sourcing research projects & publishing papers Long ver. linkedin.com/posts/yoshitom… Webpage: yoshitomo-matsubara.net Thanks🙏

En route to Singapore 🇸🇬 for #EMNLP2023! I'll present (virtual poster Sun @ 9am) our work on training-free example selection for In-Context Learning. Come say hi if interested in chatting about ICL, reasoning, and generalization with LLMs or even exploring Singapore!

(1/6) 🚀🚀 Thrilled that our paper arxiv.org/abs/2305.14907 has been accepted to #EMNLP2023 findings! 🎉 tl;dr: Selecting in-context examples that together cover all the salient aspects of the test input yields training-free methods that beat even trained SoTA methods! 💪🔥

Dear ACL community, ACL is considering multiple proposals to change its anonymity period policy. It seeks immediate feedback from the community about the proposed changes. Please add your voice until Friday, September 22nd (AOE): aclweb.org/portal/content… #NLProc

@kalpeshk2011 @MohitIyyer @andrewmccallum @HamedZamani @YejinChoinka @Google Congrats Kalpesh!

Happy to share that I've defended my PhD thesis! A big thank you to my advisor @MohitIyyer and my committee @andrewmccallum, @HamedZamani and @YejinChoinka for all their support and feedback! Next step: I am super excited to be joining @Google Bard as a Research Scientist!

New! DHS has added 8 new "fields of study" to the STEM OPT eligible list: - Landscape architecture - Institutional research - Mechatronics - Composite materials tech - Linguistics and comp sci - Developmental and adolescent psych - Geospatial int. - Demography and pop studies.

Happy to share this work won honorable mention for best paper at the TSRML workshop at NeurIPS 😀🎉 @hima_lakkaraju @sameer_ @SatyaXploringAI