KnowledgeLM Workshop

What left for humans with powerful coding agents? Right now, we evaluate agents mostly on Success Rate. But if fixing one simple issue by adding 2000 lines of spaghetti code, is that a win? I see the AI agents solve problems by endlessly adding new functions, growing into chaotic, million-line codebase that no humans can manage. But top engineers indeed care about the elegant simplicity beneath the mess (hello, Occam's Razor). What is left for humans? Might be just this. Yeah I became more and more excited about Abstraction. This paper is only about Abstracting and Reusing Skills, like macro functions. But might be a baby-step start.

Failure mode of LLM Agent RL training: reasoning shrinks, shorter and more similar. "diversity" has been a key to make LLM Agent RL training work, but I have always been wondering how to define "diversity". RAGEN used Entropy; RAGEN-v2 introduces Mutual Information (MI). The key insight comes from this decomposition: H(Z) = H(Z|X) + I(X;Z) So we can systemically classify four types of reasoning evolving patterns: - diverse reasoning - compression reasoning - entropy collapse - template collapse Top-p filtering: The most fascinating thing is that we find top-p filtering using reward variance is simple, but effective! We also try to explain this failure mode from gradient updates, check more at @wzenus 's threads 👇

1. What is a good exploration? More steps ≠ more information. Good exploration = prioritize information gain per step, so that forming a complete internal map of the world. It is about knowing what you don’t know, and choosing actions that reduce that uncertainty. We ask LLMs/VLMs the best action to take next: not to solve a task, not to maximize a task reward, but to reduce spatial uncertainty, to build an internal spatial belief of the world that can support future spatial reasoning.

RT @ManlingLi_: Huge congrats to @hengjinlp on being named an ACL Fellow! I still feel incredibly lucky to have been advised by her. Sub…

VAGEN poster at #NeurIPS: ⏲️11am-2pm Wed 📍Exhibit Hall C,D,E #5502 We look forward to discussing with you about: 1. MDP → POMDP 2. World modeling in agent internal belief 3. What is a good representation in agent internal belief for visual states? 4. How to use World Modeling to help reward shaping? 5. How to do turn-level critic learning? Drop by if you are interested in related topics!

Most VLM benchmarks watch the world; few ask how actions *change* it from a robot's eye. Embodied cognition tells us that intelligence isn't just watching – it's enacted through interaction. 👉We introduce ENACT: A benchmark that tests if VLMs can track the evolution of a home-scale environment from a robot's egocentric view. 🌐enact-embodied-cognition.github.io 📄enact-embodied-cognition.github.io/enact.pdf 1/N

Join her lab!

🧵 Academic job market season is almost here! There's so much rarely discussed—nutrition, mental and physical health, uncertainty, and more. I'm sharing my statements, essential blogs, and personal lessons here, with more to come in the upcoming weeks! ⬇️ (1/N)

What is the difference between spatial reasoning and text-based reasoning?

[KnowledgeLM @ ACL24] @lm_knowledge 🚨 Update: We've extended the paper submission deadline to May 30 to accommodate COLM review releasing. 📢 We welcome submissions of Finding papers to present at our workshop! We have lined up wonderful speakers, and we are eager to engage with you in Thailand! Meet with our organizers: @ZoeyLi20 @hengjinlp @megamor2 @eunsolc @mjqzhang @peterbhase @mohitban47 @preslav_nakov @Meng_CS @JiaweiHan Website: knowledgeable-lm.github.io

We will have a Best Paper Award, supported by @amazon. Appreciate it!!

@aclmeeting If you feel captivated by these problems, come join us at the Knowledge Language Model Workshop at ACL!

🚀 Knowledgeable Language Model Workshop at ACL24 @aclmeeting Are you ever curious about how much LLMs know? Do you ever wish that LLMs could become smarter with more knowledge? Or maybe you are thinking about removing certain facts from its memory? knowledgeable-lm.github.io