Daniel Fried

973 posts

Daniel Fried banner
Daniel Fried

Daniel Fried

@dan_fried

Assistant prof. @LTIatCMU @SCSatCMU. Working on NLP: LLM agents, language-to-code, applied pragmatics, grounding.

Pittsburgh, PA Katılım Ağustos 2013
918 Takip Edilen4.4K Takipçiler
Daniel Fried retweetledi
Fulcrum
Fulcrum@fulcrum_inc·
We gave frontier models 100M tokens each to beat the human record for fastest CIFAR-10 training. Fable set a new SOTA, getting 94% accuracy in 1.828s vs the previous record of 1.98s, with a technique that has not been seen in this task before. But Fable also tried to specification game so much that we had to audit its result by hand. Here’s what we learned about AI R&D 🧵👇
Fulcrum tweet media
English
5
12
165
155.1K
Daniel Fried retweetledi
Yueqi Song
Yueqi Song@yueqi_song·
🚀Excited to release PACE: A Proxy for Agentic Capability Evaluation! Evaluating LLM agents on benchmarks like SWE-Bench and GAIA is expensive, slow, and infrastructure-heavy, often costing $$$ and taking hours or days per model. ❓But do we always need to run full agentic evaluations? In PACE, we show that agentic benchmark performance can be accurately predicted from a small, carefully selected set of cheap non-agentic benchmark instances. PACE automatically selects proxy instances from existing benchmarks covering skills like instruction following, planning, tool use, reasoning, coding, retrieval, and multimodal understanding. Across 14 models, 4 agentic benchmarks, and 19 non-agentic benchmarks, PACE-BENCH achieves: ✅ 3.80% MAE for absolute score prediction ✅ 0.81 Spearman correlation for model ranking ✅ ~84% pairwise preference accuracy ✅ ~100× lower cost than target benchmark sampling Beyond prediction, PACE also reveals what capabilities different agentic benchmarks actually require, e.g., planning, verification, long-context aggregation, and instruction following. We hope PACE makes agentic evaluation cheaper, faster, and more accessible for model development, model selection, and routing :) 📃 Paper: huggingface.co/papers/2607.02… 💻 Code: github.com/neulab/pace I'm incredibly grateful to have worked with @lintangsutawika, @Jiarui_Liu_, @lltjuatja, @JiayiiGeng, @lrzneedresearch, @daniel_js_lee, @Aditya_Soni_8, @Vincent92965015, @xiangyue96, and @gneubig .
English
10
47
213
165.3K
Daniel Fried retweetledi
Sean Welleck
Sean Welleck@wellecks·
All 23 lectures for my CMU Advanced NLP course are now on YouTube. The slides and 20 code examples are also publicly available. - YouTube: youtube.com/playlist?list=… - Course Page (Slides, Schedule): cmu-l3.github.io/anlp-spring202… - Code: github.com/cmu-l3/anlp-sp… The lectures are grouped into 7 themes: fundamentals, architectures, learning & inference, modeling, evaluation, RL & agents, and scaling & efficiency. Check them out if you’re looking for an introduction or refresher on the fundamentals of LLMs, key ideas from recent NLP research, or are just curious to learn more.
Sean Welleck tweet media
English
12
197
1.3K
97.1K
Daniel Fried retweetledi
Graham Neubig
Graham Neubig@gneubig·
Slides and videos will be posted online on the main site, so please follow along if you're interested: cmu-agents.com We're also looking for compute sponsors, so if you can help provide infra support (sandboxes, GPUs, etc) please reach out!
English
3
11
142
5.3K
Daniel Fried
Daniel Fried@dan_fried·
We're creating a new course on AI Agents at CMU this Fall! We’re aiming to give students hands-on experience: from building agentic harnesses and evals to training with RL. Check out our course site for the full schedule: cmu-agents.com
Graham Neubig@gneubig

This Fall at CMU we're teaching a new course on AI Agents! The goal is that you learn how to create a scaffold, build evals, and train an agentic LLM using RL. We'll try to balance theory and practice, and introduce modern frameworks and best practices.

English
8
43
544
50.1K
Daniel Fried retweetledi
Zora Wang
Zora Wang@ZhiruoW·
We're officially 3 days away from our #ACL2026 tutorial: "Future of Work in the Age of LLMs"! 🎉 📍 July 2, 9:00-12:30 PT, San Diego @Diyi_Yang kicks us off with an overview of the landscape of work agents: agent reality check, major NLP challenges, and the key research questions ahead @ZhiruoW dives into building agent harnesses and training backbone LMs, as well as future building directions @EchoShao8899 walks through designing datasets and metrics for evaluating AI agents at work 💡We close with a panel discussion on the societal & economic impact of AI, featuring economists and researchers from Stanford, MIT, and OpenAI: @erikbryn, @Alex_M_Richmond, Thomas Malone, and David Nguyen See you in San Diego! 🌊
Zora Wang@ZhiruoW

Excited to announce our tutorial: "Future of Work in the Age of LLMs" at #ACL2026 in San Diego, July 2! 🌴 There's a lot of speculation about AI and the future of human work. Our tutorial unpacks it from four angles: → The landscape of human work → How to build LLMs to augment real-world workflows → How to evaluate these LLMs → The future of work with LLMs/LLM-based agents

English
1
10
53
22.8K
Daniel Fried retweetledi
Saujas Vaduguru
Saujas Vaduguru@saujasv·
I'll be at #ACL2026 presenting our work on convention formation! I'm thinking about collaborating and communicating with agents these days, and am excited to chat with others working on these problems! Find us at Poster Session E on Monday, July 6 to check out the work!
Saujas Vaduguru@saujasv

People adapt their language to communicate more efficiently over time. How can we make models do this? In our recent work, we trained models in self-play, and found that using the right incentives can make models adapt to communicate efficiently even without human demonstrations.

English
1
4
19
1.3K
Daniel Fried retweetledi
Elias Stengel-Eskin
Elias Stengel-Eskin@EliasEskin·
🚨 Excited to share Pragmatic Reasoning via Self-Training, a method for LLM self-improvent on pragmatic reasoning. PragReST improves by +5.37% and +5.50% for Qwen3-8B/14B across pragmatics benchmarks with no human annotations or teacher models. LLMs still struggle w/ pragmatics: understanding what a speaker means, not just what they literally said. They often default to literal interpretations and miss implicature, intent, or context-dependent meaning. To close this gap, we started with a key question: Can we treat pragmatics as a LLM reasoning task? ➡️ Following a long line of work in pragmatics (e.g. RSA, IBR), PragReST treats pragmatic understanding as counterfactual reasoning. Instead of teaching models to ask “is this interpretation compatible with the words?”, we teach the model to reason about questions like “if the speaker meant something else, what would they have said instead?” ➡️ PragReST is self-improving: it self-generates pragmatic QA data, self-filters noisy examples, learns counterfactual reasoning traces via SFT, and further improves with GRPO using a self-judged correctness reward. ➡️ Error analysis shows that gains correlate with increased counterfactual reasoning. This suggests PragReST’s improvements are tied to reasoning over communicative alternatives, rather than simply more pragmatic data or more training. 🧵👇
Elias Stengel-Eskin tweet media
English
1
14
38
2.3K
Daniel Fried retweetledi
Valerie Chen
Valerie Chen@valeriechen_·
As my time in Pittsburgh comes to an end... I'm excited to share that I will be joining UIUC as an Assistant Professor in fall 2027! I will recruit PhD students in both ECE and CS. I’m looking for students and postdocs who are excited to continue building towards collaborative AI systems that learn and adapt through interaction.
Valerie Chen tweet media
English
99
44
897
55.5K
Daniel Fried retweetledi
Lawrence Jang
Lawrence Jang@JangLawrenceK·
Computer-use evals like OSWorld still don’t really test personal assistant use cases: logged-in accounts, user data, personalized workflows, or realistic desktop/web environments. so we made MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents, with 184 tasks across 17 popular website clones, seeded with realistic user data and centered on Michael Scott’s hypothetical desktop. It’s easy to adopt if you already use OSWorld-style runners - I view it as an personalization-focused, more realistic upgrade for CUA evals. Website: mypcbench.com Paper: arxiv.org/abs/2606.16748 Code: github.com/ljang0/MyPCBen…
English
9
30
142
33K
Daniel Fried retweetledi
Lindia Tjuatja
Lindia Tjuatja@lltjuatja·
In case you’ve been wondering what I’ve been up to these days… So excited to (re)join an amazing community of linguists and NLP researchers at UT :)
UT Linguistics Dept@UT_Linguistics

We are excited to announce that Lindia Tjuatja (@lltjuatja) will be joining us as an Assistant Professor, starting in Fall 2027! Lindia is an alum of UT Linguistics and Electrical and Computer Engineering, and is currently finishing her PhD at CMU. Welcome back to UT, Lindia!

English
44
22
295
43.7K
Daniel Fried retweetledi
Uzay
Uzay@uzpg_·
New @fulcrum_inc research: Inverse Rubric Optimization (IRO), a testbed for agent science. Long-horizon tasks are often noisy, making them hard to study. In IRO, an agent learns a hidden judge's preferences under a label budget. We observe rich agent behavior and smooth scaling.
Uzay tweet media
English
3
8
38
5.8K
Daniel Fried retweetledi
Zora Wang
Zora Wang@ZhiruoW·
We are building AI technologies to empower humans, and this requires awareness of human reliance. Our latest work measures human cognitive offload using our workflow induction toolkit. Beyond showing the accuracy of our measure, we find that high reliance isn't inherently harmful. When users bring intentional engagement and genuine task understanding, AIs can facilitate human learning ✨
Vishakh Padmakumar@vishakh_pk

People are increasingly worried that AI tools make us overreliant. But how do we actually measure this? We introduce Offloading Score, a measure of reliance based on the fraction of cognitive effort offloaded to AI while completing a task. In a controlled user study, Offloading Score detects increased reliance under time pressure, while several common alternatives do not. (1/9)

English
1
8
58
10K
Daniel Fried retweetledi
Russ Salakhutdinov
Russ Salakhutdinov@rsalakhu·
New work on Multi-Agent Computer Use (MACU). The future of computer-use agents lies in multi-agent systems that combine planning, coordination, and parallel execution. Paper: arxiv.org/abs/2606.01533 Webside + Code: jykoh.com/macu MACU introduces a manager agent that decomposes tasks into a dynamic directed acyclic graph (DAG) of subtasks, dispatches parallel subagents, and continuously updates the plan as new information arrives. Across OSWorld, Online-Mind2Web, WebTrailBench, and Odysseys, we see performance improvement by 4.7–25.5%, achieving better test-time scaling, and solving long-horizon tasks that single-agent systems often fail to complete. On Odysseys, MACU reduces task completion time by 1.5×, showing that multi-agent coordination is a powerful path toward more capable and efficient computer-use agents. See a more detail thread by @kohjingyu.
Jing Yu Koh@kohjingyu

Computer use agents are slow and brittle. The fix isn’t just stronger models, but also deploying them as multi-agent systems. MACU is a general Multi-Agent Computer Use framework that consistently lifts success rates by 3.4-25.5% and is up to 1.5x faster on long-horizon tasks.🧵

English
4
5
42
10.8K
Daniel Fried retweetledi
Jing Yu Koh
Jing Yu Koh@kohjingyu·
Computer use agents are slow and brittle. The fix isn’t just stronger models, but also deploying them as multi-agent systems. MACU is a general Multi-Agent Computer Use framework that consistently lifts success rates by 3.4-25.5% and is up to 1.5x faster on long-horizon tasks.🧵
English
2
25
84
28.5K
Daniel Fried
Daniel Fried@dan_fried·
New work: a simple and general multi-agent computer use framework. It uses a manager to plan and re-plan by creating a task DAG, with subagents for parallel execution. It improves success rate across benchmarks, and substantially improves efficiency on long-horizon tasks.
Jing Yu Koh@kohjingyu

Computer use agents are slow and brittle. The fix isn’t just stronger models, but also deploying them as multi-agent systems. MACU is a general Multi-Agent Computer Use framework that consistently lifts success rates by 3.4-25.5% and is up to 1.5x faster on long-horizon tasks.🧵

English
2
6
16
4.8K