Nicole Chen

@buildinpublic @jothi_ramaswamy and i want to learn more about ai token costs! wondering what methods there are to still ensure quality but also reduce costs per API request? what is the opinion of outsourcing cost reduction (rather than implementing it in-house)?

What are you working on this week?

@Andrey__HQ would love to read!

Just wrote one of the most interesting pieces of prose I've ever written in my life on where I think the world is headed with the agent-first economy. Would love to have at least 3-5 people review the "essay" and provide their honest feedback. Just dm me :)

@JessLozanoS @stripe whoo!! excited for many more two-weeks-before-launch grind sessions 🫶

Life update: I joined @Stripe's Comms team in New York! As someone who's loved working all across product, venture, and storytelling, this truly feels like a dream job. I'll be working across product and developer socials, comms and experimental projects. So excited to continue working closely with founders and users! If anyone is at Sessions in SF this week and wants to chat, my DMs have officially re-opened!

token theft is a huge problem but part of it is also bc companies haven't really cracked the right usage-based billing structure... the pay-as-you-go abuse pattern could be solved if ai businesses used top-ups and credit draw-downs

@jiaxin_pei prediction topping out at 0.39 correlation makes sense... token cost depends on tool outputs the agent hasn't seen yet. if the environment hasn't revealed it, the agent literally doesn't have the information needed to predict cost

Why are AI agents so expensive? Do more tokens actually lead to better performance? Which models are more token-efficient? Can agents predict their own token costs before execution? These were the questions bugging us, so we wrote a paper to find out. Excited to share "How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks", led by @Longju_Bai, with co-authors at Stanford, MIT, Google DeepMind, Microsoft AI, and All Hands AI. A few findings that surprised us: 🔹 Agentic coding tasks consume ~1000× more tokens than chat or reasoning workloads. And input tokens, not output, become the dominant cost driver, because each round re-feeds the entire trajectory back into the model. 🔹 More tokens ≠ better outcomes. Runs on the same task can vary by up to 30× in token use, and accuracy often peaks at intermediate cost. Beyond that, extra spending tends to reflect redundant exploration and does not bring further performance gain. 🔹 Models differ substantially in token efficiency. On the same successfully solved tasks, Kimi-K2 and Claude Sonnet-4.5 use roughly twice as many tokens as GPT-5.2. The gap becomes even larger when all the models fail. 🔹 Human-rated task difficulty weakly predicts actual cost. "Easy" tasks for humans can be surprisingly expensive for agents, and vice versa. The classic "Moravec's Paradox" is also true for coding agents! 🔹 Agents struggle to predict their own costs. Self-prediction correlations top out around 0.39, and every model we tested systematically underestimates what a task will cost. Result-based pricing still has a long way to go when we cannot even figure out the token cost beforehand. Together, these results suggest that cost prediction is a genuinely challenging task for current agents. We think this opens up real research questions around self-modeling, calibrated cost estimation, and pricing mechanisms that work under residual uncertainty. Huge thanks to my collaborators: @Longju_Bai, Zhemin Huang, @sunjiao123sun_ , @xingyaow_ , @radamihalcea , @erikbryn , @alex_pentland paper: arxiv.org/abs/2604.22750 website: longjubai.github.io/agent_token_co…