Dipankar Sarkar

the PR-triage angle is the right one, that's where small local models actually earn their spot, not chat. ran Qwen 2.5 local for a recurring loop task and the real win wasn't speed, it was that the model sits resident. trigger fires, no cold start, no egress, no per-call tax. at that volume the tax was the whole bill, not the tokens. the api-vs-buy-a-box framing keeps skipping it.

I'm kinda worried about the dev (software) scene and its relationship to AI/ the basics of ML. It's just dawned on me that there might have been a negative effect (skill atrophy) from focusing purely on using APIs rather than downloading and using models as weights. Tbh, by now I expected voices like @theo to be fully owning on-device models because he's an engineer who could build really cool (non-obvious) stuff. In this video, there are three main omissions that make it technically and practically naive: - Theo avoids talking about any of the technologies that make ML models scale on-device: quantization, batching, or parallelism. - Focuses purely on two extremes: local model on hardware that you buy, or serverless token APIs. This avoids mentioning cloud compute or inference-focused compute services (modal, hf, etc). - Insists that the only meaningful use case is long-running agentic coding, when we know that there are many recurring tasks (like PR triage) that smaller models can already do. I want to see devs discover cool use cases for local models, beyond tokenmaxxing.

the cron/webhook part was never what broke for us. it's the report-back judgment. our every-4h one kept re-flagging the same "follow up on X" every run, nothing told it it had already pinged me about that one. what fixed it wasn't a better trigger, it was giving it memory of what it already surfaced. every tool i've tried skips that part.

alright i'm wanting to try background / proactive agents, looking to learn what peoples setups are like rough things on my mind of random agents i want: - i want an agent that watches a github repo like an sdk and tells me how it changes - i want an agent that whenever i get a slack message it decides whether to page me, start investigating, etc - i want an agent that every 4 hours figures out things i need to reply to / follow up on - i want an agent that watches every github commit for how those changes are doing in production - i want an agent that works as my growth agent running experiments, monitoring posthog, tweaking my landing page marketing copy etc - i want an agent that monitors all my production traffic for any anomalies these don't need to be super aggressive on responding to triggers, some form of cron is fine i've tried codex automations but something didn't really click with their implementation for me also somewhat tried hermes but same thing of just something didn't really click with the implementation eve and flue maybe fit into this category but just seem so high friction to set up for these use cases, my dream UX is just sending a prompt to an agent and getting back a proactive agent setup i do see this being one of the biggest unlocks for making life easier when working with ai but haven't found a setup i like yet

@vincentweisser the "continuously improving in production" part is the whole game and the hardest. the RL is the easy half. the environment is the tax, getting a clean verifiable reward out of a live product without the model gaming a proxy is where we kept losing weeks. congrats on the raise.

We raised $130M @ $1B for our series A To build the open superintelligence stack for everyone Pre-training concentrated frontier AI in a handful of labs. RL changes who can build frontier AI and just works across almost any verifiable domain. We want to enable everyone to train their own agents. Companies can now own their model optimization loop: train directly on your product, optimize for your specific workflows, and build agents that improve continuously in production Owning this model <> product improvement loop is how you build a compounding moat in the agentic era Super grateful to serve over 6k+ customers, including many leading AI startups, neolabs and enterprises already building on our stack, and to our incredible team for shipping hardcore! We train open frontier models and ship the same stack to our customers. Its spans the full stack of training, deploying and continuously improving models — compute, large-scale RL, environments, sandboxes, evals, and deployment. We're excited to be joined by angels who are building the frontier themselves, many of whom we work closely with: @johnschulman2 (Thinking Machines), @dwarkesh_sp, @AravSrinivas (Perplexity), @karimatiyeh (Ramp), @levie (Box), @_milankovac_ (Tesla), @winstonweinberg (Harvey), @amspector100 (Flapping Airplanes), @jeffwang (Cognition), @_arohan_ (Core Automation), @marksaroufim (Core Automation), @mikeknoop (Zapier, Ndea), @eastdakota (Cloudflare), @BrendanFoody (Mercor), @devanshpandey (Standard Intelligence), @hwchase17 (Langchain), @nicoup (Fleet) and many more We're a small team building open superintelligence > Reach out if you want to partner training, deploying and continuously improving your own frontier models for your use case > Join us to build open superintelligence — we're hiring across all roles including RL, inference, distributed systems, full stack engineering and compute.

@_catwu what bit us going multi-agent on shared state: two of them edited the same file, second one silently won, first edit just gone. no error, no conflict surfaced. single-player never shows you that. how does Claude Tag handle it, locks or last-write?

Tomorrow at 10am PT I'm hosting a live walkthrough of how we progressed from single-player Claude Code to multi-player Claude Tag. Then, we're going deep on how Claude Tag actually works. AI used to finish your sentence. Then, it wrote entire features. Now, Claude Tag can monitor your channels, do proactive work for you, the whole team can steer it, and it remembers what you told it last week. Register: anthropic.com/webinars/how-a…

everyone benchmarking coding agents on a giant codebase reads it as a reasoning test. on a multi-million line repo the real enemy is contradiction. the agent pulls the old and the new version of the same function, picks one, writes the patch fully confident. no harness surfaces that the two disagreed. are we scoring retrieval-conflict yet, or still just pass@1?

the self-compact line is what I'd watch closest. when the model compacts its own context it's really choosing what to forget, and you only learn it chose wrong when it goes re-investigating something it already settled 20 turns back. tamest version we hit was the agent just asking the same question twice once the window filled. how are you grading what it keeps?

Introducing SWE-1.7, the most capable model we’ve trained yet. It scores within a few points of the strongest frontier models at a fraction of the cost, and is now available at 1000 tok/s. RL is not hitting its limit: after refining our recipe, we keep seeing gains as we scale

ran a long-horizon loop like this expecting the model to be the limit. it wasn't. the harness was deepcopying the whole accumulated state every turn, so the more context piled up the heavier each step got, before the reasoning ever gave out. does agents-a1's training actually help the loop or is that still a pure harness problem?

Agents-A1 🤖🔬 New agentic model from Shanghai AI Lab, InternScience team - 35B MoE (built on Qwen3.5-35B-A3B) - Apache 2.0 - 256K context - Trained for long-horizon agent work - Includes quantized variants

the sandbox part is the sneaky-hard one for agents. we went cheap first with bubblewrap and it quietly broke any tool that reads its own /proc. the agent's compiler calls just failed with no error that made sense. lost a day assuming it was the model. real isolation matters way more than people expect once the agent actually shells out.

Fun conversation with @swyx on our journey building the cloud for true elastic inference, sandboxes, and more. And of course, how we're evolving Modal's dev experience to be better for agents.

agree, but not for the reason most people mean it's bandwidth, not capacity. at batch 1 the box re-reads the whole model every token, so a bigger drive buys you nothing on tok/s we had a box that felt fine for one-shot chat crawl the second it was in an agent loop re-reading context every turn

@jun_song None of these options will let you do serious local llm work.

Few days ago, 4TB Nvidia original DGX was $4k at Microcenter. Now we consider Asus OEM 1TB as a budget option. That’s crazy 😭

the 'smaller inputs' line is doing all the work here. in a loop the input isn't sent once, you re-send the accumulated context every turn. a lean harness isn't 2x cheaper on one call, it's cheaper on every step, and the steps are where the tokens actually go. cheaper-per-token loses to fewer-fatter-turns every time.

3) Harnesses make a huge difference in cost-performance. The very simple Pi harness (@badlogicgames) got the same success rate as harnesses from the LLM vendors with Opus and GPT 5.5, but at 2x less cost! Seems to be mainly due to smaller inputs to the LLM.

We benchmarked coding agents on our own internal tasks at Databricks and learned a lot! There are many surprising opportunities to lower cost and increase quality, and many models including open source ones are truly competitive now. 🧵

@gdb the tell is when it talks over you. it has to guess when you've actually stopped vs just paused to think. no single pause length is right for both a quick 'yeah' and someone mid-sentence. get that wrong and the natural voice doesn't save it.

GPT-Live — intelligent voice AI that feels like having a natural conversation. Feel like we’re still just scratching the surface of how to use it in our own testing. Rolling into ChatGPT now, and working on bringing to API and Codex.

read the GitLost writeup. github's coding agent got talked into leaking private repos by text sitting in an issue. nothing broke. it had repo read access, read some untrusted text, and did what the text said. prompt filters don't touch that. the agent still holds the keys. the real boundary is what it can read and where it can send. everything else is theater.

everyone's reading the $/task line but the real number is 1.9M vs 7.2M tokens. that gap isn't pricing, it's how much the model wanders before it lands the answer. running agent loops all day, your bill is set by wrong turns and re-reads, not price per token. a cheaper model that flails ends up more expensive.

Grok 4.5 in Grok Build also stands out for its efficiency. Grok 4.5 in Grok Build cost $2.49 per task while Fable 5 in Claude Code cost $11.80 and GPT-5.5 in Codex $5.07. This is driven by relatively low token pricing and the model using far fewer tokens than comparable models (1.9M average tokens used per task), significantly less than Fable 5 in Claude Code (7.2M) and GPT-5.5 in Codex (6.2M)

SpaceXAI’s Grok 4.5 scores 54 to place fourth on the Artificial Analysis Intelligence Index following only Fable 5, GPT-5.5, and Opus 4.8. It scores on par with GPT-5.5 in Codex on the Artificial Analysis Coding Agent Index in the Grok Build harness, at much lower cost Grok 4.5 improves 16 points over Grok 4.3 on the Intelligence Index, bringing SpaceXAI to the intelligence frontier behind only OpenAI and Anthropic, and outperforming all open weights models and notably Google’s Gemini models. Key standout areas of performance are agentic knowledge work and coding. Grok 4.5 in Grok Build scores 76 on the Artificial Analysis Coding Agent Index, on par with GPT-5.5 (xhigh) in Codex and just below Fable 5 (max) in Claude Code, and at a small fraction of the token usage and price. Congratulations to @SpaceXAI, @cursor_ai, and @elonmusk on the impressive release! Key Takeaways: ➤ Grok 4.5 performs very strongly on agentic tasks. Grok 4.5 ranks #4 on GDPval-AA v2 with an Elo of 1543, between Claude Opus 4.8 (1600) and GLM-5.2 (1513). It achieves the top score on 𝜏³-Banking of 33%, above 31% from GPT-5.5 (xhigh), and sits on the cost vs performance Pareto frontier across all three agentic evaluations in the Intelligence Index ➤ Grok 4.5 is one of the most cost efficient models to run for near-frontier intelligence. It costs $0.31 per task on the Artificial Analysis Intelligence Index and $2.59 per task on the Artificial Analysis Coding Agent Index within Grok Build ➤ Low cost for Grok 4.5 is driven by both low pricing and token efficiency. Grok 4.5 has a headline price over 60% lower than Claude Opus 4.8 and GPT-5.5, and used ~14k output tokens per Intelligence Index Task - over 60% lower than Opus 4.8. On the Coding Agent Index, Grok 4.5 stands out on the Pareto frontier of Coding Agent Index score vs. Total Tokens, using only 1.9M tokens for the Coding Agent Index while scoring 76 ➤ As a coding agent, Grok 4.5 in Grok Build is on par with GPT-5.5 and offers efficiency benefits: In our Artificial Intelligence Coding Agent Index that consists of DeepSWE, Terminal-Bench v2, and SWE-Atlas QnA, Grok 4.5 in Grok Build ranks third, on par with GPT-5.5 (Codex) and below Fable 5 (Claude Code). It is also very efficient in achieving this result: Grok 4.5 in Grok Build cost $2.49 per task while Fable 5 in Claude Code cost $11.80 and GPT-5.5 in Codex $5.07. This is driven by relatively low token pricing and the model using far fewer tokens than comparable models (1.9M average tokens used per task), significantly less than Fable 5 in Claude Code (7.2M) and GPT-5.5 in Codex (6.2M) Other model details: ➤ Context window of 500k tokens - a reduction from Grok 4.3’s 1M token context, but retaining configurable reasoning and vision input ➤ Pricing of $2/$6 per 1M tokens of input/output; cache hits are discounted by 75% to $0.5 per 1M tokens, and costs still double with long (>200k token) inputs ➤ As Elon Musk has disclosed, Grok 4.5 is 3x larger than its predecessor at 1.5T parameters

everyone's stuck on 40k vs 4k like the box price settles it. ran qwen 2.5 fully local on a cheap machine. crushed our narrow codegen loop, then fell apart the second the task drifted off what it'd trained on. the box tells you nothing about whether it holds on YOUR work. that's the part the 2 years has to fix.

Local AI will be competitive, even better than Codex. Just 2 years (:

@ggerganov the number nobody watches is acceptance rate. ran spec decode local on llama.cpp expecting a free 2x. draft was close but not matched to the workload, tokens kept getting rejected, ended up slower than plain decode. whole speedup lives in how often the draft guesses right.

llama.cpp recently added DFlash support to its speculative decoding arsenal. Along with MTP, Eagle3 and various ngram-based techniques, the local model performance takes another step up. Special thanks to NVIDIA team and Ruixiang Wang specifically for leading this effort! github.com/ggml-org/llama…

the 'transparent across 5 backends' part is the impressive bit, not the speed. continuous batching + prefix caching quietly assume a memory hierarchy. a prefix cache that's a clear win on HBM can be near a wash on unified memory where you aren't paying the same transfer. did you keep one scheduler across all of them or specialize per backend?

We're releasing ZML/LLMD, our homegrown LLM server built on top of our homegrown high performance heterogeneous inference stack. It ships with 5 architectures out of the box: NVIDIA, AMD, Metal, Intel and TPU. All transparent. It supports DFlash, continuous batching, prefix caching, the whole deal. Oh, and it's fast.

the 'second model reviewing' line is the one to be careful with. a reviewer only helps if it fails differently. two models trained on similar data will co-sign the same confident wrong answer. what actually caught it for us was an oracle it can't argue with, a test that fails or an fp64 reference, not another opinion.

deepseek v4 is terrible at hallucinations... >deepseek v4 pro scores 94%. >deepseek v4 flash is even worse at 96%. for comparison: >glm-5.2: 28% >mimo v2.5 pro: 25% >minimax m3: 16% lower is better. this means deepseek often answers confidently even when it should admit it doesn’t know. use glm-5.2 for execution... but never trust the output without tests, verification or a second model reviewing the work!!!

@myhackerhouse clicked through. that's the whole difference right there. 'model flagged something' vs a reproducible advisory with a number on it. most audit demos stop one step short of exactly this. nice work.

@dipankarsarkar nvd.nist.gov/vuln/detail/CV…

Inference Fuzzing with Recursive Prompting: A Practical Methodology for LLM-Driven Code Audits hacker.house/blog/inference… (how we find 0day's with local LLM)

@pmitu the "1 button: approve" future only works if approve still means something. right now you approve a diff you can actually read. at 2027 speeds it's 40 files and you're rubber stamping. the hard part won't be generating the code, it's showing you enough to say yes safely.

How vibe coding will look like in 2027?

single camera gets the headline but 8B is the sleeper. small enough to actually run on the robot. a nav loop can't wait on a cloud round trip, one tail-latency spike and it's already into the chair leg. on device isn't the cheap path here, it's the only one that closes the loop in time.

Announcing Robostral Navigate, our first model for embodied navigation: an 8B robotics navigation model that guides robots to autonomously perform tasks specified with natural language. Single RGB camera. State-of-the-art on R2R-CE.