Drew Breunig

@kdaigle @github Fascinating. Do these all share the same system prompt?

Hot take from looking at @github Copilot telemetry: benchmarks make coding models look wildly different. Production workflows make them look much more similar. 👀 We looked at 23M+ Copilot requests and examined one simple metric: code survivability.

There’s so many issues with prompt adherence with a mono prompt, especially with personas (gstack could learn from the RPG hackers on this…) Talked with someone yesterday who has Claude write, Codex do code review, let’s them go back and forth, then sorts changes by controversial.

Overall there are n thousand ways to throw more tokens at the problem and we need better ways to eval “review this plan and find all the things I didn’t think about” vs a 100+ instruction monolithic mega prompt

Tried plan-review-ceo from gstack yesterday. I’m not sure if this is good or bad, intentional or not intentional, but when I felt like pushing back on the agent*, something in my brain feels like I’m arguing with Garry directly 🤣 Anyways milestone 1 of a big feature shipping with RPI/QRSPI + Gstack shipping today, will report back * (which @garrytan had stated is part of the process - “your job is to know when the model is gassing you up and call it out” or something) I have some technical concerns with the sheer volume of instructions in the prompt and the amount of adherence you will actually get (@0xblacklight cited an interesting arxiv paper in post linked below) - I think we might be better served by a router that routes to specific modes, rather than explaining every single mode in a single monolithic prompt, but there’s tradeoffs to consider in plumbing and Ux for the end user. I think some may complain that it’s overly verbose and thoughtful and brings up things that are irrelevant but I actually think that’s good. I want a clean braindump of everything that might be relevant so I can edit and prune down to just what’s important

@pamelafox I don’t know, but will find out. Video will be released.

@dbreunig The DSpy meetup tonight was fantastic - even though I've never used DSpy, the case studies were full of general insights. Will speakers be sharing their slides?

@mjamei @Dropbox If you set your reflection_lm to a Claude, that is what DSPy/GEPA does, but in a systematic manner over many calls.

@Dropbox Did you try giving Claude the original prompt and the list of disagreements and ask it to “rewrite the prompt to address the disagreements “? I bet you get 90% of the lift right away.

How we used DSPy to turn our relevance judge into a measurable optimization loop, making it more reliable and scalable in Dropbox Dash.

It was inevitable. AI is unlike other technology purchases because AI has embedded judgment. x.com/Techmeme/statu…

A quick note on the Anthropic news this week: we're talking about allowed uses, but the key issue is embedded judgment. dbreunig.com/2026/03/01/the…

This is THE sleeper reason teams stick with DSPy.

@MingtaKaivo It is such a win. You go from: "A new model just dropped looks great, we should test it. But to see its potential we're going to have to tweak the prompt and it takes weeks," so you never do it. To: "A new model just dropped. Update the model string and run GEPA, why not?"

@dbreunig the model-swap-without-prompt-rewrite is the part I hadn't solved cleanly. at AudioWave we abstracted routing early but still had to retune prompts per model. GEPA optimizing the program itself instead of the prompt is the right layer to fix at

Incredible case study that's readable yet filled with practical approaches and tactics: ✔ Use GEPA to get near-frontier performance at 1/100th the cost. ✔ Quickly evaluate new models, without having to rewrite prompts. ✔ Evolve & maintain your programs w/o full rewrites

@MilksandMatcha Also matters which em-dash you’re using! The easy one to access in MacOS is short enough I always put spaces around it.

One way to tell if someone is using claude vs. chatGPT is by looking at the em-dashes. claude ALWAYS adds spaces before and after the em-dash, chatGPT doesn't. Claude: ' — ' ChatGPT: '—'

@swyx @NVIDIAGTC @NVIDIAAIDev

caption this

@FaroukAdeleke3 That is 100% a line @ChrisGPotts said to me that I think about all the time.

this reminds me of @dbreunig's chroma context engineering talk so many concepts and frameworks rooted in common sense that nobody cared about until andrej karpathy tweets about them on a random tuesday x.com/ivanbokii/stat…

--dangerously-skip-permissions

@Keshavatearth Try it yourself!

@dbreunig I wonder what's the score with newer models. all the models in this chart are non-reasoning models and no longer exist

Excellent “benchmark”. So many of the context engineering and general AI questions I get can be answered with, “Because we trained it to do this.” This is another good example.

@swyx @fabknowledge Doing the back of the envelope math...how many tokens will I have to spend to fit in a standard instance...to save $150/month...

@dbreunig @fabknowledge optimization is for ai winters

btw every single compute infra provider’s chart, including render competitors, is looking like this. something broke in Dec 2025 and everything is becoming computer. forget GPU shortage, forget Memory shortage, the @fabknowledge pod on LS was right, there is going to be a CPU shortage

exe.dev has the BEST coupon code execution I’ve ever seen: $ ssh [coupon-code]@exe.dev

Some questions that can be answered with this response: - Why does context rot occur? - Why are models so good at coding and meh at other things? - Why do models fight formatting requests? - Why are models good at reasoning with code? - Why do models try to not be shutdown during testing? - Why do models seem so human like? - Explain Moltbook?

@eliebakouch @kiranvodrahalli Oolong? Also likely wasn’t public during training. github.com/abertsch72/ool…

@kiranvodrahalli yes i agree! i was mainly impressed since previous anthropic models were not super good on this benchmark, and now it's sota also curious to get your take if you have some public long context eval that you like out of NIAH style one, maybe graphwalk or ones that are in HELMET?

wtf, they almost saturated MRCR v2, still unsure if this translates to solving context rot for good but i'm shocked