Under-discussed problem right now with most frontier coding models. A leading contributor to slop and incidental complexity and daily pain Great read @vansickn !

YES I have to spread that in all my prompts, garbage like: "implementation work must replace vestigial object-shape assumptions outright. Do not preserve compatibility in code APIs. Write database migrations. No `TODO`/`pending`/`xit`/`skip` markers, no "implementation deferred" stubs, no dead buttons or unreachable routes. Do not defer something until some "later phase". Do it now." yada yada yada

@MattRogish @vansickn Dead on. It's comical the extent you resort to conventionally bad advice to get them to comply Imagine being a senior eng in 2017 telling your juniors "Never worry about backwards compatibility" "Break interfaces aggressively and update all callers"

Ha! Yes, it must've been trained over and over on "don't break things" that it has a pathological over-cautiousness. I see it in commit time, too: * LLM writes code * runs tests, they pass * "Hey human! I wrote the code, please review!" * LGTM, commit and push * "Lemme run the tests a few more times, just in case. Committed the code. Let me run the tests to be sure before I push. I'll run them one last time" It wants to run the full suite ALL THE TIME. "I made a docs change. Lemme run the tests to make sure it didn't break them" - WTF who has their markdown tested?!

@MattRogish @vansickn Totally and that's the dead giveaway because RL envs want strong verifiers which are actually pretty hard to construct without over fitting to said pathologies The less verifiable factors suffer in return

@itsyourcode @vansickn Also: burns more tokens. Vendor and User incentives, uh, bit at opposition here

@MattRogish @vansickn Oh man yes I have so much to say about this. Planning a blog post on it soon actually

@itsyourcode @MattRogish Excited to read

@vansickn @MattRogish here's a draft teaser. will publish when our blog goes live in general which should be any day now