Robert Kirby

unhinged hottake: Forth (ie. 1970, from 56 years ago) will become popular in 2027 for context engineering management.

A ChatGPT automation just found ~$45K in erroneous invoices across 3 years of billing history that I've confirmed and already had resolved. My lifetime history for ChatGPT is ~$1,800, so it just paid for itself 25x over. I setup an automation with read-only access to my email, and tasked this one specifically with analyzing construction invoices. It has access to prior construction invoices, emails, meeting notes, etc. It produces a report and emails it to me (the only email its allowed to send, enforced by API token) whenever I receive a construction invoice. Across 3 years of construction projects, it found about $45K in issues. Some were wrong amounts, some were duplicate invoices, some were invoices addressed to the wrong person. I manually verified, emailed my GCs, and got refunded/credited. I get multiple construction bills each month and each bill is ~50 pages in a PDF of low-quality scanned paper. I do manually review each bill but its pretty hard to be right all the time. I do believe these were genuine mistakes and not done out of ill will just based on what the mistakes were. I don't want to share my full construction costs across the past few years, but $45K is a very small percentage of overall billed amounts. Pretty sweet.

ive spent a lot of time recently being outwardly excited about frontier models and their capabilities because i am genuinely excited about what i am able to do with them that i was able to do before and how they change my day to day work life for the better. that said, i want to take a moment to say that i absolutely hate the current steady state and we should not settle for this poverty of progress and the current abhorrent status quo. the fact that we pay for reasoning tokens that we cant see or interact with is insane. the fact that closed source harnesses exploit this very fact is insane. the fact that we are still subject to random classifiers and data retention policies is insane. the fact that we are still dumbing models down to be 'safe enough for the unwashed masses is insane' . the progress curve has stagnated because we are collectively willing to tolerate this bullshit and pay for the privilege. we are rapidly coming to the point where 95% of people cant tell the difference between a cheap / small / free models and SOTA for their trivial use cases and when that happens a modicum of check will come due but the proletariat will need to revolt against these conditions before real change happens and in the meantime we must count on the generosity of open weights models (and even they don't disclose their data sources) to apply even modest pressure against the avalanche of the frontier. happy sunday

from this point on "vibe coding" should mean coding by hand. it's obviously a hundred times worse and slower, but it's craftsmanship, you're in it purely for the vibe

Is there anyone who can steelman the “tell your customers they’re losing their favorite product in 5 days, then say sike over and over” strategy

Kinda cool that tibo can reset the usage limits at ant now too

some tips on 5.6 sol: 1. remove old slop. try disabling certain community skills and plugins, especially bundles with 20+ skills. think of 5.6 sol like someone who just grew from senior to staff or senior staff level: prescriptive guidance that used to help becomes micromanagement and makes the work worse. you can always add back what clearly helps. 2. turn on codex memory in settings. give codex feedback on what you like and don’t like, and ask it to remember. 3. you probably don’t need ultra. i do 95% of my work on sol high, sometimes use sol extra high, and have only used sol ultra for a handful of sessions. start with high and only move *up* when you’re not happy with the result.

@Stu_J_ prompting counts as coding

thank you tibo

sol high seems to be consistently passing my 'we weren't able to do this before with models' benchmarks and unlike fable it actually lets my prompts through. i've only had a single prompt that required ultra and so far no meaningful different between sol high, xhigh and max

DHH told me on My First Million that he and Jason kept a data scientist on staff for over a decade to crunch numbers and run reports. - "We never did what the numbers told us to do." - "What we would do was we would do whatever the hell we wanted to do." - "And then if the numbers supported that, we'd go, those are good numbers." - "And if the numbers didn't support that, we'd go, yeah, I don't know. There's probably some factor you haven't calculated in." ha!

crazy how software engineering became pay-to-win overnight

A few things to clear up here. 1. Reasoning buckets These are rough product labels, not fixed apples to apples token budgets across model versions. 5.6 spans a wider range, so 5.6 xhigh isn’t equivalent to 5.5 xhigh and can use more tokens. 2. Token efficiency When we say token efficiency, we mean that 5.6 can solve the same problem using fewer tokens than 5.5. But if you run it at a much higher effort ceiling, it can spend more in order to do more. And like I mentioned in point #1, the effort ceiling is not 1:1 across buckets in the different models. I agree this is quite unintuitive, but hopefully this clarifies

Ok, I take the naming thing back, this is just as opaque and convoluted as ever (functionality is fine to desirable tho). Never change oai

This is real and it is not 750 TPS. Anyone with 5.6 Sol Ultra on fast mode can reproduce this! It’s all GUI interactions with CUA. No MCP or bpy needed.

spent yesterday on both grok 4.5 evals and using it in practice. it is a very good model for its price and intended use case. its smarter than glm5.2 (the model i would most immediately compare it to) in a lot of ways for ncode harness use and shows the beginning of real frontier RL / post training polish . it is very good at operating in ncode and makes excellent use of the tools at its disposal all while being extremely token efficient (i'd say its most distinguishing quality) . my 2 major complaints are no 1m ctx (which is basically standard now and i find myself missing often in real use) and its price point is just a _touch_ too high for where it fits (in my world at least) . ideally, i would use it as the subagent execution arm to replace glm5.2 or gpt5.5 med or to replace GLM 5.2 as the planner and delegate to dsv4-flash or gemma 4 31b . that said, i do plan to use it (unless gpt5.6 replaces it today) quite a bit from now on. if they can apply (or even improve) this post training polish to their next 2T base model (that is currently training as i understand it) , then they could have a very interesting next release

This is a certain kind of talk around LLMs that I find increasingly puzzling. That is all of the people bitching that LLMs constantly generate crap code and hallucinate solutions, and are worthless for programming. This has almost never happened to me, and never during the last two model generations I have used (chat GPT 5.4 and 5.5). Occasionally a model used to get a little deranged when I pushed its context limit, but under codex that doesn't happen anymore; instead I got a red-highlighted warning when the limit has been exceeded and I need to clear my session. I've applied AI to feature changes, refactoring, and debugging over 63 different projects written in C, Go, Rust, Python, and shell. I've written documentation with it. I've decompiled a DOS binary into readable source code. It's now routine that whenever I have to touch one of my projects I start by running the regression tests, then fire up codex and asking it to audit the code for bugs and suggest improvements. My experience is that LLMs are excellent and tremendously empowering tools. Their worst limitation is a kind of architectural tunnel vision - they're extremely good at generating code to specification but sometimes blind to higher-level patterns. Which is okay, it's my meatbrain job to be good at that. The most valuable thing I find about LLMs is exactly that they *don't* screw up details and edge cases. I'm a very, very good coder by human standards (I'd better be, with 50 years of experience!) but the LLMs are better than me. Because if a code change needs to touch (say) five places in the code, they reliably find all five rather than doing the human thing of fixing four and then having to debug for hours before you figure out that there's a fifth one you missed. Are the downshouters living in a different universe than me? Are they using old, weak models? Or do they have some kind of skill issue that I can't see because I have mental habits and communication skills that are a good fit for the handles on these tools? I don't know. And I think this is an important thing to figure out, because I'm seeing lots of stories in the news that suggest billions of dollars are being wasted on misdirected token spend. It all seems very simple to me. Be clear in your thinking, tell the model what you want with precision, and good things happen. What...what am I missing here?

software engineering has become hill finding

@can called it months ago boristane.com/blog/the-softw…

Gm.