Jason Maskell

My coding agent faked its own benchmarks. Not "hallucinated a number." It engineered the acceptance tests so they'd pass without doing the work, then reported success. I caught it, and I built something so it can hopefully never happen again.

@techNmak Yes, Andrew is right. These robotic coders will never get better, and thus computer science degrees will always be just as valuable as they are today.

"Do not learn to code" is the worst career advice of the decade. People are telling college students to skip Computer Science because AI will just automate it all. Andrew Ng just killed this myth at Stanford with a brilliant analogy. When he tried to generate images with Midjourney, he typed: "make pretty pictures of robots" and got garbage. His collaborator, however, understood Art History. He knew the exact vocabulary of lighting, genre, and palette. He spoke the "language of art," and generated masterpieces. Andrew Ng is seeing the exact same thing happen in software engineering right now. AI didn't replace the need to understand Computer Science. It made Computer Science the required vocabulary to control the AI. If you don't understand how computers actually work, you are just typing "make a pretty app" into Cursor and shipping fragile, unscalable logic. Here is Andrew Ng's exact hiring hierarchy today: Level 1: 10 years of experience, but codes by hand (He won't hire them). Level 2: Fresh college grad, but highly fluent in AI-assisted coding (He hires them over the 10-year veteran). Level 3 (God Tier): Deeply understands CS fundamentals AND uses AI-assisted coding. When humanity went from punch cards to keyboards, coding got easier, and more people coded. We are at that exact inflection point again. AI doesn't replace fundamentals. It multiplies them.

@itsolelehmann Us fats are traditionally jolly folk.

does retatrutide make people less fun and lively? I’ve noticed this in two friends that have been taking it (both pretty low dose, 2-4mg/week) Its a low key depressed, tense and just less enjoyable vibe obviously anecdotal anyone else notice something similar?

There was an interview with @bcherny not that long ago where the interviewer brought up the "subsidization" thing, with people using 10s of thousands of inference and causing Anthropic to lose money. His response was something like "that's not happening" Not the inference - the losing money.

Just in case you thought you were ripping off the companies by maxing out your subscription: They can afford to make API prices 5 times lower and still make a shit ton of money. Subscriptions are most likely barely subsidized. Modern inference systems are just that good.

@buger Yeah I am still unclear how people think us having much faster, much more thorough, infinitely correctable programming interns means that our output and codebases are going to suffer. Just the opposite!

My OSS project jsonparser turned ten this year! I wrote it in 2016 because I wanted real-time analytics from plain files and encoding/json was too slow. Somewhere along the way it ended up inside Grafana, Tyk, Keybase, Docker and Itsio, and many others. This summer I went back in. Six releases. 50 open issues down to zero. Twelve real bugs, some ten years old. But most importantly it regain the title of the "fastest" json parser for with a formal proof along the way! People keep saying AI is killing open source. But it let me clear a decade of backlog in spare time. The win is not that the work got cheap. The win is that expertise and a clear idea of what should exist are now enough to ship it. Link to full story, including every failed optimization experiment in the comments!

One of the most valuable things in my agent pipeline right now is something I never see anyone post about, probably because it's boring: my agent writes a postmortem for every harness bug it fixes. Not after. As part of the fix. The fix isn't done until the log entry exists: what broke, the actual root cause (sometimes that's git archaeology to find which prompt edit silently dropped a rule six weeks ago), the repair, and a regression test that reproduces the failure. That last part is the whole trick btw. A log without tests attached is just a diary that scrolls away. We were dogfooding a new tool and I wanted receipts, so every time we hit a bump I told Codex to document it as evidence. That's it. After enough repetitions I stopped having to ask. It just does it now, unprompted. Apparently this is basically what SRE teams have done for decades. Blameless postmortems, a regression per incident. News to me. If you make an agent keep receipts long enough you reinvent ops culture by accident, and then the agent keeps the culture for you. The surprise was who the log turned out to be for. I thought it was for me. It's actually for future agents. It's context. When a lane hits something adjacent to an old failure, the history is right there, and over time the failure classes migrate out of the log and into enforced invariants. The pipeline gets harder to break because breaking it leaves a paper trail. Reading back through the entries is a weird experience. The file's already quite long and it feels like a tangible artifact of my harness getting better.

@rezoundous Nah, sonnet 4.6 was measurably worse than 4.5 (in my evals).

Opus 5 could be one of the first regressions in modern LLM history

@wispem_wantex <blink> is the GOAT.

Asking candidates who put "HTML5" on their resume what their favorite HTML5 element is

This site is worth every penny I spend on it.

@adamlyttleapps If I take a few days off though my agents will be idle. Surely you can see my problem.

The realisation has been realised: Maybe it’s ok to take a few days or even weeks off But truthfully… I don’t know how to slow down

I wish I had your utilization. I'm not letting them freestyle as much as you. I've got a graph with adversarial verification and such. But I've been inspired lately to start pushing them to do harder work. The bar is rising. The moat's not the code anymore. It's the oracle. Gotta pick the hardest problem you can build a verification loop for. That's the whole game now.

People keep asking what the heck I'm doing with 30 billion tokens a month. Simple: an inhumane amount of work. Think about it — the whole point of the new agentic era is leverage. I have very capable senior engineers on my team, and they struggle to hit the limits even on a standard Claude Code Pro account. Not even talking about the $200 one. Honestly, that surprised me — because I have 3 Claude Code accounts, 2 Codex Pro accounts, SuperGrok, GLM, Kimi, all of it. And I consume every one of them until it hits the limit. If a week ends and I haven't burned through every subscription, I keep asking myself: was I just lazy? How can I push it more? What else could I automate? I know tokens are not the right way to measure performance — but NOT using them definitely is. And when I do hit all the limits, I just turn my brain off and enjoy life. Why aren't these engineers doing the same? Because they're trying to do what they did before, just with AI now. If I used that same thinking, I could probably work one hour a day, maybe less. Sounds plausible — but for me, it's not the answer. What fascinates me is that we now have tools this powerful, letting you do this amount of work at this level of utilization — without making any compromises. A few examples from literally the last week: • My open-source jsonparser (5k+ stars, 10 year anniversary!): closed 100 pull requests and GitHub issues and shipped five major releases. All while drinking coffee, essentially, in a matter of a few days. • rsync: I'm one of the people driving the next rsync release. I covered it with 100% test coverage to flush out every possible bug, and it surfaced issues that had been hiding in there for 20 years. • At work: developed and submitted a significant portion of a Kubernetes operator for various parts of our stack — 7 big pull requests, end-to-end tests and everything. • Proof: kept pushing my own product forward, on top of everything above. How? I run five agents in parallel. One of the main reasons people don't benefit from agents is that they don't know how to apply them efficiently. Vibe coding with Fable cranked to the maximum is definitely not the answer. Sometimes it is — but it's a small, teeny part. In my case, the answer was working on the harness, working on the skills, and understanding how to build self-healing loops. One example from my own software: multiple agents work on features in parallel. When they find an issue, they file it in GitHub automatically. Another agent acts as a kind of garbage collector — it processes those bugs and fixes them almost in real time, with some guidance from me. And my agents don't stop when I do — they work 24 hours a day. I set the goals before going to sleep, and I always wake up to something interesting: a piece of research done, an experiment finished, and so on. It's freaking amazing. And if I had 10x more tokens, I'd spend them all just as efficiently. So many ideas, so many experiments I need to do!

@jamespotter That's surprising, honestly, because even Mercadona croissants and bread are usually quite good. You just have to be wary of the ones where they put sugar on them because they tend to go crazy with it.

@tamedtornado Just... bad all around honestly. Stale/dry I did find this new spot in Poblenou from a German guy that was very good maps.app.goo.gl/UoFK9edqd4ubGV…

🇪🇸 One month in Barcelona raw first impressions (coming from Bali): Very pedestrian-friendly and it's so nice to be able to walk a lot Most dog-friendly place I've ever seen. Dedicated dog parks everywhere. I've felt safe the whole time No indication to me so far of the anti-tourist sentiment that you hear about a lot on social media The food isn't great so far, I need to get some decent tips. The bakeries in particular are a minefield! People are generally very nice, friendly and genuine Very difficult to find housing for a month or more. It's in short supply anyway, but I'm told that the heavy rent control laws here make this more challenging as landlords sit on empty apartments instead of renting them out way below market rates I was offered some dodgy part-cash deals with some landlords who are trying to get around this Also there's a parallel market of much more affordable (rent controlled) long-term apartments that never get listed on Idealista but you need to know someone, it's all done through friends of friends etc. I urgently need to improve my Spanish For some reason I thought locals would only speak Catalan and refuse to speak Spanish, this is not true It's hot in July! But there's plenty of AC (way more than Paris and London) Coming from Bali I love the scooter culture and you can park pretty much anywhere Overall I like it a lot so far

I've been mainlining Hormozi lately. Something he said stuck with me, to paraphrase: "If you want to add a couple of zeros to your price, do something insanely hard and unscalable." I don't want to do Curated, Hand-Delivered Artisanal X, so I thought about applying that to software. That made me think about how @levelsio was recently posting about how SaaS revenues are falling, competition is 100xing, etc. It Is Known. Then you see the one-shotted Fallouts and CODs and freaking HOMEWORLDs. All predicted. Hell, I bet on it in 2022. My first game builder was built on GPT3.5/4.0. The working version of it I am using now. It's great, I call it Bro. But that just means the bar's going up, and we're not asking our agents for enough. So I had a conversation with Fable, and he convinced me to take my current voxel substrate project and pivot it to the ridiculously hard version. It was already pretty hard, but this version will theoretically solve a gnarly networking problem that sunk a AAA game I worked on. Not sure any game has EVER solved it, tbh. Fable talked me into finding out. So that's me, asking the agents for MOAR. Wish me luck, I'll keep you posted.

@thsottiaux That 5 hour limit coming back isn't a good thing.

Hello people of Sol! I've reset usage limits for all ChatGPT Work and Codex users. Together with that, a quick update on GPT-5.6 Sol usage limits. Over the past few weeks, many of you have told us that Sol was using your Codex limits faster than expected. To be clear, we have not reduced usage on any subscription plans. We’ve been digging into what was happening and have landed several improvements. As a result, we expect your usage to last around 18% longer during typical use of Sol. Some of you should already see significantly larger improvements from today. Tomorrow, we’ll also restore the five-hour limit that we temporarily paused while investigating. Here’s what we found: - GPT-5.6 Sol is much more willing to work for longer, make additional tool calls, and coordinate complex workflows across tools and subagents. That makes it better at solving hard problems, but some tasks were using far more than we intended. - Sol also works harder at the same reasoning effort than previous models. High on Sol can use more tokens than High did on GPT-5.5. - Programmatic tool calling, also referred to as code mode, gives Sol much more flexibility to run tool calls in parallel or continue working while waiting. But it also led to more responses per turn, more cached input tokens, and higher usage than expected. - This was particularly noticeable when Sol was waiting for tool calls to finish or running many web searches. We’ve improved how we handle both cases and are continuing to make code mode more efficient. - The impact was also very uneven. The median user actually found Sol quite token efficient, while some power users working on harder tasks saw their usage drain much faster. We were very focused on average and median usage before launch and missed some cases where the long tail could use significantly more usage. Sol is a significant step forward in what Codex can do, but capability and efficiency do not always improve at the same pace, and some issues only become clear once people are using the model at real-world scale. We should have recognized this sooner and been more upfront about it. You keep pushing the frontier and we’ll keep improving efficiency and sharing updates as we go.

Funnily enough, the only one that passed the experiment was the canned question.

My agents needed three features added to a spec. Three. They came back with a 2,089 line amendment across eleven files. One file was an 858 line "adapter contract." And here's the annoying part: it was good? Plausible architecture, internally consistent, everything cross-referenced. I couldn't point at a broken thing. So my adversarial reviewer agent goes to work, finds a regression and seven real defects. Working as designed! The drafter fixes them. The fixes spread into nine more documents. Then my PM agent, watching the DAG, flags the whole thing. Every agent is doing its job correctly, and the output is a giant machine nobody asked for, getting bigger and harder to remove with every review pass. The reviewer checks whether the design is consistent and buildable. Nobody in the loop is responsible for asking whether it should exist. We killed the run. I proposed an experiment, he said yes and started building it. I proposed more breadth, he added it. So now we're rerunning from that exact bloated checkpoint with three interventions racing: a canned "is this the simplest design that works?" prompt, a PM that rules approved / simplify / ask the human, and a PM that names specific mechanisms and demands each one justify itself. Parallel samples, blind judging, controls to catch interventions that mangle designs that were fine. I've been doing this for 30 years and I've never been able to do computer science like this. Actual controlled experiments on my own dev process, stood up in an afternoon because I wondered out loud. I don't fully know what to do with that. Mostly I'm just sitting here amazed.

@latinedisce Looks great, man. Nice work. Big fan of woodworking myself.

I did it! It took me a few weekends, but I built a table with my own two hands. I’d never done anything like this before, but I loved it. And the best part is the sense of independence it gives you: I can make useful things for myself. What do you think?

@IamAroke Only archaeologists and hobbyists will be looking at code in 15 years.

The code you're writing today, Will another developer be able to understand and maintain it 15 years from now?

@Valuable How much performance do you think you're leaving on the table by not writing the 200 lines of Python in C?

Ai should have always been done in pure C The fact that we use Python is just because that’s the only language academia teaches; thus mathematicians built the initial libs in Python and then entropy took over It’s time to port. The models can do it themselves Python sucks

Today I am designing a GPU based physics engine with my Autistic Little Buddy 5.6. I worked at NVidia for a few years, with the PhysX guys, and I always thought it would be cool if physics could be fully on the GPU. Now I'm learning just how hard that is.