Neeraj

@cdngdev Nice! Be prepared to see a LOT of ads.

i gave 5.6 sol access to my camera roll and had it extract pictures of every piece of clothing i own from my photos then, told it to find new outfits for me and render them on me with gpt-image! its kinda cool to see your entire wardrobe in a collection like this

@doesdatmaksense @gridinoc @HamelHusain By failures do you mean tool call failures or final outputs being wrong? If your labels were on final output correctness, did you have access to a sme for the final labels? Very relevant to my usecase and I struggle with access to SMEs.

Do automated evals actually work? I and @HamelHusain spent last few weeks testing the auto-evals efficacy of different evals platforms. We took 100 real production traces from an apartment-leasing voice agent, manually reviewed the failures, masked the labels, and asked different systems to do the same error-analysis task and discover failure modes. We tested dedicated eval platforms like Braintrust, Arize, and LangSmith, along with ChatGPT, Codex, Claude Code, and Factory Droid. Full post here: parlance-labs.com/blog/posts/aut…

@CSProfKGD @davsca1 For a moment, thought this was plitvice lake

Today’s office view 😎

@xinyunsunshine @icmlconf Very interesting. Already a GEPA fan. Paper link?

Come find us at the RLxF workshop at @icmlconf — we'll be presenting there at 11 in hall A. Ask us any questions, or just come say hiiiii 👋

You already know prompting can change what an LLM does. Turns out it can change what a robot does too — we made a robot learn a task it kept failing just by rewriting the prompt. No retraining. No new data. Better prompt in, better robot out. 🧵 1/8

@vikramchandra Remember seeing you on ndtv an age back Vikram. You commenting on fable and the need for orchestrators was not on my bingo card (no snark, just surprise). Give omniagent from Databricks a shot!

This is almost certainly the way things will eventually turn out. There is no reason to pay a fortune to use Fable for routine tasks. Use low cost low token models for that. Use Fable or an equivalent only for tasks where superior capability and IQ is required. And an orchestra conductor (perhaps sitting in a hyperscaler) who manages the process for you.

@deliprao Congrats!

3 of 4 submissions to #COLM2026 accepted. Yay!

@mitsuhiko Nice car. Did you consider the electric cousin, ex90? Loved it myself.

We finally got the new car but I’m a bit sad that we’re leaving the minivan life behind. But there are just no nice options on the European market right now that work for us. So now it became a Volvo XC90.

@rakyll It’s bad enough that you need private repos :-) if anybody can implement anything, guarding ideas becomes the new flex. I would argue this is just true for prototypes though (today), having a head start on a production grade system still dissuades forking without consequences

Coding agents have a deep impact on engineering in general but the way they challenge the traditional org charts and ownership is much bigger. With this new kind of forkability, there is no real ownership unless it's enforced top-down -- which is often bad.

I never shared my thoughts about coding agents and there is a lot to be said in this topic. Some thoughts: For most of my life the real cost of building something that solves a hard problem inside a large organization was not always about typing. It was often inventing it in the first place, then coordination. [...] Execution overhead was almost always dwarfed by social overhead. You spent your best energy earning permission to start, and then had to do it again and again just to continue. Coding agents remove this traditional permission system. They benefit some while simultaneously creating unmanageable chaos for others.

@joshwoodward @demishassabis @GeminiApp I have a daily briefing set up that reports on markets from specific geographies and updates on armed conflicts. It made up an aftermath of a conflict. And last week it made up movement of $QQQ. Admitted to hallucination when I sent a Robinhood screenshot. Lost all trust.

What's something that you're surprised @GeminiApp can't do well, and we should have fixed a long time ago?

@sh_reya @gridinoc Have you looked at @kunchenguid ‘s excellent lavish? github.com/kunchenguid/la…

This was super fun. The most useful thing I talked about was how to use the Monitor tool in the Claude agent sdk (which watches background tasks and reacts to each stdout line as an event) so that user actions in an agent-generated UI actually go back and steer the agent, instead of the UI just being read-only. This is very powerful; you just instruct your coding agent to, as part of generating the html artifact, generate JS code to compile user interactions into logs (eg “user clicked X, user typed Y in this feedback box”) and the agent will, thanks to the Monitor tool, subscribe to those logs in real time and adapt the UI or provide new insights. The key novel idea is the UI code is no longer just output, it's the input too, and it's generated by agents so it’s custom-built for whatever task you're doing. I have not seen people use Monitor this way before. So I am preaching it everywhere I go. See my skill in the comments

@sidin Very interested. But what’s a line-edit? Don’t have a company but I author docs at work and my god the over the top, this is the greatest thing ever, Claude tone is irritating.

Guys look if you are using AI to write drafts of your content for your company then please give it a solid line-edit. It will make unbelievable difference to the prose and reader experience. And if this is a problem then please talk to me. I will set up your pipeline for you.

@sidin I mainly use Hermes via telegram. Have heard of something called OpenWork, never used it though.

Random question, but is there a claude Cowork equivalent that works with any model?

@kunchenguid Very well put @kunchenguid, gonna try no-mistakes this week. Hopefully then I can then stop checking Claude code’s output with a fine toothed comb.

my hot take on how much AI code we should review - you should review as much code from AI as your engineering director reviewed your code before AI here’s the chain of thought: - why do we even use AI to code? it’s to allow us to ship more - how much more should a single developer be able to ship now, compared to pre-AI? i see us going from 1-10x in the past 3 years, and on a trajectory to hit the 100x magnitude soon - that means every developer is going to own as much scope as a pre-AI director of engineering - i haven’t met a single eng director who said their team’s codebases were perfect and exactly how they would like it to be. why? because people who try to achieve that will fail to become a director - how do directors handle that level of complexity? it’s absolutely not by reviewing and micro-managing every engineer’s code. it’s through managing the culture, workflows, resource allocation, guardrails and measurable outcomes - when a director sees the team struggle on productivity or quality, they might lean in and try to understand the state of the codebase to develop some intuition for how to improve things systematically. even this is often done with the help from their principal engineers - i believe this is the right balance for how we should manage AI so, if we want to get a massive boost from AI, we must be prepared to operate in a way that allows us to manage much higher complexity, which requires that we remove ourselves as a bottleneck and manage the outcome at a different level shape your AI agents’ workflows - are they doing adversarial review? are there good automated tests? are they presenting evidence before shipping? are they doing phased rollout? are there good metrics to catch problems? survey your agents for feedback - ask them to reflect on their past sessions and report biggest problems causing them to struggle, and allocate enough tokens to get those problems fixed focus on outcomes - are your agents doing busy work? do you truly understand customer requirements and what work is worth doing? are your agents’ work generating the business outcome you expect? that’s how we truly scale

@Shashikant86 @aiDotEngineer @Teknium He’s always been an open ai marketer. With periodic Claude glazing to avoid suspicions

I suspected why this Theo guy posting so much against the Hermes Agents lately. At @aiDotEngineer World's Fair, I saw him marketing at the OpenAI booth. @Teknium If this guy post something against the Hermes Agents don't take it seriously, this guy is fully sponsored by the OpenAI.

@Shashikant86 @aiDotEngineer Gary who?

I heard a lot saying they will never apply to YC after the Gary’s talk at @aiDotEngineer which promoted writing markdowns to build companies and agents. As per him you can boil the ocean with bunch of markdown files!

@phalgooon Never managed to get past season 3, post which the actors somehow aged like a 20 year leap in Indian serials; it just got exponentially less funny.

the problem with modern family was it literally had two laugh out loud moments per episode

@prajdabre But you still ordered (like a sane person) for 11 years?

What sane person orders pizza in Japan? I spent 11 years there and I never had one good pizza experience.

@theshawwn Sorry to hear Shawn. Sent you a connection request on LinkedIn. If you see anything at Amazon, very happy to put in a good word with the hiring manager.

My LinkedIn is at linkedin.com/in/shawnpresser

I’ll be actually-homeless soon (living out of my car) if I can’t land a job. Our house is being sold within two months if I don’t find work. I’ve been trying everywhere, but very few companies are answering. So please, email me or DM with literally anything involving computers, or ask a friend, or your business’s HR person. (I’d love to work with you.) I’ll do a good job, and I can learn and adapt to any working style you need. I was employee #2 at Carmack’s AI lab, Keen. I worked at Groq helping to design and implement their LPU chips. I made Books3, a dataset trained on by LLaMA and Claude. I’ve been programming for 25 years. Location: Lake Saint Louis MO Remote: Yes Willing to relocate: yes Technologies: Python, PyTorch, JAX, C++, JS/TypeScript, Objective C, Swift, Rails, Linux, Lisp, and anything else you give me a week to learn. Résumé/CV: shawwn.net/docs/shawn-pre… Email: shawnpresser@gmail.com I wanted to avoid the humiliation of begging, but nothing is working. I need money or else our 3yo daughter loses her birth home. I’ve been through dozens of interviews from the companies that do answer, but the market is bad right now. You can have me for a steal.

@joelniklaus @harvey Thanks for the post. Would be interesting to compare the meta harness implementation against @LakshyAAAgrawal ‘s GEPA with optimize_anything()

New blog post on harness optimization. We hit Sonnet 4.6 performance with a 7x cost improvement. Fable 5 was the first frontier model release that evaluated on legal tasks. It only scored 13%, the worst performance among all benchmarks evaluated. @Harvey released this benchmark called Legal Agent Benchmark (LAB) just a month prior. It contains a set of realistic legal matters. Each task gives the agent a closed workspace of documents (contracts, emails, spreadsheets, slide decks) and asks for a concrete deliverable: a diligence memo, an issue list, a redline, a draft. An LLM judge grades the deliverable against a long rubric containing 61 distinct binary criteria each on average. Many frontier models such as Gemini 3.1 Pro don't surpass 0% all-pass rate (all rubric criteria passed). With automatic harness optimization, we manage to push DeepSeek V4 Pro from 0% to 5% all-pass rate, achieving parity with Sonnet 4.6 for 1/7 of the price. Read the blog post for the details: huggingface.co/spaces/joelnik…

@jessegenet @peteskomoroch @AnthropicAI You cannot use said dangerous superintelligence for classification. Too slow, too expensive. Honestly, it would be good to use a classical ml model but that’s not in vogue. Maybe they are using very strict word based filters to be sure.Can’t fault them given the gov reaction

Hot take: I believe @AnthropicAI is on the wrong side of history This is not a good way to roll out a revolutionary technology, they want people afraid instead of empowered and it’s gross to watch it play out