Abel

@hy3na_xyz @eringriffith there is weakness everywhere haha

A detailed and brutal look at the tactics of buzzy AI compliance startup Delve "Delve built a machine designed to make clients complicit without their knowledge, to manufacture plausible deniability while producing exactly the opposite." substack.com/home/post/p-19…

@itsCathyDi @agentmail @adisingh have you seen the @PrimeIntellect hoodie ? both of you got turbo mogged irl

didn’t think anything could beat our dedalus hoodie but @Agentmail just raised the bar

is microsoft the only company named after a medical condition? sounds like a diagnosis. macrohard sounds like the cure you would want. @elonmusk

@comma_ai how good is the real2sim transfer ?

Fellow e2e enjoyers, We've got something special for you today. Introducing openpilot 0.11: the first robotics agent fully trained in a learned simulation. Shipped to real users.

@nicole_clash is part of the training data now kek right @grok

AI can’t generate this yet so I had to generate it myself

@ManikantaChavv7 @saafwater @theresidency wow saaf water

Moved to SF in Jan! We just closed a $600K order last week at @saafwater. In the same few weeks, we grew from monitoring 560M → 670M litres of water daily. Our deployments paid for themselves in 3 months!! Presenting it all in a few hours at @theresidency Demo Day in SF. Who's coming? 🚀 @HrishikeshMB @SanketMarathe09 @jehunix

@rasmus_up I think there are studies here has to a lot to do with perplexity and burstiness

Someone should make a study analyzing what makes a piece of text seem like AI written. My theory: the more it correlates with feeling of cringe, the more likely it is to be AI

@aidaxbaradari @be_inaudible wow congrats !

Today, we're introducing Spectre I, the first smart device to stop unwanted audio recordings. We live in a world of always-on listening devices. Smart devices and AI dominate our world in business and private conversations. With Deveillance, you will @be_inaudible.

I think folks are being misled by "high performance" on browser use "benchmarks". It's not appreciated enough just how different they are to LLM benchmarks, and why they're difficult to do right and currently extremely flawed. LLM benchmarks are "closed world": the model generates text, and you verify it against some fixed ground truth that doesn't change. Even 'hard' benchmarks like Humanity's Last Exam fit this pattern. The benchmark dataset fully defines the expected inputs, outputs, and validation function. Browser use benchmarks, however, are fundamentally different because they're not closed world. "Actions" - things that change state on a website - are especially difficult. You can't go around willy nilly and mutate state on Twitter, Salesforce, etc, every time you run the evals. That especially applies to the websites we care about: internal enterprise software being the most obvious category. Even data retrieval can be difficult: websites and data change. Restaurant availability changes every hour, flight availability/prices change even faster. It's _slightly_ easier than actions since you can cache the HTML and make it closed world, as some benchmarks do, but this doesn't work for actions, and ages badly. Other benchmarks get around this by trying to fix the date of a check ("find me flights on 1 March 2024"). Ofc that trick doesn't work for most tasks (like that flights example - you can't view historical flight availability). Then there's CAPTCHAs, which exist on basically every high-value web task (even if hidden). Current benchmarks exclude all these 'inconvenient' tasks, which massively skews them to be totally unrepresentative of how humans use websites. Pure computer use have it easier because they're often closed world: the start and desired end state can be well-defined and evaluated inside a network-less container. Updating an Excel sheet has no harm (which tbf represents a lot of economic work). But once you're doing things in a browser, on websites over the internet, this nice property doesn't apply anymore. WebArena's answer to this conundrum was to create 'fake' websites that were supposed to be representative of real ones. The problem is, they're not. OSWorld makes it kinda closed world by providing cached versions of HTML, but this only really works for data retrieval. They're also very unrepresentative. WebVoyager is especially egregious: just 15 (!!) websites are represented, and the tasks are ridiculously easy. Take a look yourself: github.com/MinorJerry/Web… So, how does this translate to the claims made by browser startups? Well, WebVoyager (the extremely easy one) is the benchmark the avg browser startup reports 85%+ accuracy on. Claude's performance is reported for computer use, and against OSWorld which is dominated by closed-world tasks. So really, high reported accuracies should be taken with a huge grain of salt, and there's still a long way to go before computer use is solved. That said, there's at least one other team thinking about these problems (@yutori_ai, with their release of Navi-Bench). From first principles, this is a really tricky problem to solve. The infra and data to properly benchmark web agent performance is extremely nascent and underdeveloped. It's a problem we think a lot about at Indices -- please reach out (DM) if you do too!

@sama can I’ve .001%

We have raised a $110 billion round of funding from Amazon, NVIDIA, and SoftBank. We are grateful for the support from our partners, and have a lot of work to do to bring you the tools you deserve.

@dezzie_rezzie loooooooool

maximus decimus meridius

@AnthropicAI everyone knows this

We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax. These labs created over 24,000 fraudulent accounts and generated over 16 million exchanges with Claude, extracting its capabilities to train and improve their own models.

@PalmerLuckey call them black whales

muuu I love cows

@bluewmist nothing stop buying things you don’t need and it will change your life

What life changing item can you buy for less than $100?

@itanmaymaliwal @bluewmist yikes

@bluewmist A $20 book that changes how you think. Most life upgrades aren’t expensive they’re just applied consistently.

@Hamzeml do you have a Rust .md file 👀🦀 that you would recommend ?

Stop Feeding Your Rust Code Raw Primitives (It Deserves Better), by @Hamzeml open.substack.com/pub/hghalebi/p…

@METR_Evals why is the variance so high ?

We estimate that GPT-5.3-Codex with reasoning effort `high` (not `xhigh`) has a 50%-time-horizon of around 6.5 hours (95% CI of 3 hrs to 17 hrs) on our suite of software tasks. OpenAI provided API access for this evaluation.

@brian_lovin yeah I’m on a Mac M2 with 16GB of RAM and it crashes my Mac half of the sessions

This looks amazing but The #1 thing holding me back from using the Claude Desktop app for more things is performance. It's so slow and buggy. I'm on an M3 Max with 96GB of RAM and it drops frames when switching between the Chat and Code tabs...