Benjy Boxer

3.9K posts

Benjy Boxer

Benjy Boxer

@boxerbk

Katılım Haziran 2009
481 Takip Edilen1.2K Takipçiler
sarah guo
sarah guo@saranormous·
Rundown of the very bad week in security: - TeamPCP (sophisticated hacking group) attacks: Hackers broke into the system that builds a oss popular security scanning tool called Trivy. This was a supply chain attack (when bad code is slipped into widely used software tools or libraries, so it spreads automatically to anyone who downloads or uses them). They used the stolen access to poison other tools like LiteLLM (a popular Python gateway library with ~100m million monthly downloads that lets developers easily call many different AI models through one simple interface) and Telnyx (a comms platform, that for example lets devs make phone calls and send texts). The bad code stole passwords and secrets from developers’ computers, which then let them break into Mercor (stealing a lot of sensitive data) and Cisco (stealing source code for their AI products). - Axios npm attack: Someone took over the main contributor’s account for axios (a popular JavaScript library that makes it easy to send and receive data over the internet in browsers/apps). This was another supply chain attack. Hackers released two fake versions that secretly installed evil software on anyone who downloaded them, giving the hackers full remote control of those computers. - Claude leaks: Anthropic accidentally published their AI coding tool with a big hidden file that contained all its internal code, secret instructions, and planned new features. They did not leak the model weights. They did have documents about their next unreleased AI model “mythos” exposed online. These were accidental mistakes, not hacks. - Railway issues: Railway (a cloud platform for devs) made a short mistake in their settings that let random people see other users’ private information for <1 hour. Separately, bad actors used Railway’s platform as a tool to help run phishing attacks against Microsoft accounts. To clarify for non security friends - these are not the result of some “rogue cyber AI.” They are affecting companies in the dev/AI ecosystem. But AI software abundance/pace of development, agentic library selection, and ai-automated builds/ai-managed secrets are surely amplifying classic supply chain issues and human error.
English
7
10
67
6.9K
Benjy Boxer
Benjy Boxer@boxerbk·
I felt this all day today reviewing some competitive intel work I asked Claude to do and then had ChatGPT verify. And they still had a bunch of information wrong. If I hadn't spent the time verifying the work, which felt so tedious, we would have looked so dumb for the inaccuracies.
Rohan Paul@rohanpaul_ai

Wharton’s latest AI study points to a hard truth: “AI writes, humans review” model is breaking down Why "just review the AI output" doesn't work anymore, our brains literally give up. We have started doing "Cognitive Surrender" to AI - Wharton’s latest AI study points to a hard truth: reviewing AI output is not a reliable safeguard when cognition itself starts to defer to the machine.when you stop verifying what the AI tells you, and you don't even realize you stopped. It's different from offloading, like using a calculator. With offloading you know the tool did the work. With surrender, your brain recodes the AI's answer as YOUR judgment. You genuinely believe you thought it through yourself. Says AI is becoming a 3rd thinking system, and people often trust it too easily. You know Kahneman's System 1 (fast intuition) and System 2 (slow analysis)? They're saying AI is now System 3, an external cognitive system that operates outside your brain. And when you use it enough, something happens that they call Cognitive Surrender. Cognitive surrender is trickier: AI gives an answer, you stop really questioning it, and your brain starts treating that output as your own conclusion. It does not feel outsourced. It feels self-generated. The data makes it hard to brush off. Across 3 preregistered studies with 1,372 participants and 9,593 trials, people turned to AI on over 50% of questions. In Study 1, when AI was correct, people followed it 92.7% of the time. When it was wrong, they still followed it 79.8% of the time. Without AI, baseline accuracy was 45.8%. With correct AI, it jumped to 71.0%. With incorrect AI, it dropped to 31.5%, worse than having no AI. Access to AI also boosted confidence by 11.7 percentage points, even when the answers were wrong. Human review is supposed to be the safety net. But this research suggests the safety net has a hole in it: people do not just miss bad AI output; they become more confident in it. Time pressure did not eliminate the effect. Incentives and feedback reduced it but did not remove it. And the people most resistant tended to score higher on fluid intelligence and need for cognition. That makes this feel less like a laziness problem and more like a cognitive architecture problem.

English
0
0
1
166
Benjy Boxer retweetledi
Felix Rieseberg
Felix Rieseberg@felixrieseberg·
Today, we’re releasing a feature that allows Claude to control your computer: Mouse, keyboard, and screen, giving it the ability to use any app. I believe this is especially useful if used with Dispatch, which allows you to remotely control Claude on your computer while you’re away.
English
907
1.5K
18.8K
4.7M
Benjy Boxer
Benjy Boxer@boxerbk·
Oh my god. ChatGPT, I’m just trying to learn details about dinosaurs with my son. I already spend every waking moment thinking and fretting about my company, now, ChatGPT needs to remind me to think about it?! 🤣 By the way, that Netflix Dinosaurs show is incredible. Some scenes hit so hard.
Benjy Boxer tweet media
English
0
0
1
85
Benjy Boxer
Benjy Boxer@boxerbk·
@andrewchen For non critical systems or proofs of concept, let it rip. For anything that matters for long term stability, I think it would be a mistake to not understand the code base that customers depend on.
English
0
0
16
479
andrew chen
andrew chen@andrewchen·
One question I've been asking founders is: do you try to review all the code that the LLMs write or do you just accept it? I think it's about 50-50 right now but the momentum is towards just accepting the AI-generated code and I think that number will eventually go to 100% This is one of the most telling indications of how AI-native a team is. It's hard to get super high throughput if you are reviewing every line Poll: what do you do?
English
261
11
289
108.8K
Benjy Boxer
Benjy Boxer@boxerbk·
“The economy is transitioning from a regime of scarce intelligence to one of scarce verification.” @ccatalini The tail risk introduced with cheap execution work being done via agents is quickly accumulating in our economy. It’s partially a moral hazard problem as companies overvalue the marginal cost of work dropping to the cost of compute and under appreciate the growing risk to their business and the overall economy. The incentives to automate work are too high. Satya Nadella, among other CEOs, boasts the percentage of code written with AI (in 2025, it was 30%). The concern is: how much of that is being verified and who or what is doing the verification before ghosts in the machine start to feed off one another to create tail risks that increase the likelihood of outages? CDOs were celebrated for their ability to eliminate risk. These complicated instruments were considered genius. But because they were so complicated, the correlation was underestimated, and the tail risk scenario unfolded in a catastrophic way from 2007-2009. The challenge will be: how do we get companies to appropriately price the cost of verification and invest in it? And if we don’t, who will bear the costs of this risk? One of my favorite sections of the paper also focused on individuals and an analogy to photography and painting. When the marginal cost of photo realistic representation went to zero, painters retrained and focused on their unique interpretation of a scene. Thus, expressionism, impressionism, and abstract painting were the best way to demonstrate talent and perspective. Today, people and businesses will need to lean into their unique perspective to stand out from the crowds of bot produced intelligence and work. Uniquely human things will be competitive advantages - meeting in-person and finding ways to connect with community will generate alpha. People will gravitate to verified human work and communities online if they cannot gather in-person. Bottom line - we need to properly price insurance and verification or the abundant intelligence our agents produce could unravel in spectacular ways. papers.ssrn.com/sol3/papers.cf…
Benjy Boxer tweet media
English
0
0
1
125
Benjy Boxer
Benjy Boxer@boxerbk·
It’s inevitable. And we are also underinvesting in the tail risks. CDOs were a genius contract to eliminate risk until we mispriced their correlation risk. Since we don’t understand the way the models come up with answers and work together and we can’t afford to verify their outputs, something will eventually break and a flash crash scenario may happen across all of our economy. papers.ssrn.com/sol3/papers.cf…
English
0
0
0
62
Benjy Boxer
Benjy Boxer@boxerbk·
@bznotes The customer impact of the compound startup strategy.
English
0
0
1
459
Bilal Zuberi
Bilal Zuberi@bznotes·
Didn't expect Rippling to have such atrocious UI and processes. Every third thing we want to do requires emailing customer service. Ugh.
English
34
2
158
26.3K
Benjy Boxer
Benjy Boxer@boxerbk·
@bryce And here I thought it was coffee. Time to switch 🤷‍♂️
English
1
0
0
75