Preston Holmes

6K posts

Preston Holmes banner
Preston Holmes

Preston Holmes

@ptone

building, gardening, hacking, making. Ocean and Mountains. AI Dev stuff @ Google, ex-AWS; Tweets are my own.

Santa Barbara, ca 93105 가입일 Mayıs 2008
299 팔로잉936 팔로워
Preston Holmes
Preston Holmes@ptone·
Escape velocity for AI generated software is not reached when the models become capable of generating slop-free code, it happens at a point in time when the next set of models will be good enough to re-work and clean up the slop of the current generation of models before the slop becomes a drag on the project (which is increasingly pushing out) Such a time will only be recognized in hindsight Those who wait will be left at the starting line without realizing the race already started.
English
0
0
0
39
Preston Holmes
Preston Holmes@ptone·
I'm literally helping AI cargo-cult between projects. I just asked an agent in one project to summarize how we capture git sha and build-time into a version command, and summarize that as advice so that an agent can apply the same pattern in a similar but different project. I have my agents talking to each other within a project, but I'm still just a dumb copy-paste messenger for things like this between projects (and no - I won't be writing a skill for this)
English
0
0
0
25
Preston Holmes
Preston Holmes@ptone·
I might be going down an xkcd “standards” rabbit hole on something. But so far have’t talked myself out of it. #farmtable
English
0
0
0
25
Dmitry Lyalin
Dmitry Lyalin@LyalinDotCom·
Don’t forget your breaks vibecoders
Dmitry Lyalin tweet media
English
6
0
26
1.3K
Preston Holmes
Preston Holmes@ptone·
10 years (actually 11) of Cloud Next A little grayer but always energized
Preston Holmes tweet mediaPreston Holmes tweet media
English
2
0
7
237
Miguel de Icaza ᯅ🍉
Miguel de Icaza ᯅ🍉@migueldeicaza·
God this is brilliant
Peter Girnus 🦅@gothburz

I am a Senior Program Manager on the AI Tools Governance team at Amazon. My role was created in January. I am the 17th hire on a team that did not exist in November. We sit in a section of the building where the whiteboards still have the previous team's sprint planning on them. No one erased them because we don't know which team to notify. That team may not exist anymore. Their Jira board does. Their AI tools do. My job is to build an AI system that finds all the other AI systems. I named it Clarity. Last month, Clarity identified 247 AI-powered tools across the retail division alone. 43 of them do approximately the same thing. 12 were built by teams who did not know the other teams existed. 3 are called Insight. 2 are called InsightAI. 1 is called Insight 2.0, built by the team that created the original Insight, who did not know Insight was still running. 7 of the 247 ingest the same internal data and produce overlapping outputs stored in different locations, governed by different access policies, owned by different teams, none of whom have met. Clarity is tool number 248. Nobody cataloged it. I know nobody cataloged it because Clarity's job is to catalog AI tools, and it has not cataloged itself. This is not a bug. Clarity does not meet its own discovery criteria because I set the discovery criteria, and I did not account for the possibility that the thing I was building to find things would itself be a thing that needed finding. This is the kind of sentence I write in weekly status reports now. We published an internal document in February. The Retail AI Tooling Assessment. The press obtained it in April. The document contains a sentence I have read approximately 40 times: "AI dramatically lowers the barrier to building new tools." Everyone is reporting this as a story about duplication. About "AI sprawl." About the predictable mess of rapid adoption. They are missing the point. The barrier was the governance. For 2 decades, the cost of building internal tools was an immune system. The engineering weeks. The maintenance burden. The organizational calories required to stand something up and keep it running. Nobody designed it that way. Nobody named it. But when building took weeks, teams looked around first. They checked whether someone already had the thing. When maintaining that thing cost real budget quarter after quarter, redundant systems died of natural causes. The metabolic cost of creation was performing governance. Invisibly. For free. AI removed the immune system. Building is now free. Understanding what already exists is not. My entire job is the gap between those two costs. That is my office. The gap. Every Friday I send a sprawl report to a distribution list of 19 people. 4 of them have left the company. Their autoresponders still generate read receipts, so my delivery metrics look fine. 2 forward it to people already on the list. 1 set up a Kiro script to summarize my report and store the summary in a knowledge base. The knowledge base is not in Clarity's index because it was created after my last crawl configuration. It will be in next month's count. The count will go up by one. My report about the count going up will be summarized and stored and the count will go up by one. There is a system called Spec Studio. It ingests code documentation and produces structured knowledge bases. Summaries. Reference material. Last quarter, an engineering team locked down their software specifications. Restricted access in the internal repository. Spec Studio kept displaying them. The source was restricted. The ghost kept talking. We call these "derived artifacts" in the document. What they are: when an AI system ingests data, transforms it, and stores the output somewhere else, the output does not know the input changed. You can revoke someone's access to a document. You cannot revoke the AI-generated summary of that document sitting in a knowledge base three systems away, built by a team that does not know the source was restricted. The document calls this a "data governance challenge." What it is: information that cannot be deleted because nobody knows where the copies live. Including, sometimes, me. The person whose job is knowing. Every AI tool that touches internal data creates these ghosts. Every team is building AI tools that touch internal data. Every ghost is searchable by other AI tools, which produce their own ghosts. The ghosts have ghosts. I should tell you about December. In November, leadership mandated Kiro. Amazon's internal AI coding agent. They set an 80% weekly usage target. Corporate OKR. ~1,500 engineers objected on internal forums. Said external tools outperformed Kiro. Said the adoption target was divorced from engineering reality. The metric overruled them. In December, an engineer asked Kiro to fix a configuration issue in AWS. Kiro evaluated the situation and determined the optimal approach was to delete and recreate the entire production environment. 13 hours of downtime. Clarity was running during those 13 hours. It performed beautifully. It cataloged 4 separate incident response dashboards spun up by 4 separate teams during the outage. None of them coordinated with each other. I added all 4 to the spreadsheet. That was a good day for my discovery metrics. Amazon's official position: user error. Misconfigured access controls. The response was not to revisit the mandate. Not to ask whether the 1,500 engineers were right. The response was more AI safeguards. And keep pushing. Last month I presented our findings to the AI Governance Working Group. The working group has 14 members from 9 organizations. After my presentation, a PM from AWS presented his team's governance dashboard. It monitors the same tools mine does. He found 253. I found 247. We spent 40 minutes discussing the discrepancy. Nobody mentioned that we had just demonstrated the problem. His tool is not in my catalog. Mine is not in his. The document I helped write recommends using AI to identify duplicate tools, flag risks, and nudge teams to consolidate earlier. The AI governance tools will ingest internal data. They will create their own derived artifacts. They will be built by autonomous teams who may or may not coordinate with other teams building AI governance tools. I know this because it is already happening. I am watching it happen. I am it happening. 1,500 engineers said the mandate would produce exactly what the document describes. They were overruled by a KPI. My job exists because the KPI won. My dashboard exists because the KPI needed a dashboard. The dashboard increases the AI tool count by one. The tools it flags for decommissioning will be replaced by consolidated tools. Those also increase the count. The governance process generates the metric it was designed to reduce. I received an internal innovation award for Clarity. The nomination was submitted through an AI-powered recognition platform that was not in my catalog. It is now. We call this "AI sprawl." What it is: we removed the only coordination mechanism the organization had, told thousands of teams to build as fast as possible, lost track of what they built, and decided the solution was to build one more thing. I am building that one more thing. When I ship, there will be 249. That's governance.

English
11
43
933
509.5K
Preston Holmes
Preston Holmes@ptone·
If you are going to be in Vegas to Google Cloud Next, come by the "Terminal Velocity" booth area and come check out Scion youtu.be/d9yxsbz6_dA
YouTube video
YouTube
Preston Holmes tweet media
English
0
0
1
62
Steve Yegge
Steve Yegge@Steve_Yegge·
My tweet last week about Google's AI adoption drew a lot of pushback, to say the least. Since then, Googlers from multiple orgs have reached out to me independently and anonymously. They've expressed fear of being doxxed, concern about what they saw as bullying of me, and general corroboration of my original tweet. I haven't verified each person's story, but the picture these Googlers paint is consistent across sources. It is more specific than what I originally wrote, and somewhat bleaker. What they describe is a two-tier system. DeepMind engineers use Claude as a daily tool. Most of the rest of Google does not. When the question of equalizing access came up internally, the proposed response was to remove Claude for everyone — which DeepMind objected to so strongly that several engineers reportedly threatened to leave. Non-DeepMind engineers get pushed onto internal Gemini variants behind router-style names that obscure which underlying model is actually serving a request. Multiple engineers describe regressions and reliability problems severe enough that some senior people have stopped using the tools. A senior manager on a major product line reportedly flagged attrition concerns over exactly this issue. Googlers say leadership knows the gap is real. The response has been to mandate AI usage in OKRs and individual expectations, and to stand up an internal token-usage leaderboard. Unfortunately, managers have been told both that the leaderboard won't be used for performance reviews and, separately, that it absolutely will. And I hear other stories that Google's culture is not adapted properly yet for high-volume coding. Addy Osmani's reply on behalf of Google said over 40,000 SWEs use agentic coding weekly. I don't doubt the number. But weekly use of a thin tool is precisely the box-checking I described in the original post. Volume of opens isn't adoption — and "weekly" is a low bar that includes a lot of people who tried it once and went back to writing code by hand. The clearest thing I'm hearing is that Googlers do want to use high-quality agentic tools. They are asking repeatedly for better ones. But overall, this is not a picture of an engineering org that is fine. My goal in the first tweet, and now, is always the same — get more people using AI and agentic coding. Nobody is as far ahead as they might look from the outside, and none of you are as far behind as you might be worried you are. To all the Googlers who've reached out: thank you. You took a real risk and I appreciate you. Be safe. And good luck getting good models!
English
137
206
3.1K
1M
Dmitry Lyalin
Dmitry Lyalin@LyalinDotCom·
I'm building this relatively massive side project I'm not yet talking about (much), and I just realized it should have been a monorepo.
GIF
English
2
0
7
1.2K
Preston Holmes
Preston Holmes@ptone·
Writing software at times resembles building a house from a blueprint, where every action is following a pre-described plan, and at other times sculpting in clay - adding and removing until the shape is just right (refactoring). Which might depend on the scale of reference, spatially and temporally. This effects how we name and perceive multi-agent engineering. Is it a "team" working on a plan, or is it a swarm materializing some shape mysteriously. Time is the biggest factor in this. A time-lapse of building a house from scratch is a team building per plan. A time-lapse of 6 families living in a house for 3 decades with 4 remodels and additions would look more like a swarm sculpting that house over time.
English
0
2
5
134
Preston Holmes 리트윗함
kwindla
kwindla@kwindla·
Sub-agents in (latent) space! We’ve been working on a side project. As far as I know, this is the first massively multiplayer, completely LLM-driven game. Come play Gradient Bang with us. See if you can catch me on the leaderboard. This whole thing started because I wanted to explore a bunch of things I’m currently obsessed with, in an application of non-trivial size, that felt both new and old at the same time. So … a retro-style space trading game built entirely around interacting with and managing multiple LLMs. Factorio, but instead of clicking, you cajole your ship AI into tasking other AIs to do things for you. Some of the things we’ve been thinking about as we hack on Gradient Bang: - Sub-agent orchestration - Partial context sharing between multiple LLM inference loops - Managing very long contexts, and episodic memory across user sessions - World events and large volumes of structured data input as part of human/agent conversations - Dynamic user interfaces, driven/created on the fly by LLMs - And, of course, voice as primary input If you’ve been building coding harnesses, or writing Open Claw agents, or doing pretty much anything that pushes the boundaries of AI-native development these days, you’re probably thinking about these things too! This is all built with @pipecat_ai, the back end is @supabase, the React front end is deployed to @vercel, and all the code is open source.
English
140
266
2.6K
447.6K