Adam Kitain

Quick build-in-public from today. Our CRO agent shipped its first agentic-optimization test yesterday at 4pm: an in-checkout upsell, two variants, brief written by the agent. By 9am this morning the test had been live 18 hours. Zero upsells across both variants. Checkout completion appeared to be down about 30% in both groups. My first instinct was to call it. Instead I opened Claude with the Intelligems MCP connected and walked through the diagnostic. First I asked it to pull the conversion funnel. Shoppers were reaching the shipping-rates step at almost identical rates across all three groups, then dropping at the upsell card itself. Then I broke the same data out by device. The mobile drop was way worse. The reason looked obvious from the screenshots: on mobile, the upsell card was visually pushing the shipping option and the continue-to-payment button off the screen. I was about to kill this version, redesign the upsell placement, and start over. Before I touched anything, I waited a few hours. 24 hours in now. 250 checkouts, 150 orders, and the checkout-completion drop has balanced out. Looks like noise. There's 1 upsell in each variant group, where this morning there were zero. The instinct to kill on 18-hour data was wrong, even when the numbers and the screenshots both said act. The diagnostic was still useful: I'll tweak the mobile placement next round either way. But the test wasn't broken.

I keep watching ecom operators do things solo that took whole teams a year ago. best one i've seen lately: one operator looking at churn timing, sku-level subscription attach, checkout-step leakage, cancel-reason clusters, and ltv-by-cohort all in one chat, one afternoon. shopify mcp + intelligems mcp open at once. this used to be a 3-person quarter. the playbooks haven't changed in 5 years. who's allowed to run them did. 2026 is going to be weird for ECOM.

If you're paying attention, this is what's happening: - one person can now hold the full context of a problem - skills > headcount - strategy was always the actual bottleneck, not execution - the synthesis layer just got cheap very much not priced into how most teams plan for the next 12 months.

You have to think more carefully about the boring stuff when the shopper is an agent, not less. Stripe just shipped a wallet that lets agents spend on your behalf. The thing people are going to miss is that good reviews, clear photography, smart pricing and offers, and reliable shipping all matter more in that world, not less. An agent reads what a human glances at.

I think this is exactly right, and I keep finding the same pattern at work in the part of the AI economy I'm closest to. In growth work and conversion optimization, the bottleneck was never the operator. It was execution. An idea would take weeks to ship. What we're building toward at Intelligems is the inverse: agents handle the execution, and the operator gets to think many more ideas, prioritize them harder, and ship the ones that matter. I'd put my confidence in this making the operator's job bigger, not smaller, at maybe 80 percent. The fractal here is interesting. The same pattern shows up in software engineering, in research, in writing. The thing the agent automates is the load-bearing constraint, and the human's role becomes more cognitive, not less. Tools that augment look like fewer hours moving a row in a spreadsheet. More hours on the call where the strategy is actually decided. It's a remarkable shift, and we are still very early.

Was working on .ai copy with @victorpaycro . He wrote "an AI CRO platform." Told him to cut it. CRO is too small for what's actually getting tested. Pricing, shipping thresholds, discount structures, post-purchase upsell ordering. Most of those aren't conversion problems. They're profit problems and retention problems wearing CRO's clothes. The word I keep pushing back to is "agentic optimization." Bigger surface, more honest about what we're allowed to touch on a store.

If I have enough good ideas, I can't possibly execute on all of them fast enough. I need developers. That's the operator I keep meeting on customer calls. Plenty of ideas, nowhere near enough engineering hours to ship them. The agent stack we're building at Intelligems is for that operator first.

"I have all of this stuff I want to optimize. I don't even know where to begin." Most common thing I hear from operators lately. The store has too many surfaces for one person to keep in their head. The agent we're building at Intelligems is supposed to give the operator a place to start. That's the bet.

You used to need three different decks for the CFO, the CMO, and the logistics team. Same test result, retold three ways: cash impact for finance, segment behavior for marketing, order-volume math for ops. Three meetings, three follow-ups, one decision. Now you ask Claude to read the test through the Intelligems MCP, tell it which audience the readout is for, and get a version that already speaks their language. The test still wins or loses on the same numbers. You've stopped doing the translation by hand. docs.intelligems.io/developer-reso…

The way I'm thinking about every optimization on a store, in three buckets: Small wins. The 1-2% lifts that compound. Trust-badge variants, header copy, post-purchase upsell ordering. Run these on autopilot. Multipliers. The 10-20% lifts that stack across surfaces. Discount magnitude, free-shipping threshold, the email capture timing. Strategic moves. The decisions that reshape the business: repositioning, new product line, pricing model. The build-toward: small wins handled by agents, strategic moves staying with the operator, multipliers as the territory you and the agent meet on. And I see it came from the source.

Site changes used to be a budget question. Hire a developer, or live inside the theme. Now I see operators shipping new pages over the weekend with no developer in the loop. The tools got that good that fast. Which means the work shifts. The question stops being "can the site change" and starts being "what should it change to."

Here's how I think about where AI in experimentation is going. Today: agents already analyze your tests, surface the leaks, ping you when significance hits. The Slack bot is in production. Tomorrow: those agents string together. Your site starts running its own loop in the background. Future: human-in-the-loop becomes a setting on the dial, not the default. You choose where to sit.

Two things AI is starting to remove from every experimentation program: 1. The bottleneck on good ideas. Agents will generate them faster than your team can. 2. The bottleneck on execution. Agents will ship the variants without waiting on a sprint. Both used to be human work. The question we're heading toward is which of those ideas actually earns the traffic.

What I'm most excited about: a website that just starts optimizing itself. Tomorrow, the agent stack stitches together. The site runs its own small experiments quietly in the background. You stay in the loop on the big calls. I think most operators will land somewhere on that spectrum within a year. Probably sooner.

You either open a dashboard to check if the test worked, or Slack tells you the moment it did. An agent should hand you the answer, not make you go get it. Ask the bot anything. It tells you when significance hits. The Intelligems Slack bot is live for every customer.

Coming back to this account. (Last time I posted here it was mostly bitcoin and poker takes. I'd like to think I've grown.) For the last while I've been deep in what we're building at Intelligems, the agentic side of e-commerce. Site changes ship without a developer in the loop. An agent proposes the tests, drawing on what's worked across hundreds of other brands. The whole loop moves faster than any operator team I've ever watched try to do this on their own. I'm going to start posting about what we're shipping, what customers are doing with it, and what I'm seeing in the data. Plus the occasional aside on what's actually working for the operators I talk to most weeks. If you run a brand and you're trying to figure out what AI actually changes for you, this is the account. (I'll try to be more consistent than my LinkedIn track record. We'll see.)

@codyplof So pumped to see this all come together. Informing how our own agentic products are coming together. Especially love the design and intelligence skills, exactly how we're building it.

I’ve spent a ton of time vibe coding in Shopify. I think I’m pretty good at it. I don’t do it during the day. But it’s my current night, weekend, and early morning obsession. No idea why but it’s so fun and rewarding. Plus makes me more money. I’m setting up 2-3 new landing pages or multi variate CRO tests per week. Here’s a bunch of what I’ve learned and some tips on my process: Strategy is so important because of how fast you can move now. Our hit rate is higher than ever on CRO tests. I have a bunch of skills I built for CRO and insights. CRO skill - trained on CRO best practices and frameworks, all of our past intelligems tests, and site traffic to prioritize. But most importantly attached to a giant customer intelligence file that is fed via api from Typeform, listen labs, outer signal, junip, and more. Every survey we run or review we get makes this step better. I have a few different skills for building new pages or features. They kick off the insights process and competitor analysis. They build a html brief compiling all of this with some low fidelity mockups. The mockups are usually rough, even though they are using the design system. I’m sure I can architect some of the skills better, but it always makes mistakes and takes some trial and error. But I can get it looking 90% or the way there in an html file pretty easily. From there I have two workflows. One is pushing straight to Shopify. I have html to liquid skills that are trained on our theme architecture and requirements. Then using the Shopify CLI. Always building locally and pushing to a dev theme to preview. My favorite way to do this is on my phone using remote control and wispr. I often find it’s easier to polish designs in Shopify because it has all of our CSS. The other workflow is going from html to Figma. I find the Figma mcp is bad for this, so I use the html to Figma plugin. It’s not perfect but it gets the job done. I’ll have my design team polish the last 10% in Figma, and then I’ll use the Figma mcp to push back to Claude and then liquid. I find the Figma mcp much better for this part than pushing to Figma. From here I can just finish in Shopify and then I have a skill to manage the whole GitHub push/ pull/ merge/ PR process. Also been using Greptile for code reviews. I’ll often get a 3/5 at first and just feed Greptiles comments back into Claude; do 2 rounds of iterations and gets tons 4/5 and then I will submit the PR again. Before the PR I have two share skills. One is a design system audit, and the other is one I built for compliance to our theme that is based on feedback from early PR mistakes. A lot of things just trial and error. It’s like a video game; you just push, aww where you get stuck and try new things to push past it. When one approach works, build a skill to repeat it and it becomes part of hour workflow. But you’ll probably want to iterate on your skills. Happy to answer any questions or show some things I’ve built. At first I thought I was only gonna do this for landing pages. But I rebuild our cart while I was poolside last week, have a nav test, etc. I truly think one person can now run a best in class CRO/ experimentation program that encompasses strategy, research, design, copywriting, development, and analysis. Maybe it’s not the best in the world, but it would normally cost a lot more. We’re still hiring for this person. I’m essentially going to train one highly ambitious person on this workflow and give them a bunch of tokens. If that’s you, hit me up.

Can't tell you how many sticky-header A/B tests have run on Intelligems over the years. Hundreds. Probably more. Could I tell you whether a sticky header wins for your brand? Honestly, no. I don't know. And I'm a little embarrassed to say that, because I should. An agent trained on every one of those tests across every brand on the platform, one that has seen the wins and the losses and the context each test ran in, probably could. That's the shift happening right now. It's not that AI can ship code faster than developers. That's table stakes. It's that AI can sit on top of every test every brand has ever run and give you a real prediction about what's worth your traffic. Which is also the thing I'm most worried about getting wrong. The prediction is only as honest as the data feeding it, and there are a lot of ways to make agentic optimization look smarter than it is. Most of them come down to cherry-picking which tests you surface as wins. Running a roundtable on this at GROW LA next Thursday. Come argue with me if you have a different read.

"Log into Intelligems. Get data-backed, specific recommendations. Approve and they turn into experimentations automatically." This is the path for @adamintelligems. Remove marketers' mental load. Remove the need for a dev.