Convert.com

The 2026 GuessTheTest Best In Test Awards are officially open. And the case study deadline is coming up fast. After our CRO Month conversations with agencies like Open Partners, ROI Revolution, Inc., EchoLogyx Ltd and Browser To Buyer - CRO Agency, one pattern became pretty clear: The best agencies are sharing what they learn, explaining the thinking behind the work and helping the wider experimentation community get better. And right now, that matters more than ever. Experimentation is becoming more accessible, in part because of AI. More businesses are testing, more teams are curious, and more people are entering the space who may not have been part of the market before. That is exciting. But it also means the loudest advice is not always the best advice. Conversations about statistical rigour, sound process and stakeholder management can easily get buried under hype, shortcuts and “just launch more tests” thinking. The agencies and organizations pushing the message of good experimentation, not just convenient experimentation, deserve recognition. Especially the ones creating content that challenges myths, raises standards and helps teams make better decisions. So this year, alongside the award for standout A/B testing case studies, the GTT Awards will also recognize Educational Content Excellence. If your agency has been creating frameworks, resources, webinars, videos, research or practical content that helps people test smarter, this category is for you. And if you have a strong case study to put forward, entries are open for that too. Submission deadline: Friday, May 15 at 11:59 p.m. Eastern Will your agency be on this year’s list?

Full MCP access puts your live experiments in the LLM's hands. Here's how to take back control. The fix is MCPO. It sits between your LLM and your MCP servers and converts them into standard API endpoints. You decide which endpoints to expose. The LLM only gets access to what it actually needs for the task. From there, you build the workflow in n8n. A form collects the request and the page URL. From that, a small model fetches the HTML and generates the JavaScript for the test. Two API calls then create and configure the experiment in Convert. Every team member runs the same safe, optimised process every time. One technique worth taking from this: before building the n8n workflow, use Claude Code once to run the task manually. Watch which API calls it makes and in what order, then extract the JSON. Rebuild those exact calls as n8n HTTP nodes. You end up with a workflow based on what actually works, not what you assumed. The whole setup runs on small, cheap models. Cost and token usage stay low. Full guide by Iqbal Ali: convert.com/blog/ai/build-…

Simbar Dube came to CRO from journalism. His warning about AI: the hard thinking still has to be yours. Simbar is a Conversion Research Specialist at Enavi. Research synthesis that used to take days now takes hours. He's direct about the value of that, and equally direct about where the acceleration ends. Surfacing signals is faster now. Knowing which ones actually matter still requires context, segmentation, and judgment. There's also an offline experiment in there worth reading on its own. A retail client had just opened a new store, with higher in-store order values than online and customers too anxious to buy without seeing the product first. The play was a CRO test built around Buy Online, Pick Up In Store, designed to shift nearby demand into a more valuable channel. Classic experimentation thinking applied somewhere most experimenters haven't gone. When asked to define optimization: "Curiosity. Tested. Proven. Repeat." Full interview: convert.com/blog/optimizat…

AI search is eating top-of-funnel traffic. Sitting this out is still a decision. The shift is real and accelerating. People are searching in ChatGPT, Perplexity and Google AI Overviews instead of clicking through pages of blue links. The intent is the same. What has changed completely is the discovery mechanism. Traditional SEO was built around crawlers, rankings and keywords. AI search runs on a different logic. It synthesises, summarises and cites. The brands that show up are the ones with content that AI systems can find, understand and trust enough to reference. That is a different brief to what most content teams have been working to. The market is already moving. Gushwork just raised $9M to deploy networks of AI agents that autogenerate and continuously update search-optimised content and backlinks, built specifically for AI discovery. The infrastructure for competing in this space is being built right now by startups whose entire thesis is that the window to adapt is closing. There are guides and expert frameworks appearing, but the playbook is still incomplete. Brands adapting are doing it in real time, without a clear map, testing what gets cited and what gets ignored. Carmel Makaya walks through where the shift stands, what the early movers are doing differently, and the question that should be sitting at the top of every content and growth conversation right now. If your brand is not showing up in AI search results, who or what is showing up in its place?

The best CRO thinking in South Africa usually stays locked inside agencies. On 6 May, Cameron Calder, Johann Van Tonder, and Nicholas Wright are opening it up. Hype Digital and Convert.com are hosting an invite-only evening at Deer Park Café, Vredehoek, for 40 digital leaders with a view over Table Bay. Between the three speakers, they have built experimentation programmes for Ackermans, Travelstart, BMW, Umbro, Nike, Canon and Woolworths, co-authored the definitive guide on e-commerce optimisation, and are running AI-powered experiments for fast-growing brands right now. The evening covers why visitors drop off, how to personalise without guessing, and where the biggest revenue leaks on South African websites are hiding. Expect real examples of testing velocity, prioritisation and stakeholder buy-in, with open discussion on what experimentation maturity actually looks like inside South African organisations. At the end of the night, South Africa's Most User Friendly Website gets announced live. Every registration enters your site automatically, and the winning team takes home a high-value offsite. Register your interest below before the forty seats are gone. hypedigital.co/hype-digital-e… 6 May 2026 | Deer Park Café, Vredehoek, Cape Town

60 Minutes of AI Truths: How Are Experimentation Agencies Adapting in 2026? x.com/i/broadcasts/1…

The Intersection of Healthcare and eCommerce: Streamlining the Path to Purchase with CRO x.com/i/broadcasts/1…

Healthcare eCommerce has a problem nobody wants to say out loud. Craig Smith and Tellef Lundevall are tackling it tomorrow. Last seats. The data that would help most is the data users are most afraid to share. And that is just one of the tensions that makes CRO in healthcare categorically different from anything a standard eCommerce playbook was built for. Users are not browsing. Every page element is being evaluated for trust. Compliance constrains what you can test. Traffic volumes make statistical significance genuinely hard to reach. And success cannot be measured at the click, the outcomes that matter happen weeks or months later. CRO practitioners working in this space are operating on assumptions borrowed from a different category entirely. Tomorrow, Convert is joining Craig and Tellef from OuterBox to work through the specifics. Craig built and exited a five-time Inc. 5000 optimisation agency, and now lectures at Harvard and NYU. He knows the gap between what looks good in a case study and what actually holds up in practice. Tellef came out of Google's Accelerated Growth Team, built a rigorous attribution-focused firm, and has spent years working on the specific places where standard measurement models stop making sense in healthcare. The session covers the personalisation dilemma in regulated environments, how to approach statistical significance when traffic is low, and what measuring success actually looks like when the outcome is not the click. It goes well past "choose a HIPAA-compliant vendor." April 22, 11:00 AM EDT. Replay available. Register in the first comment.

Your A/B test reached significance on day 4. Calling it a winner now is like leaving an NBA game after the first basket. P-values fluctuate. Early results lie. The moment that looks decisive is often just a peak in normal variation, and if you call the test there, you are acting on noise dressed up as signal. This is the most common mistake in experimentation programs, and it compounds. As Sadie Neve puts it: early stopping distorts not just the one test, but the entire iteration pipeline built on top of it. Wrong learnings shape the wrong roadmap. Teams deprioritise ideas that actually had potential and can't figure out why results never replicate. So, how long should you actually run a test? There is no universal number, but there are inputs that give you one. Traffic, baseline conversion rate, minimum detectable effect, significance threshold, statistical power. Get those right before you start, and the duration follows. A few things worth knowing from practitioners who have done this at scale: Kateryna Berestneva waits for a minimum of 100 conversions per variation and 95% significance. In practice, that translates to four to six weeks on average. Sadie Neve maps traffic and baseline performance across the full site before committing to any experiment, so she only designs tests the underlying traffic can actually support. Gerda Thomas uses a pre-test calculator with built-in parameters. You put in your weekly traffic and conversions, and it tells you how long to run the test. No guesswork, no rationalising a stop because the numbers looked good on Tuesday. May Chin describes statistical power with a fishing net analogy. A loosely woven net catches the big fish but lets the small ones slip through. A low-powered test does the same, and you never know what you missed. There are scenarios where stopping early is the right call, but they are narrow. Ioana Iordache identifies three: - clear harm to primary metrics - conditional power that has dropped below 20% (futility) - a broken test with integrity issues. Outside of those, the full duration is the only defensible path. convert.com/blog/a-b-testi… What is the earliest you have ever stopped a test, and did the result hold up?

Shoes go in a cart. Medication decisions do not. The CRO playbook changes completely when health outcomes are involved. Healthcare eCommerce users arrive already scanning for signals of expertise and authority. They expect the same navigability as any other site, but every element of the experience is being evaluated for trust. Friction that a standard DTC brand absorbs becomes a reason to leave. Layered on top: compliance constraints that limit how you can test, personalisation questions that don't have clean answers, and traffic volumes that make reaching statistical significance genuinely difficult. CRO content skips past all of this. On April 22, OuterBox's Craig Smith and Tellef Lundevall are joining us to go into exactly this. Craig founded and scaled Trinity into a five-time Inc. 5000 eCommerce optimisation agency, had a private equity exit in 2023, and now lectures on web optimisation at Harvard and NYU. Tellef built Accelerated Digital Media from the ground up after Google's Accelerated Growth Team, with a rigorous focus on attribution and measurement in exactly the spaces where standard models break down. The conversation will cover statistical significance with low traffic, what success looks like beyond the click in healthcare, and whether personalisation is even viable in a regulated environment. It goes well past "choose a HIPAA-compliant vendor." April 22, 11:00 AM EDT. Replay available. us06web.zoom.us/webinar/regist…

Ask an LLM which A/B testing tool to buy today. Ask again next week. You'll get a different answer, from the same question. Rand Fishkin's research at SparkToro makes this concrete. LLM recommendations are non-deterministic. They shift with model updates, prompt variations, and training data. We've observed that they may also skew toward brands with the biggest marketing budgets. That's one half of the problem. The other half is price. Buyers under pressure default to cheapest available. But cheap tools cut corners on privacy infrastructure, and the fines that follow cost more than the saving ever justified. Wynter's research on how CXO buyers start their software journey shows the discovery phase in 2026. By the time most buyers reach evaluation, the shortlist is already shaped by noise. We built a video series with Ruben de Boer to address exactly this. He has spent 15 years consulting with 50+ brands on experimentation programs. He has watched this play out enough times to have an opinion worth hearing. Three videos. 15 minutes. Free. Video 1 covers how to research past the noise and build a shortlist based on actual criteria, not whatever an AI surfaced that morning. Video 2 is on vendor behaviour. The line Ruben put in there that we keep coming back to: "Mature tools are honest about what they don't do well. If everything sounds perfect, it could be a red flag." He also covers the question that reframes every demo: ask the vendor what their tool does badly. Video 3 is on free trials and what a proper evaluation looks like when the right people are in the room from the start. Your experimentation program deserves a deliberate decision. convert.com/6-mistakes-of-…

We analyzed 29 stats on how the CRO industry actually runs tests in 2026. A few of the numbers genuinely surprised us. The encouraging part: 70% of teams are now running tests to 95%+ statistical confidence. Nearly half reach 99%+. Three years ago that number looked very different. The industry is maturing. The alarming part: 52% of businesses have no formal QA process before launching an experiment. Half. No checklist, no review, nothing goes live clean. And then there's this: 1 in 10 experiments runs with fewer than 1,000 visitors. Teams are calling winners on data that will not hold up in production. The stat that puts all of it in context though: less than 0.2% of all websites run structured experiments at all. Among the top 10,000 sites by traffic, 32% do. That gap is where the real competitive advantage lives right now. Covers adoption rates, test quality, team maturity, agency benchmarks, and where most programs actually break down. Worth a look if you run or build experimentation programs. convert.com/blog/a-b-testi…

Trina Moitra and Roshan Nagekar are going to Jaipur on April 18 to learn from the best in D2C experimentation. Johann Van Tonder, Akshay Shivpuri, Varun Rajwade, Gursimran Gujral, Kamal Sahni, and Himanshu Gaur will be in that room - practitioners who have spent years building experimentation programmes that hold up outside of case studies. The roundtable OptiPhoenix & सादा / SAADAA has put together is built around a problem D2C leaders know well. Tests go out. Some win. The dashboard looks fine. And somehow, the business does not move the way it should. The wins are not transferring. The velocity drops the moment performance marketing needs attention. Experimentation stays a side project instead of becoming infrastructure. The session gets into three things specifically. 1. How to build experiment velocity without it falling apart under pressure. 2. How to measure what actually matters beyond conversion rate. 3. How to structure a stack that enables testing instead of slowing it down. Small room. Senior people only. Manual approvals. Not a webinar where nobody talks. If you are a D2C founder or growth leader doing serious revenue and want to be in that room, the invite is free. Secure your spot: optiphoenix.com/d2c-roundtable

Innovation is coming into healthcare from the outside. OuterBox's Tellef Lundevall sees this every day. A split landscape: On one hand, startups treating digital experience as a core part of the product. On the other, legacy systems where the patient interface has barely moved in a decade. For healthcare ecom companies sitting in that first group and interacting with the second, this creates a specific pressure. Patients arrive expecting the same navigability they get on any modern site. Then they remember their health is what is actually at stake, and they start scanning for signals of expertise, authority, and trust. Standard ecom optimisation is not built for this combination. This is what Tellef and Craig Smith - both from OuterBox - are joining us to talk through on April 22. What CRO actually looks like in healthcare ecom. Where standard testing assumptions break. How you experiment responsibly when the stakes are higher than a conversion metric. The session covers low-traffic testing constraints, what success looks like beyond the click, and whether personalisation is even possible in this space given the data boundaries that exist. April 22, 11:00AM EDT. Replay available. Save your seat: lnkd.in/dpR8WaaA

The best experiment Pieter Boonstra ever ran started in a packaging department. Not in a spreadsheet. He was visiting a client's organic soup factory and saw packers tearing open boxes of 6 jars, repacking them into whatever quantity the customer had ordered. 7 jars. 4 jars. 2 jars. Hours of manual labor every day. So he tested something simple: give webshop visitors a small discount for ordering in multiples of 6, 12, or 24. Revenue per customer went up 60%. And orders could ship straight from the factory floor without repacking. Pieter runs ConversieKracht, the fastest-growing CRO agency in the Netherlands. And this observation-first approach runs through everything he does. Another one from his playbook: A client selling high-fiber protein bars had a problem. User research showed the biggest "aha moment" was taste. Once someone tried a bar, they were hooked. Research also showed some customers were cutting bars in half because they felt too full. So Pieter and his team designed a tasting box: 5 best-selling flavors, half-size bars. They tested it on the homepage. +15% purchase uplift among new visitors. And the repeat purchase rates from tasting box customers matched those of regular buyers. We asked Pieter where he sees optimization heading. His answer: the companies growing fastest are the ones where marketing and product share ownership of results. Not separate silos. Shared accountability. On AI in experimentation, he's optimistic. He sees a future where optimizers spend more of their time on what they do best, coming up with creative solutions for real people, and let AI handle the setup, coding, and statistics around it. We asked him to describe the discipline of optimization in 5 words or less. His answer: "Create proven business impact." Full interview on our blog. Link in the comments. We're collecting more playbooks like Pieter's. If you're running experiments worth talking about, we want to hear from you.

''Dark patterns are lazy.'' Jon Crowder has been saying that long before it went mainstream. Another Web Is Possible just became a Convert certified partner. We've been working with Jon for a while now - he built Convert Chorus, one of the first Convert apps, we co-launched the Hall of Shame together, and there have been plenty of projects in between. Fast, sharp, same-day replies, and always, something useful in the feedback that we hadn't thought of. His agency, Another Web Is Possible, runs on something Jon has been saying out loud for years - you do not need manipulation to get results. No fake urgency, no manufactured scarcity, no consent flows designed to trip people up. Just solid experimentation, honest UX, and serious CRO work. Welcome officially, Jon!!

"We do personalization." You do segmentation. There's a difference. Daphne Tideman put it well in one of our last sessions - most of what we've been calling personalization is actually segmentation. We bucketed people, swapped a headline, and called it done. Showing a different banner to mobile users isn't personalization. Changing a headline based on traffic source isn't personalization. Four broad audience buckets with slightly different copy isn't personalization either. That's segmentation. And for years, that's all teams could do, not because they weren't smart enough, but because the data and the tools weren't there yet. What's changed is that gap isn't inevitable anymore. With Nexus, you upload a CSV or sync your CRM and every lead, account, or customer lands on a page built for them specifically. Their name, their company, their context - in the headline, the CTA, the images, the copy. Not a segment of 10,000 people that kind of resembles them. Actually them. And you do it yourself inside Convert, without touching engineering. All of it sitting on the same infrastructure that's been running experiments across 40,000+ sites for 15 years. Curious where you draw the line, where does segmentation end and real personalization start for you?

A/B test data can look clean and still be completely wrong. For a portion of users, your test scripts may never load at all. When your scripts run from a third-party domain, browser privacy controls and ad blockers like EasyList intercept them. Silently. No flag. Your test runs, your dashboard shows traffic, but a chunk of your visitors got the default experience the whole time. You're making decisions on data that was never right. That's the problem Custom Domain (CNAME) fixes. Point a subdomain - experiments. yourdomain. com - to Convert and your scripts run first-party. Same as the rest of your site. Ad blockers stop caring. Your security team has something they can actually approve. Your tracking holds up across Chrome, Firefox, Safari, and every privacy-first browser your visitors are using. Configure it yourself inside your account, no back and forth needed. If your cross-browser data has never quite added up, if your security team keeps flagging third-party scripts during procurement, if you're running headless and your deployment keeps getting complicated, this is the one to look at. Available now on Pro and Enterprise plans for Convert Experiences and Nexus.