Vlad Chubakov (@Programmatic 101)

JUST IN: LinkedIn rolls out a “Seems like AI slop” button, amid growing concerns over low quality, AI-generated content on the platform.

We simulated a year of marketing decisions 36 million times, and we are now sharing everything we found. I thought about not posting this on a Saturday but no need to pretend that we aren't thinking about causal inference 7 days a week over here at Haus... First, why do we invest so much in our estimators and improving experiment precision at Haus? Because an experiment doesn't create value just because it produces a lift estimate. It creates value when that estimate leads to a better decision – scale the channel, pull back, hold, retest. If the signal behind those decisions is noisy, doing more of them doesn't average you toward the truth, it means you make more wrong turns, faster. We've watched this play out anecdotally for years. But anecdotes aren't numbers. So Patrick Hillery built a simulation that modeled a full year of experiment-driven budget decisions, varying only the two levers a team actually controls: how precise their measurement is, and how often they test. What we found: - Acting on noisy results left the business worse off than doing nothing 38% of the time, more than twice as often as the precise approaches. - You can't test your way out of noisy signals with higher testing volume. Brands running noisy experiments finished ahead no more often at 15 tests a year than at 6. - A strong experimentation program is worth double digits. The precise, high-cadence approach delivered a 13.2% average revenue lift, more than double the noisy approach's payoff (5.6%) at the same cadence and limit. - A tight confidence interval can be manufactured. Some methods look precise by hand-picking a few well-matched markets, but the result only holds for those markets. In these cases, precise results aren’t necessarily accurate ones. I'm linking in thread the full article, methodology walkthrough and companion spreadsheet we are calling Monte Carlo, and we are making it available to all.