Sven Schmit

How come Bayesian and frequentist guarantees can be so different, even when the results are the same? In 1994, Philip Dawid published a fantastic paper about this. Let's look at this from an experimentation perspective! schmit.github.io/statistics/202…

We are building Allyon: Battlecry, the first monster trainer where creatures actually learn from you. It's closer to the Pokemon anime than the games: Allyon understand your voice, learn bespoke behaviors, have personalities and memories, and form lasting bonds with you. 1/7

In the January/February 2025 @HarvardBiz (and online now): a new article that explains the steps that companies should follow to get better at experimentation hbr.org/2025/01/want-y… from me, @IavorBojinov, @rameshjohari, @spschmit and Martin Tingley

Eppo lands new cash to grow its app, website and AI experimentation business tcrn.ch/3X70jNf

@jxnlco or you just fitted a linear trend

growth phase is over?

The pie is growing, time to shill a bit more in march. More cookbooks, more stuff on the hub. more SEO.

@jxnlco In your bio?

If you have AI in your bio, you have to prove it by demonstrating your ability to perform 3x3 matrix multiplication. Afterwards, calculate the determinant, gradient descent, and explain bias-variance trade-off.

@jxnlco Sounds familiar

It’s just going to be an actor model with a decentralized router based on an auction.

i should write the agent framework thats been in my notebook for 5 months now...

Fuck it ship it. I wrote this while kinda being annoyed by people asking for advice. But know if that I sound angry and haughty, it’s only 40% intentional. jxnl.github.io/blog/writing/2…

The last couple of months I've been working on a game called Thus Spoke Zaranova. A game where you as a human must pose as an AI. (1/7)

@seanjtaylor And 💯 on priors are great; or rather they are awesome. Bayesian experiment analysis with "proper" priors is very compelling imo. Not leveraging them is throwing away the baby with the bathwater, which might as well be the tl;dr of the blog post 😀

@seanjtaylor Thanks for the kind words! Thinking of the MDE in a power calculation as some sort of prior is interesting -- I have always thought of them more as a "worst-case analysis" which feels very frequentist compared to the "average case analysis" of Bayesian analysis.

I like @spschmit's post and the point is well-taken geteppo.com/blog/beware-of… But it's peculiar to distinguish between Frequentist and Bayesian tests, IMO. Normal advice for a Frequentist test is to do a power analysis and assume an effect size, & I can't argue that's not a prior.

@jxnlco Learn Phoenix LiveView :)

What’s the easiest way to build a multiplayer. Web app in nextjs? I just want to sync some buttons clicks. Be able to edit, highlight and leave comments on multiple text areas

@jxnlco When thinking about Json, I think about Jason

If I’m not the guy you think of when you ask an LLM to output json. Why not.

@teej_m @paul_rietschka This! The polars API is much nicer. The pandas API makes me want to write SQL

@paul_rietschka The pandas api is bad

@teej_m I'd say this is more an issue of using ratio metrics rather than averages; it's often unclear if a ratio metric going up (on average) is good or bad. Histogram doesn't really help here (a shift to the right can still be both good and bad)

You should know the distributions that a good stats class will teach you. But I’m talking about the histograms generated by human behavior, they’re not pretty. “AOV went up, is that good?” Look at the distribution. Oops, we lost low value orders, that’s bad! Averages lie.

What makes a data analyst great? One skill I think is important that I want to talk about – thinking in distributions aka “just look at the histogram already”. Distributions are so much more informative than just the average.

We are now accepting submissions for @CODEConference 2023!!! We welcome any and all research related to digital experimentation, whether it comes from academia, industry, or somewhere else. If you have any questions about whether your work is a good fit, shoot me a DM/email.

@Chris_Said That p-value does not inspire too much confidence in that point estimate though; some quick math suggests that the corresponding 95% confidence interval is -50 to 260

Clips of Trump speaking positively about vaccines are so effective that you can turn them into an advertisement and drive actual vaccinations at a rate of $1 per vaccine. science.org/doi/10.1126/sc…

When does it make sense to bandit, and when to experiment? Let @spschmit walk you through the decision-making process geteppo.com/blog/bandit-or…

Fantastic talk, highly recommended!

@PhDemetri @colin_fraser @Get_Eppo On it!

@colin_fraser @Get_Eppo get on this

Someone should make an A/B Testing framework that uses this

@hspter @Chris_Said ^ this

@Chris_Said If you get the real ones from The Netherlands, they are incredible! But agree that these are garbage

Has there ever been a food product more overrated than these “waffles”?