AnalyticsToolkit

The new standard for state-of-the-art statistical planning and analysis of online A/B tests is here! A game-changing overhaul of Analytics Toolkit brings with it a ton of improvements, new features and functionalities. Full list of changes and updates: blog.analytics-toolkit.com/2021/new-stand…

Is any of your colleagues skeptical of the value of A/B testing? The business case for experimentation (#abtesting) is explored and showcased through simulations, numbers, and graphs in this article: 👉 blog.analytics-toolkit.com/2024/the-busin… #experimentation

What if the observed effect in an A/B test is smaller than the minimum detectable effect (MDE) used in planning it? Is it cause for concern? Does it make a test less trustworthy? The answers to the above and more in blog.analytics-toolkit.com/2024/what-if-t… #nhst #statistics #mde

Observed power is a tempting concept, but is misused almost by definition. Are you curious about the true power of your statistical test? You're barking the wrong tree of post hoc power. Full article at blog.analytics-toolkit.com/2024/comprehen… #statistics #statisticalpower #nhst

"Stop AbUsing the Mann-Whitney U Test (MWU)" #abtesting #statistics #DataAnalysis blog.analytics-toolkit.com/2024/stop-abus…

Not sure whether you should adopt sequential A/B tests? Think it's all about testing velocity? Check out this article showcasing the primary value of sequential tests in an accessible manner: blog.analytics-toolkit.com/2023/sequentia…

Worried about the false positive risk of your experimentation program, or a particular test? Can you estimate what proportion of your "winners" are in fact "losers", and does it even matter? This and more #abtesting #statistics in blog.analytics-toolkit.com/2023/false-pos…

After more than a decade, we are dropping our Google Analytics API integration and thus all related functionalities. Learn more about why and what you can do to use GA4 A/B test data going forward: blog.analytics-toolkit.com/2023/analytics… #googleanalytics #GA4 #abtesting

How to run shorter & more efficient A/B tests? This article covers the basics and can lead to 40%+ improvement in the expected test duration. #abtesting #statistics blog.analytics-toolkit.com/2022/how-to-ru…

A deep discussion on statistical power, minimum detectable effect, and how to design statistical tests in #abtesting: blog.analytics-toolkit.com/2022/statistic…

What is the best design for a statistical test with sequential evaluation of data at multiple points in time? Should you do group sequential or go full sequential? Is there a time & place for each? blog.analytics-toolkit.com/2022/fully-seq… #statistics #abtesting #conversionrateoptimization

/2 Seems they are now attempting to add more unrelated queries to suggestions related to our brand name, following a similar pattern. While the damage is not great at this point, the phony suggest is now #2, and if this continues all our suggests will soon be 100% irrelevant.

@JohnMu Help needed with weird Google Suggest spam. 100% fake traffic from India brings a completely unrelated query "brandname earn money" up in suggests for "brandname" queries slowly over 4 months. A few real users are now clicking on the phony suggest. What can we do?

Session-based metrics see use in online #abtesting, but they can easily lead to erroneous test conclusions. While well-known by some, it seems this issue is underappreciated by others. This article presents the issue in detail: blog.analytics-toolkit.com/2022/when-sess…

P-values and confidence intervals: an accessible explanation for practitioners of #cro and #abtesting. #statistics blog.analytics-toolkit.com/2022/p-values-…

A new glossary entry on A/B testing: analytics-toolkit.com/glossary/a-b-t… is now up. Should be a useful resource when introducing someone to online testing and experimentation. #abtesting #conversionrateoptimization #cro

A little update for those using Analytics Toolkit to analyze tests performed in Google Optimize. Now there is the option to import data from tests with a paused/disabled default Control group. Variant 1 has to be assigned the default user experience. #googleoptimize #abtesting

"Statistically rigorous A/B tests are impractical". This and similar beliefs may be hindering your A/B testing program, even if you do not subscribe to them. This article takes on debunking the top misconceptions about scientific rigor in #abtesting : blog.analytics-toolkit.com/2022/top-misco…

@_mightydouche The reseller of our domain apparently had access to make the change in the authoritative DNS servers, so they pointed the records to their own DNS servers (and obviously copied the records there).

Our service should be restored shortly for everyone. We were finally able to switch our DNS away from eNom (who have still not solved their issue) and the site should start resolving shortly. Apologies for this quite unprecedented downtime!

Analytics-Toolkit.com is affected by what appears to be a prolonged unexpected downtime of the eNom nameservers. As the registrar is down as well, this is currently impacting hundreds of thousands of domains with no solution. We expect to be back soon.