Bayesian statistics
Bayesian statistics reads a result as how likely it is that one version is better than another, updating that reading as visitors and orders arrive.
Bayesian statistics is a way to measure how likely one version of an experiment is to perform better than another. Instead of simply checking whether a difference is statistically significant, it calculates the probability that a variation will outperform the original.
As more data comes in, these probabilities are updated, giving you a clearer picture of which version is performing better.
Elevate A/B Testing uses Bayesian analysis to evaluate experiment results. You'll also see this expressed as the probability to beat baseline, which shows how likely a variation is to outperform the original version.
Worked Example
Imagine you're running an A/B test on your Shopify store. Both the original version (A) and the variation (B) receive 4,000 visitors.
Version A generates 120 orders, giving it a conversion rate of 3.0%. Version B generates 132 orders, with a conversion rate of 3.3%.
At this point, Version B is performing better. Bayesian analysis estimates the probability that B will outperform A, while your chosen significance threshold determines how strong that probability needs to be before the result is considered conclusive.
Now imagine the test continues for another two weeks, with both versions receiving equal additional traffic. Version B gains 20 more orders, while Version A gains only 10. The gap grows from 12 to 22 orders, providing more evidence in favor of B. If B starts performing worse instead, its probability of beating A could decrease.
This is why it's important to let an experiment collect enough data before choosing a winner. Bayesian probabilities change as new results come in, helping you make decisions based on the available evidence rather than early trends.
These figures are illustrative. The actual probability depends on the full experiment data, including visitor counts and the statistical model used.


