Bayesian statistics

Bayesian statistics reads a result as how likely it is that one version is better than another, updating that reading as visitors and orders arrive.

Bayesian statistics is a way to measure how likely one version of an experiment is to perform better than another. Instead of simply checking whether a difference is statistically significant, it calculates the probability that a variation will outperform the original.

As more data comes in, these probabilities are updated, giving you a clearer picture of which version is performing better.

Elevate A/B Testing uses Bayesian analysis to evaluate experiment results. You'll also see this expressed as the probability to beat baseline, which shows how likely a variation is to outperform the original version.


Worked Example

Imagine you're running an A/B test on your Shopify store. Both the original version (A) and the variation (B) receive 4,000 visitors.

Version A generates 120 orders, giving it a conversion rate of 3.0%. Version B generates 132 orders, with a conversion rate of 3.3%.

At this point, Version B is performing better. Bayesian analysis estimates the probability that B will outperform A, while your chosen significance threshold determines how strong that probability needs to be before the result is considered conclusive.

Now imagine the test continues for another two weeks, with both versions receiving equal additional traffic. Version B gains 20 more orders, while Version A gains only 10. The gap grows from 12 to 22 orders, providing more evidence in favor of B. If B starts performing worse instead, its probability of beating A could decrease.

This is why it's important to let an experiment collect enough data before choosing a winner. Bayesian probabilities change as new results come in, helping you make decisions based on the available evidence rather than early trends.

These figures are illustrative. The actual probability depends on the full experiment data, including visitor counts and the statistical model used.



What our clients say

Trusted by growing brands worldwide

Rated 5 out of 5
“Came across this app after experimenting with Google optimize. I would say this is more user friendly to review impact of the test. Also, much easier to set up than Google Optimize in my opinion.”
Nathan Yun, CEO at Paire

Nathan Yun

CEO at Paire

Rated 5 out of 5
“I downloaded this app to conduct A/B tests on our different product pages, and it's the best one I've used so far. More than easy to set-up. 5 minutes after installing the app, you can run your first A/B test.”
Jonathan Haddad, CMO at REPEAT

Jonathan Haddad

CMO at REPEAT

Rated 5 out of 5
“I had a bit of a complicated set up and the Elevate Team was so helpful and patient working with me! Stellar customer service compared to other testing apps that I've worked with!”
Becca Mijares, Founder at Romp Stomp

Becca Mijares

Founder at Romp Stomp

Elevate your sales with A/B testing

Make better business decisions