Sample size
The number of visitors or orders each variant needs before a test result can be trusted.
Small samples swing wildly. A result from a few dozen orders can flip completely over the next week. The sample you need depends on your baseline metric, the smallest effect you want to detect and the statistical power you want.
Decide it before the test starts and run at least one full week, so every day of the week is represented. Elevate will not call a winner before each variation has 500 visitors and 20 conversions (2,500 and 100 for stores above $100,000 in monthly revenue) and the test has run for 3 days. Most tests reach significance within 1 to 3 weeks.
The AOV A/B test calculator shows how many orders per variant a test needs and how long that takes at your daily order volume.
Put it to the test on your store
Test prices, themes, pages, shipping and checkout on Shopify, then ship the winning variant.
