Insider One lets you test more than one version of almost everything you build: onsite experiences, app content, customer journeys, emails, and recommendation strategies.
This page brings those capabilities together in one place, so you can see where multi-variant testing is available, how winners are selected, and how control groups measure the real impact of your campaigns.
Where you can test multiple variants
Product | What you can test | How variants work |
|---|---|---|
Website changes: text, images, colors, buttons, and layout elements | Create variations of your page, allocate a share of traffic to each variant group, and hold back a control group that sees your website as it is. | |
Mobile app content defined as variables in your app code | Your development team defines variables once; you then create multiple variants of their values in the panel without an app store release. | |
Journey paths: channels, content, and timing | Branch a journey into up to four paths and set an allocation percentage for each path, or distribute users evenly across paths. | |
Subject lines and email content | An experiment campaign compares two variants, A and B, on a test pool you define. | |
Recommendation strategies, design elements, and placements | Compare two or more algorithms with different configurations before rolling out the winner. |
How winners are selected
Insider One uses Bayesian inference to calculate the probability of allocating traffic to each variant, so you can read results as the probability that a variant wins rather than waiting for a fixed sample size.
A/B Auto-Winner Selection in Architect selects the winning path automatically based on the winner metric and the calculation duration you choose, and then directs all users to that path.
Email experiments run for a test window of 1 to 24 hours and pick the winner by the metric you choose: highest unique open rate, highest unique click rate, lowest unsubscription rate, or lowest spam rate.
Onsite Experiment tracks goal-based results for each variation, so you can compare variants on conversions, revenue, or custom goals.
Measure true impact with control groups
Control groups tell you what would have happened without your campaign.
A/B Split Control Group holds a share of users out of a specific journey, so the analytics show the incremental effect of that journey.
Architect-level Control Group holds a percentage of your entire audience out of all journeys. It keeps its proportion constant as your audience grows, and you can update the allocation anytime under Architect > Settings > Control Group. Use this setting when a buyer or analyst asks for a global holdout.
Onsite Experiment control groups show your website as-is to a share of visitors, so variant performance is always compared against the untouched experience.
Test combinations of changes
Each variation in Onsite Experiment can contain more than one change. If you want to evaluate combinations, create a separate variation for each combination you care about and split traffic between them. Keep the number of changes per variation small: more changes in a single variation increase the risk of HTML conflicts, so this approach works best for minor adjustments rather than full page redesigns.
FAQ
Q: Does Insider One offer a multivariate testing mode?
A: Insider One focuses on multi-variant A/B testing. Every experiment compares distinct variants that you define, and you can cover combination scenarios by creating one variant per combination, as described above. Email experiment campaigns compare exactly two variants.
Q: Can I run a test across more than one channel at the same time?
A: Yes. Branch a journey with the A/B Split element and put different channels, content, or timing on each path. Auto-Winner Selection can then move all users to the best-performing path.
Q: How do I know my test result is reliable?
A: Results are calculated with Bayesian inference, which expresses each variant's probability of being the best. For journey tests, set a calculation duration that matches your traffic volume; the winner is announced when the duration completes.
Q: Do users see the same variant if they come back?
A: In Architect, you can keep re-eligible users on the same A/B Split path, so a user who re-enters the journey follows the path they were originally assigned to.