Multivariate testing

Type

Versatility

Definition

Multivariate testing (MVT) is a method of comparing multiple variables to determine which combination of variations performs the best.

Some benefits include:

• More efficient: Saves time and resources compared to running separate A/B tests.
• Deeper insights: Identifies not just which elements work best, but also how they interact with each other.
• Increased chance of finding the optimal combination: Tests all possible combinations to ensure you don't miss out on the best option.

Example

Image you product is an an e-commerce website selling sneakers. Your conversion rate (percentage of visitors who buy something) is currently 2%, and you want to improve it.

You believe that changing the following elements on your product page could increase conversion:

• Headline: Change from "Shop Our Latest Sneakers" to "Find Your Perfect Pair: Shop Trending Sneakers"
• Call to action (CTA) button: Change from "Buy Now" to "Add to Cart & Create Your Style"
• Product image: Use a lifestyle image showing someone wearing the sneakers instead of a plain product shot

Instead of running three separate split tests (one for each element), you run a multivariate test. This allows you to test all possible combinations of these elements simultaneously and see which combination works best.

Some possible outcomes could be:

• The test might reveal that the new headline and CTA button both have a positive impact on conversion, while the lifestyle image doesn't.
• You might find that the original headline works best with the new CTA button, and vice versa.
• The test might even show that all three changes together have a synergistic effect, leading to the highest conversion rate.