A/B Test overview

A manual A/B Test activity compares two or more versions of your website content to see which version best improves your conversions during a pre-specified test period.

NOTE

In addition to the Manual (Default) A/B Test activity (discussed in this section), Target provides two additional types of A/B Test activities: Auto-Allocate and Auto-Target.

See Types of A/B Testing activities below for more information.

A manual A/B Test activity (sometimes referred to as an A/B…N test) compares two or more versions of your Web site content to see which best lifts your conversions, sales, or other metrics you identify. Use an A/B test to compare changes to your page against your default page design to determine which experience produces the best results.

Manual A/B tests are particularly useful when you have a clear hypothesis of ways to improve your page performance based on success metrics or alternative content delivery.

Manual A/B tests are well-suited for large changes that might involve new layouts or drastically different treatments of the elements. If your test design does not easily break down into individual page elements, you should run an A/B test before a multivariate test.

When you set up your A/B test, you can determine the percentage of visitors that see each experience. For example, you might split traffic evenly between the control and a second experience, or you might test out a new, more risky experience by showing it to only 5% of your audience.

NOTE

For detailed information about determining the optimum sample size for an A/B test, see Plan Your A/B Test.

When the number of different experiences exceeds five and span two or more locations, it’s a good idea to consider an MVT test before running your A/B tests. The multivariate test shows which areas on the page are most likely to improve conversion. These are the locations that a marketer should focus on. For example, the MVT test might show that the call to action is the most important location for meeting your goals. Once you have determine which locations and content are most useful for helping you meet your goals, you can then run an A/B test to further refine the results, such as to test two specific images against each other, or to compare the wording or colors of a call to action. By following an MVT test with one or more A/B tests, you can determine the best possible content for the results you desire.

Types of A/B Testing activities

In addition to the manual A/B Test activity (discussed in this section), Target provides two additional types of A/B Testing activities: Auto-Allocate and Auto-Target.

Activity Type	Description
Manual A/B Test	Compares two or more experiences to see which best improves conversions throughout a pre-specified test period. This section describes how to set up a manual A/B Test activity, but the steps for the other types of A/B Test activities are similar.
Auto-Allocate	Identifies a winner among two or more experiences, and then re-directs traffic to the winner, increasing conversion as the test runs and learns. To learn about the benefits of using an Auto-Allocate activity, see Auto-Allocate in How long should you run an A/B Test and Auto-Allocate overview.
Auto-Target	Uses advanced machine learning to personalize content and drive conversions by identifying multiple high-performing, marketer-defined experiences, and then serving the most tailored experience to visitors based on their individual customer profiles and past behaviors of similar visitors. For more information,see Auto-Target.

For more information about which of these A/B Test activities is right for you, see the interactive Adobe Target Activities Guide PDF.

The steps for creating the three types of A/B Test activities are similar. To create an Auto-Allocate or Auto-Target activity, start by creating an A/B Test activity, but when you get to the Targeting page, choose the desired traffic allocation method, as shown below: