A/B testing

Last update: 2023-09-22
  • Created for:
  • Beginner
    User

The A/B Test functionality in Adobe Campaign allows you to define two to three email variants. Each variant is sent to population samples in order to determine which has the best result. Once determined, the winning variant is then sent to the remaining population. You can choose to vary the email’s content, subject, or sender.

Learn how to configure and send an A/B test to test the subject line of an email.

 Transcript

In this demonstration you’ll be trained how to review the A/B Testing features of Adobe Campaign Standard. Let’s see it in a live demo. Here from the home screen of Campaign Standard I can click the Create an email card to begin the wizard.

In the first part of the wizard, in creation mode, I can select one of the various email types. Here we can see we have three different A/B test options. I can A/B test on content, subject or sender.

Let’s go ahead and click on the Subject to A/B test and click Next.

In the email properties now I can see that I have a Label. Let’s change that to Adobe Video Learning A/B test on subject. We’ll leave the ID blank. We’re going to attach this to a Campaign which will make it easier in the future for organizing our marketing activities and we can optionally add a Description here.

We’ll just leave that blank for now and click Next.

Next I need to select the audience for whom this email will be sent.

For this exercise we could choose a broad audience. I will simply select Profile and we’ll choose an Age. In our case we need to make sure that our product is sent to everyone older than 21 years old.

We’ll set the rule name, Older than 21, click Confirm.

I can see my audience is 724 recipients. I’ll go ahead now and click Next.

Here is the A/B test parameters screen as part of the wizard. I can choose if I will send it to 2 A/B variants or 3 A/B/C variants of email. I can choose my Winning strategy, we’ll leave it at Best open rate for now. I can choose my Target breakdown. We’ll actually set it to Manual just to see what it is. It’s set to 5% for Variant A, 5% to Variant B and the remainder will be 90%. The Winner sending method will be Automatic and the Test duration is at one day. Let’s change that one day eight hours. Okay, let’s click Next.

Here’s my email now. For this email I’m going to choose to change the content and bring in some sample content from the system. I see that it imports that and in this case we are testing on the Subject line to do an A test and a B test. Or an A variant and a B variant. Hello this is the A variant.

Now part of the tool that makes it so easy is I can just right here click and offer the B variant. This is the B variant.

So I can quickly see I have two different variations of this email. Everything looks good and I’ll click Create. The system generates that email and I’m now brought to the preparation screen for this A/B test. Let’s quickly review all the settings. A/B test parameters, 5% will go with Variant A, 5% with Variant B and we’ll wait one day eight hours to send the remainder of 90% of my target population. My Audience is simply older than 21 and we will tell the system to send it immediately, one day and eight hours later based on the winning result of the higher open rate.

Looks good, let’s Prepare it, click OK.

And it’s preparing to send for an A/B test.

There we go looks like we have targeted 724 recipients. 616 recipients will be delivered to, for various reasons Adobe Campaign Standard has limited population based on unsubscribes or quarantines or blacklists. So we can see here Variant A will go to 30 people, Variant B will go to 30.

Looks good, I’ll go ahead and click Confirm.

And now I can see that the A/B test is in progress, it’s ready to go for one day, seven hours, 59 minutes and the clock is running. We can see that it’s delivering those emails and once they’re all sent it will send the remainder to those 90% of our population. In summary, this demonstration showed you the basics of creating an A/B test using Adobe Campaign Standard. -

See the product documentation for more details on how to test the content or the sender.

On this page