Create an A/B test

Marketo’s email A/B testing feature allows you to modify components of your emails, set testing criteria, then let Marketo automatically send the winning email to a segment of your audience. You choose what you want to test, like subject lines, sender information, dates, times, or content, then analyze and use the test results to make improvements to your email marketing efforts. In our example, we’ll be showing you how to create an A/B email test within a program with a targeted audience, set the testing variables for the A/B test, define how the winner of the A/B test will be determined, schedule when to send the test, what duration the test should run, when to send the winning campaign and how to be notified of the results and how to view the A/B test results.

To begin let’s log into your Marketo account and navigate to the marketing activities tab.

Let’s start by creating an email send program, right click the campaign folder for your marketing content and select new program. Fill out the fields in the new program window. For campaign folder, choose the campaign folder you want the email program to live in. For name, use your company’s naming convention standards for all your program names. For this example, we’ll name the program 2017-07 EM Summer Sale. For program type choose email. In order to use the email send program, for channel choose email send, for description, add a description of your program, for this example, promotional content for summer products, then click create. Once the program is created, the control panel displays four tiles with action needed to complete the email setup process. In the audience tile we’ll define the target audience we want to send our email to. We can import a new list or edit a smart list to create the target audience. Back on the control panel, we’ll take a look at the email tile. When performing A/B tests you’ll need to have an email created before setting up the test. This email will serve as the A for your A/B test. You can create a new email or select an existing email you’ve created from the dropdown. We’ll go ahead and build out an email and send a sample and approve and close. To learn more about building an email head over to the video tutorial titled Building a Scheduled Email Program.

Back in the email tile we’ll click add A/B test and Marketo will open the A/B test setup wizard in a new window. In the test setting step we’ll choose a test type. When testing subject lines, you’ll select the test email then enter additional subject lines to compare. We recommend limiting your test to no more than four subject lines so it’s easier to draw insightful conclusions from the results. Click the plus sign to add additional subject lines and the minus sign to remove any. When testing whole emails you can test up to 10 emails at one time. Here you’ll select the different emails you’ve created for your test from the dropdowns and click add for each one. When testing from address you’ll select the test email then enter the from address, names and email addresses into the provided fields. Studies have shown that people tend to open email more frequently when the sender uses an actual person’s name, rather than a company name. Using a company name can make recipients feel like they are being pressured into buying something. Whereas an email using a real person’s name can feel more personal. Click the plus sign to add additional names and addresses and the minus sign to remove any. When testing send dates and times you can add titles like morning, afternoon, or night, then set the dates and times for the send. Drag the slider to set the percentage of people you want to receive the test and how many will receive the winning campaign.

After making your selections click next to continue on to the winner criteria step. Here you’ll select the criteria that serves as the basis for the test. Do you want the winning email to be declared based on highest amount of opens, clicks, click to open percentage, engagement score or based on a customized criteria? Next you’ll select how you’ll declare the winning email. The automatic option lets Marketo calculate the winner based on the criteria set and send the winner when the time is scheduled. The manual option allows you to analyze the results and send the winning email at the time of your choice. When testing send times and dates, you’ll need to declare your winner manually.

Now let’s schedule the email for testing. Click the next button to move on to the schedule step. Here, you’ll set a date and time to send the test and a date and time to send the winning email.

The test duration calculates the amount of time in between your tests send and when your winning email is scheduled to send. If you’d like to be notified when the winning email is sent enter in the email address you’d like the notification sent to, click next to move onto the finish step. Here you’ll see a summary of the A/B test settings, then click close. Back in the email program, you’ll see the scheduled tile has been completed as part of the A/B test setup. And all you’ll need to do now is approve the program to finalize it.

When the A/B test has been sent and results begin to accumulate, you can view the engagement of your test for a deeper understanding of how it performed. On the email program dashboard you’ll see four sections of reporting metrics. Send test, where you can see the total delivered emails and the delivery breakdowns of each variant. Clicks, where you can see how many clicks each variant received. Clicks to open, where of the recipients who opened the email, how many clicked. And summary, where you can compare clicks and unsubscribes per variant. With the results of your A/B test, you’ll be able to make improvements to your communications based on what you’ve learned from your audience. -