Configure content experiments for emails

With the first release of content experimentation in Adobe Journey Optimizer, customers will be able to use A-B testing to hypothesize, explore, and prove out which content works best to drive business objectives. Content experiments can be configured for email, push, SMS, in-app, and web campaigns in A-J-O. This demo provides a walkthrough of the setup of an A-B test for content experimentation of email. Imagine a situation where a marketer is having low open rates of a product announcement email. The marketer wants to test if different subject lines would grab users’ attention and inspire them to open the email and click through a web experience. In A-J-O, the marketer can create a campaign to test and use an experiment report to see which users interact with different subject lines. After naming the campaign, providing a description, and choosing an audience, the marketer will design the first email. For this experiment, the same email template will be used for each treatment. The images, links, and text in each email will be the same. The only changes will be in the subject line of the email. With the first email and subject line created, the next step is to configure the experiment. The steps to configure an experiment are the same for any channels available in A-J-O. When configuring an experiment, the success metric is selected, the number of treatments is defined, and the distribution for each treatment is set. This experiment will test four treatments with an even distribution across all treatments. The experiment is now configured. The subject lines are added to each treatment using the content editor. In the content editor, each treatment with its associated email is shown. The subject lines are updated in the subject line field. If this experiment was testing different variations of images, links, and text in the email, these could be made by editing the email body in each treatment. Once all the treatments are configured, the remaining settings of this campaign are configured. This campaign is configured to start when the campaign is activated. The experiment will also start on activation. Campaigns with experiments can also be set to start at a scheduled time and date. Each treatment in the experiment can be reviewed before activating. The different email treatments will now be sent to the audience based on the defined distribution. The experiment will begin calculating success of the metrics defined and the lift in confidence of each treatment. Experimentation in AJO uses anytime valid confidence sequences which allow marketers to continuously monitor experiments and safely interpret results as the experiment runs. This is a demo of a report from a previous experiment. The experiment report for a campaign provides the outcome of the experiment, signals the winning treatment, the impact to the success metric, and the lift in confidence details. This experiment has reached statistical significance and treatment B is the best performer. After running an experiment, there are several possible next steps. The most common action for this scenario would be to use the winning subject line for future messages. Another follow up could be running different subject line tests or running different subject lines with different audience segments. Thank you for watching this demo of content experimentation in Adobe Journey Optimizer.