Configure content experiments for in-app messages

Hi, this is Justin. Today I’m going to show you how to make an experiment using an in-app message. So in this case we have a travel booking app that we’re interested in that can book short term stays and long term stays. And what I want to do is I want to show you how to set up an in-app message and then run an experiment with two different offers to see how they perform. So what I’m going to come and do is I’m going to create a new campaign. It’ll be an in-app type. I’ll go ahead and select my surface. I’ll go ahead and give it a name. I’m going to select all the users of the app and then I’ll go ahead and create an experiment. In this case I’ll have two treatments and a holdout. So I will add the holdout and then I’ll add treatment A, treatment B. You can see I have an even split between those. I could adjust that if I want. And then finally I’m going to select my metric. In this case I’m going to do inbound clicks. And now my experiment is created. Now that my experiment is created it’s time to go ahead and create the content. So for the first one I’m going to feature destinations from around the world. So I’ll give it a little bit different design and give it a title text. Fabulous destinations. See the world with our world class deals. And then for my second one I’m going to feature more long-term types of opportunities. So we’ll have a different image. We’ll do the same design and then we’ll do. So now that I have that all ready to go I’m going to go back, take a look at my campaign and then I will go ahead and review to activate this campaign. So I’ll take a look there looks like there’s no errors so I can go ahead and review this campaign and then activate it. So I’m going to activate it. Then once it’s activated I can browse my app. I’m going to simulate that here by hitting a send track action. And now you can see that the in-app pops appropriately and with just with a few clicks I’ve set up an experiment. I’m able to test multiple versions against that and then we’ll be able to look at the reports here shortly. Now we can take a look at the reports. Here I can see which treatment is the winner and how much lift is being generated from that. I can drill in to compare the metrics and see how they interact with each other. And then finally we have this chart with the confidence intervals that allows us to see which treatments are the ones that have the most statistical significance and what the spread of those are. With this report you can judge which treatment was the most effective for your experiment. I hope you’ve enjoyed our brief overview of experimentation.