Analyze an Auto-Target Activity using the A4T Panel

Transcript

Hi, I’m Kati McKinney, an Expert Solutions Consultant for Adobe Analytics. In this video, I’m going to show you how to use the Analytics for Target integration, to visualize results of an Auto-Target test. Analytics for Target, A4T is the integration between Adobe Analytics and Adobe Target, where you can use Analytics to analyze Target results. An Auto-Target test in Target leverages AI to serve the most tailored experience to each visitor based on individual customer profiles and the behavior of previous visitors with similar profiles. Due to this, measuring the results of an Auto-Target activity is different than measuring the result of a simple A/B test. So today, I’ll show you how you can easily measure results of this activity via Adobe Analytics. So, let’s get started. First, I’m going to drag in the Analytics for Target panel into my workspace. I want it to set a date range of my test, January 1st through the 13th, in this case. And I have a few prompts here. First, Target Activity. I’m going to go ahead and choose a test that I want to measure. And we can see when I did that my Control Experience’s updated. For Control Experience, you can choose any choice because we’re going to overwrite this later. This is because the Control Experience in an Auto-Target test is really setting a control strategy to serve random experiences. Next, we’re going to choose a Normalizing Metric. I’m going to choose Visits for an Auto-Target test. Always choose Visits as a normalizing metric for this type of experience. Auto-Target personalization selects an experience for a visitor once per visit which means the experience can change upon every visit. If we were to use unique visitors, a single user could see multiple experiences which would mean that the conversion rate was misleading. And then finally, we’ll choose Success Metric of Activity Conversions. You should generally view reports with the same metric chosen for optimization when you set up your activity within Target and we are going to go ahead and hit, Build on this panel. And we’ll see two visualizations once this builds, a Freeform table and a Line graph. We’re going to spend most of our time in this Freeform table. Couple of things to note, Lift and Confidence are not available for Control versus Targeted dimensions on Auto-Target activities. You can compute these manually by downloading the Confidence Calculator because of that we’re going to go ahead and remove them from our table. The goal of a standard A/B Target test is to understand the Experiences versus Control. For an Auto-Target test, we need to compare the control strategy versus the targeted strategy. So, we are going to select Targeted versus Control and drag those replacing Target Experiences. Now, to gain further insight into how the model is performing, we want to break down Targeted versus Control by those experiences. So, we’ll go to the dimension of Targeted Experiences and select experiences, A, B, and C. Drag them into Control to break down further and we’ll do the same with Targeted. All right, we have a few more changes that we need to make. First, we need to create a filter for visits. Why do we need to create a filter here? While Adobe Analytics default counting methodology may include visits where a user did not interact with the Target activity. To ensure that our audience interacted with the activity, we need this filter. So, let’s go ahead and create a new segment. We will title it and we can bring in, Target Activities and we’ll want to make sure that that equals the activity that we’re measuring currently. And then finally, we want it to be an Instance Attribution model, and we will go ahead and save.

Now that we have that created, we can drag it under Visits and we’ll only be seeing Visits that had that specific hit for a Target activity. Next, we need to align the Attribution model between the machine learning model and the goal metric. So, Target cannot wait on the default Attribution model within Analytics which is 30 days in order to train its models. So, we need to change the model on the goal metric to be the same touch, otherwise, we would have discrepancies. So, we’re going to hit the gear icon and choose a non-default Attribution model.

And we’re going to use participation in this case with a look back window of Visit and we’ll go ahead and hit, Apply. Finally, we need to create a calculated metric on the conversion rate to leverage both the right attribution model and the filtered Visits metric. So, we are going to create a new metric and we’ll name it.

We want to have this as a percentage with two decimal places.

And now, we’ll drag in our Activity conversions. And again, we need to change the Attribution model just like we have in our Visualization. So, we’re going to choose Participation with a visit look back, and we’ll go ahead and apply that.

And now we need to add another container and we will divide it. And we’re going to bring in that filter we just created and Visits. So, we can ensure we’re only seeing visits with that particular hit.

And we will save this calculated metric. Now, we can drag this in and replace conversion rate.

So this conversion rate will take into account the participation Attribution model for Activity conversions, as well as, the visits with the filter. So this is what your final workspace should look like after using the A4T panel for an Auto-Target test and making just a few tweaks. So, what I see here is that indeed, my Target experiences outperformed my Control experiences. Remember, the machine learning model was changing experiences to drive more conversions during the test. So, the Target experiences should always outperform Control. This is demo data. So, it’s not as compelling as your data maybe. To learn more about the important attributes in your tests, navigate to the Reports portion of Target and click on the Important Attributes icon. Thanks for watching. - -