An Auto-Allocate activity identifies a winner among two or more experiences and automatically reallocates more traffic to the winner while the test continues to run and learn. The Analytics for Target (A4T) integration for Auto-Allocate allows you to see your reporting data in Adobe Analytics, and you can even optimize for custom events or metrics defined in Analytics.
Although rich analysis capabilities are available in Adobe Analytics Analysis Workspace, a few modifications to the default Analytics for Target panel are required to correctly interpret Auto-Allocate activities, due to the nuances in optimization criteria.
This tutorial walks through the recommended modifications for analyzing Auto-Allocate activities in Analysis Workspace. The key concepts are:
Visitors should always be used as the normalizing metric in Auto-Allocate activities.
When the metric is an Adobe Analytics metric, the appropriate numerator for the conversion rate depends on the type of optimization criteria chosen during activity setup.
When your optimization metric is a Target defined conversion metric, the default Analytics for Target panel in Analysis Workspace handles configuring your panel.
For all Auto-Allocate activities created before the Target Standard/Premium 23.3.1 release (March 30, 2023) Analytics Workspace and Target display the same value for Confidence.
For all Auto-Allocate activities created after March 30, 2023, the confidence interval values seen in Analysis Workspace do not reflect the more conservative statistics used by Auto-Allocate if these activities have both of the following conditions:
If both of these conditions exist, the confidence metrics should be removed from the A4T panel. Instead, reference these values in Target reporting.
To create an A4T for Auto-Allocate report start with the Analytics for Target panel in Analysis Workspace, as shown below. Then make the following selections:
Figure 1: Analytics for Target panel setup for Auto-Allocate activities.
You can also arrive at a pre-built Analytics for Target panel if you click the link from the report screen in Adobe Target.
The default A4T panel handles Auto-Allocate activities in which the goal metric is either a Target conversion or an Analytics metric with optimization criterion “Maximize Metric Value Per Visitor.”
One example of this panel is shown for the Revenue metric, where “Maximize Metric Value Per Visitor” was selected as the optimization criteria at activity creation time. As previously mentioned, Auto-Allocate uses more conservative confidence calculations compared to the ones used in the Analytics for Target panel. Adobe recommends that you remove the confidence metric from the A4T panel, as well as the related lower and upper lift metrics. Instead, please reference these values in Target reporting.
Figure 2: The recommended report for Auto-Allocate activities with an Analytics metric “Maximize Metric Value Per Visitor optimization” criteria. For these types of metrics, as well as Target defined conversion metrics, the default Analytics for Target panel in Analysis Workspace can be used.
When an Adobe Analytics metric is used with an optimization criterion of Maximize Unique Visitor Conversion Rate, the default Analytics for Target panel in Analysis Workspace must be modified.
The success metric is now a count of unique visitors for whom the conversion metric was positive. This can be achieved by creating a segment that filters to hits with a positive value of the metric. Create this segment as follows:
An example of the segment creation is shown in the figure below, where you select Visitors With Positive Revenue.
Figure 3: Segment creation for Adobe Analytics metrics with optimization criteria equal to “Maximize Unique Visitor Conversion Rate.” In this example, the metric is Revenue, and the optimization goal is to maximize the number of visitors with positive revenue.
After the appropriate segment has been created, the default Analytics for Target panel in Analysis Workspace can be modified.
Add a second Unique Visitors metric alongside the existing Visitors metric column.
Drag the newly-created segment under the first column to produce a panel that resembles Figure 4. Notice the difference: the number of unique visitors with positive revenue is a fraction of the total number of unique visitors assigned to each experience.
Figure 4: Filtering Unique Visitors by the newly created segment
A conversion rate can be quickly calculated by highlighting both the first and second columns, right-clicking, selecting Create Metric from selection > Divide.
The default conversion rate should be removed and replaced with this new calculated metric, as shown in the image below. You might need to edit the newly created calculated metric to display as a Format > Percent up to two decimal places as shown.
Figure 5: The final Auto-Allocate panel showing the conversion rates for a binarized revenue conversion metric
The steps in this tutorial demonstrated how to correctly configure Analysis Workspace to display Auto-Allocate reporting data.