Setting up A4T reports in Analysis Workspace for Auto-Allocate activities

An Auto-Allocate activity identifies a winner among two or more experiences and automatically reallocates more traffic to the winner while the test continues to run and learn. The Analytics for Target (A4T) integration for Auto-Allocate allows you to see your reporting data in Adobe Analytics, and you can even optimize for custom events or metrics defined in Analytics.

Although rich analysis capabilities are available in Adobe Analytics Analysis Workspace, a few modifications to the default Analytics for Target panel are required to correctly interpret Auto-Allocate activities, due to the nuances in optimization criteria.

This tutorial walks through the recommended modifications for analyzing Auto-Allocate activities in Analysis Workspace. The key concepts are:

• Visitors should always be used as the normalizing metric in Auto-Allocate activities.

• When the metric is an Adobe Analytics metric, the appropriate numerator for the conversion rate depends on the type of optimization criteria chosen during activity setup.

  • The “maximize unique visitor conversion rate” optimization criterion has a conversion rate whose numerator is a count of the unique visitors with a positive value of the metric.
  • The “maximize metric value per visitor” has a conversion rate whose numerator is the regular metric value in Adobe Analytics. This is provided by default in the Analytics for Target panel in Analysis Workspace.

• When your optimization metric is a Target defined conversion metric, the default Analytics for Target panel in Analysis Workspace handles configuring your panel.

• For all Auto-Allocate activities created before the Target Standard/Premium 23.3.1 release (March 30, 2023) Analytics Workspace and Target display the same value for Confidence.

  For all Auto-Allocate activities created after March 30, 2023, the confidence interval values seen in Analysis Workspace do not reflect the more conservative statistics used by Auto-Allocate if these activities have both of the following conditions:

  • Analytics as the reporting source (A4T)
  • Analytics optimization metrics

  If both of these conditions exist, the confidence metrics should be removed from the A4T panel. Instead, reference these values in Target reporting.

Create the A4T for Auto-Allocate panel in Analysis Workspace

To create an A4T for Auto-Allocate report start with the Analytics for Target panel in Analysis Workspace, as shown below. Then make the following selections:

1. Control Experience: You can choose any experience.
2. Normalizing Metric: Select Visitors. Auto-Allocate always normalizes conversion rates by unique visitors.
3. Success Metrics: Select the same metric that you used during activity creation. If this was a Target defined conversion metric, select Activity Conversion. Otherwise, select the Adobe Analytics metric that you used.

Figure 1: Analytics for Target panel setup for Auto-Allocate activities.

Target Conversion metrics or Analytics metrics with “Maximize Metric Value Per Visitor” optimization criteria

The default A4T panel handles Auto-Allocate activities in which the goal metric is either a Target conversion or an Analytics metric with optimization criterion “Maximize Metric Value Per Visitor.”

One example of this panel is shown for the Revenue metric, where “Maximize Metric Value Per Visitor” was selected as the optimization criteria at activity creation time. As previously mentioned, Auto-Allocate uses more conservative confidence calculations compared to the ones used in the Analytics for Target panel. Adobe recommends that you remove the confidence metric from the A4T panel, as well as the related lower and upper lift metrics. Instead, please reference these values in Target reporting.

Figure 2: The recommended report for Auto-Allocate activities with an Analytics metric “Maximize Metric Value Per Visitor optimization” criteria. For these types of metrics, as well as Target defined conversion metrics, the default Analytics for Target panel in Analysis Workspace can be used.

Analytics metrics with “Maximize Unique Visitor Conversion Rate” optimization criteria

When an Adobe Analytics metric is used with an optimization criterion of Maximize Unique Visitor Conversion Rate, the default Analytics for Target panel in Analysis Workspace must be modified.

The success metric is now a count of unique visitors for whom the conversion metric was positive. This can be achieved by creating a segment that filters to hits with a positive value of the metric. Create this segment as follows:

1. Select the Components > Create Segment option in the Analysis Workspace toolbar.
2. Drag the metric used at activity creation time from the left-hand panel to the Definition box of the segment.
3. Select values of the metric that are greater than a numeric value of 0.
4. From the Include drop-down, select Visitors.
5. Give your segment an appropriate name.

An example of the segment creation is shown in the figure below, where you select Visitors With Positive Revenue.

Figure 3: Segment creation for Adobe Analytics metrics with optimization criteria equal to “Maximize Unique Visitor Conversion Rate.” In this example, the metric is Revenue, and the optimization goal is to maximize the number of visitors with positive revenue.

After the appropriate segment has been created, the default Analytics for Target panel in Analysis Workspace can be modified.

1. Add a second Unique Visitors metric alongside the existing Visitors metric column.

2. Drag the newly-created segment under the first column to produce a panel that resembles Figure 4. Notice the difference: the number of unique visitors with positive revenue is a fraction of the total number of unique visitors assigned to each experience.

   

   Figure 4: Filtering Unique Visitors by the newly created segment

3. A conversion rate can be quickly calculated by highlighting both the first and second columns, right-clicking, selecting Create Metric from selection > Divide.

   The default conversion rate should be removed and replaced with this new calculated metric, as shown in the image below. You might need to edit the newly created calculated metric to display as a Format > Percent up to two decimal places as shown.

   

   Figure 5: The final Auto-Allocate panel showing the conversion rates for a binarized revenue conversion metric

Summary

The steps in this tutorial demonstrated how to correctly configure Analysis Workspace to display Auto-Allocate reporting data.

To summarize:

• When the metric is a Target defined conversion metric or an Adobe Analytics metric with optimization criterion “Maximize Metric Value Per Visitor,” the default workspace panel configured with visitors as a normalizing metric should be used.
• When the metric is an Adobe Analytics metric with optimization criterion “Maximize Unique Visitor Conversion Rate,” you must use a conversion rate that is defined as the fraction of visitors for whom the metric is positive. This is done by creating a corresponding segment that filters the Unique Visitor metric.

Business.Adobe.com resources