This topic contains answers to questions that are frequently asked about lift and confidence when using Analytics as the reporting source for Target (A4T).
You can perform offline calculations for A4T, but it requires a step with data exports in Analytics. For more information, see “Performing Offline Calculations for Analytics for Target (A4T)” in Confidence Level and Confidence Interval.
Lift is the percent difference between your control page results and a successful test variant.
The confidence level is the probability that the measured conversion rate differs from the champion page conversion rate for reasons other than chance alone.
Calculated metrics are not currently supported in lift and confidence functions. This is due to the fact that Analytics calculates metrics at an aggregate-level, rather than at a visitor-level. Confidence, in particular, is a visitor-level calculation.
Non-calculated (standard) events are supported in lift and confidence. They become the numerator in the lift function; the numerator cannot be a calculation itself. The denominator is the normalizing metrics (impressions, visits, or visitors). Some examples of standard events include orders, revenue, activity conversions, custom events 1-1000, etc. This means that common optimization metrics, such as conversation rate (Orders/Visitor) and RPV (Revenue/Visitor) are supported in lift and confidence.
Examples of unsupported metrics or use cases include:
A4T uses non-binary metric calculations with the sum of square data. The variance is calculated using the sum of squares data. Extreme orders are not taken into account. Additionally, the confidence calculation does not apply a Bonferroni correction for multiple offers.
Lift and confidence do not work in Ad Hoc or Report Builder, and cannot be calculated yourself for continuous variables. It is possible to calculate it manually for binary metrics.