Sometimes activities don’t go as expected. Here are some potential challenges you might face while using Automated Personalization, and some suggested solutions.
There are several activity setup changes that can decrease the expected time to build models, including the number of experiences in your Automated Personalization test, the traffic to your site, and your selected success metric.
Solution: Review your activity setup and see if there are any changes you are willing to make to improve the speed at which models will build.
There are several factors required for an AP activity to generate lift:
Solution: The best course of action is to first make sure the content and locations that make up the activity experiences truly make a difference to the overall response rates using a simple, non-personalized A/B test. Be sure to compute the sample sizes ahead of time to ensure there is enough power to see a reasonable lift and run the A/B test for a fixed duration without stopping it or making any changes. If an A/B test results show statistically significant lift on one or more of the experiences, then it is likely that a personalized activity will work. Of course, personalization can work even if there are no differences in the overall response rates of the experiences. Typically, the issue stems from the offers/locations not having a large enough impact on the optimization goal to be detected with statistical significance.
In AP, the URL and template testing rules are added to the Target request entry constraint (for example, target-global-mbox), where they are evaluated only once. Once a user qualifies for an activity, the Target-request-level targeting rules are not re-evaluated. However, the targeting audience is added to location targeting rules.
Solution: Add the necessary template rules as the input-audience of the campaign. Audience evaluation happens upon each request/call.
This will be fixed in an upcoming release.
This is expected.
In an AP activity, once a conversion metric (whether optimization goal or post goal) is converted, the user is released from the experience and the activity is restarted.
For example, there is an activity with a conversion metric (C1) and an additional metric (A1). A1 is dependent on C1. When a visitor enters the activity for the first time, and the criteria for converting A1 and C1 are not converted, metric A1 is not converted due to the success metric dependency. If the visitor converts C1 and then converts A1, A1 is still not converted because as soon as C1 is converted, the visitor is released.