Customer AI, as part of Intelligent Services provides marketers with the power to leverage Adobe Sensei to anticipate what your customers next action is going to be. Customer AI is used to generate custom propensity scores such as churn and conversion for individual profiles at-scale. This is accomplished without having to transform the business needs to a machine learning problem, picking an algorithm, training, or deployment.
This document serves as a guide for interacting with service instance insights in the Intelligent Services Customer AI user interface.
In order to utilize insights for Customer AI, you need to have a service instance with a successful run status available. To create a new service instance visit Configuring a Customer AI instance. If you recently created a service instance and it is still training and scoring, please allow 24 hours for it to finish running.
In the Adobe Experience Platform UI, select Services in the left navigation. The Services browser appears and displays available Intelligent Services. In the container for Customer AI, select Open.
The Customer AI service page appears. This page lists service instances of Customer AI and displays information about them, including the name of the instance, propensity type, how often the instance is run, and the status of the last update.
Only service instances that have completed successful scoring runs have insights.
Select a service instance name to begin.
Next, the insights page for that service instance appears with the option to select Latest scores or Performance summary. The default tab Latest scores provides visualizations of your data. The visualizations and what you can do with the data are explained in more detail throughout this guide.
The Performance summary tab shows the actual churn or conversion rates for each propensity bucket. To learn more, see the section on performance summary metrics.
There are two ways to view service instance details: from the dashboard or within the service instance.
To view an overview of the service instance details within the dashboard, select a service instance container, avoiding the hyperlink that is attached to the name. This opens a right rail that provides additional details. The controls contain the following:
In the event that a scoring run fails, an error message is provided. The error message is listed under Last run details in the right rail which is only visible to failed runs.
The second way to view additional details for a service instance is located within the insights page. Select Show more in the top-right to populate a drop down. Details are listed such as the score definition, when it was created, the propensity type, and the datasets used. For more information on any of the properties listed, please visit Configuring a Customer AI instance.
If more than one dataset is used by Customer AI, a hyperlink labeled **Multiple ** followed by the number of datasets in brackets
() is provided.
Selecting the multiple datasets link opens the Customer AI dataset preview popover. Each color in the preview represents a dataset as shown by the color key to the left of the dataset columns. In this example, you can see that only Dataset 1 contains the
To edit an instance, select Edit in the top-right navigation.
The edit dialog box appears, allowing you to edit the name, description, status, and scoring frequency of the instance. To confirm your changes and close the dialog, select Save in the bottom-right corner.
The More actions button is located in the top-right navigation next to Edit. Selecting More actions opens a dropdown that allows you to select one of the following operations:
Scoring summary displays the total number of profiles scored and categorizes them into buckets containing high, medium, and low propensity. The propensity buckets are determined based on score range, low is less than 24, medium is 25 to 74, and high is above 74. Each bucket has a color corresponding to the legend.
If it is a conversion propensity score, the high scores show in green and the low scores in red. If you are predicting churn propensity this is flipped, the high scores are in red and the low scores are green. The medium bucket remains yellow regardless of what propensity type you choose.
You can hover over any color on the ring to view additional information, such as a percentage and total number of profiles belonging to a bucket.
The Distribution of Scores card gives you a visual summary of the population based on the score. The colors that you see in the Distribution of Scores card represent the type of propensity score generated. Hovering over any of the scoring distributions provides the exact count belonging to that distribution.
For each score bucket, a card is generated that shows the top 10 influential factors for that bucket. The influential factors give you additional details on why your customers belong to various score buckets.
Hovering over any of the top influential factors further breaks down the data. You are provided an overview as to why certain profiles belong to a propensity bucket. Depending on the factor, you may be given number, categorical, or boolean values. The example below displays categorical values by region.
Additionally, using drilldowns, you are able to compare a distribution factor if it occurs in two or more propensity buckets and create more specific segments with these values. The following example illustrates the first use case:
You can see that profiles with low propensity to convert are less likely to have made a recent visit to the adobe.com webpages. The “Days since last webVisit” factor has only 8% coverage compared to 26% in medium propensity profiles. Using these numbers, you can compare the distribution within each bucket for the factor. This information can be used to infer that the recency in webvisit is not as influential in the low propensity bucket, as it is in medium propensity bucket.
Selecting the Create Segment button in any of the buckets for low, medium, and high propensity redirects you to the segment builder.
The Create Segment button is only available if Real-time Customer Profile is enabled for the dataset. For more information on how to enable Real-time Customer Profile, visit the Real-time Customer Profile overview.
The segment builder is used to define a segment. When selecting Create Segment from the Insights page, Customer AI automatically adds the selected buckets information to the segment. To finish creating your segment, simply fill in the Name and Description containers located in the right rail of the segment builder user interface. After you have given the segment a name and description, select Save in the top-right.
Since the propensity scores are written to the individual profile, they are available in the Segment builder like any other profile attributes. When you navigate to the segment builder to create new segments you can see all the various propensity scores under your namespace Customer AI.
To view your new segment in the Platform UI, select Segments in the left navigation. The Browse page appears and displays all available segments.
The Performance summary tab shows the actual churn or conversion rates, separated into each of the propensity buckets scored by Customer AI.
Initially only expected rates (dotted lines) are displayed. Expected rates are displayed when a scoring run has not occurred and data is not yet available. However, once an outcome window has passed, the expected rate is replaced with an actual rate (solid line).
Hovering over the lines displays the date and actual/expected rate for that day in that bucket.
You can filter the timeframe for the expected and actual rates being displayed. Select the calendar icon then select a new date range. The results in each of the buckets are updated to display within the new date range.
The bottom half of the Performance summary tab displays the results for each individual scoring run. Select the dropdown date in the top-right to display results for a different scoring run.
Depending on if you are predicting churn or conversion, the Distribution of Scores graph displays the distribution of profiles churned/converted and not churned/not converted in each increment.
In addition to tracking the predicted and actual outcomes over time on the Historical Performance tab, marketers have even more transparency over model quality with the Model Evaluation tab. You can use the Lift and Gains charts to determine the differences in using a predictive model vs random targeting. Additionally, you are able to determine how many positive outcomes would be captured at each score cutoff. This is useful for segmentation and for aligning return on investment with marketing actions.
The lift chart measures the improvement of using a predictive model instead of random targeting.
High quality model indicators include:
The cumulative gains chart measures the percentage of positive outcomes captured by targeting scores above a certain threshold. After sorting the customers by propensity score from high to low, the population is split into deciles - 10 equally sized groups. A perfect model would capture all of the positive outcomes in the highest score deciles. A baseline random targeting method captures positive outcomes proportionally to the size of the group - targeting 30% of the users would capture 30% of the outcomes.
High quality model indicators include:
The AUC reflects the strength of the relationship between the ranking by score and the occurrence of the predicted goal. An AUC of 0.5 means the model is no better than a random guess. An AUC of 1 means the model can perfectly predict who will take the relevant action.
This document outlined the insights provided by a Customer AI service instance. You can now continue to the tutorial on downloading scores in Customer AI or browse the other Adobe Intelligent Services guides that are offered.
The following video outlines how to use Customer AI to see the output of the models and influential factors.