Service instance details
There are two ways to view service instance details: from the dashboard or within the service instance.
Service instance dashboard
To view an overview of the service instance details within the dashboard, select a service instance container, avoiding the hyperlink that is attached to the name. This opens a right rail that provides additional details. The controls contain the following:
- Edit: Selecting Edit allows you to modify an existing service instance. You can edit the name, description, and scoring frequency of the instance.
- Clone: Selecting Clone copies the currently selected service instance set up. You can then modify the workflow to make minor tweaks and rename it as a new instance.
- Delete: You can delete a service instance, including any historical runs.
- Data source: A link to the dataset used by this instance.
- Run Frequency: How often a scoring run takes place and when.
- Score definition: A quick overview of the goal you configured for this instance.
Show more insights dropdown
The second way to view additional details for a service instance is located within the insights page. Select Show more in the top-right to populate a drop down. Details are listed such as the score definition, when it was created, the propensity type, and the datasets used. For more information on any of the properties listed, please visit Configuring a Customer AI instance.
Customer AI dataset preview popover
If more than one dataset is used by Customer AI, a hyperlink labeled **Multiple ** followed by the number of datasets in brackets ()
is provided.
Selecting the multiple datasets link opens the Customer AI dataset preview popover. Each color in the preview represents a dataset as shown by the color key to the left of the dataset columns. In this example, you can see that only Dataset 1 contains the PROP1
column.
Edit an instance
To edit an instance, select Edit in the top-right navigation.
The edit dialog box appears, allowing you to edit the name, description, status, and scoring frequency of the instance. To confirm your changes and close the dialog, select Save in the bottom-right corner.
More actions
The More actions button is located in the top-right navigation next to Edit. Selecting More actions opens a dropdown that allows you to select one of the following operations:
- Clone: Selecting Clone copies the service instance set up. You can then modify the workflow to make minor tweaks and rename it as a new instance.
- Delete: Deletes the instance.
- Access scores: Selecting Access scores opens a dialog providing a link to the downloading scores for Customer AI tutorial, the dialog also provides the dataset id required for making API calls.
- View run history: A dialog containing a list of all the scoring runs associated with the service instance appears.
Scoring summary
Scoring summary displays the total number of profiles scored and categorizes them into buckets containing high, medium, and low propensity. The propensity buckets are determined based on score range, low is less than 24, medium is 25 to 74, and high is above 74. Each bucket has a color corresponding to the legend.
You can hover over any color on the ring to view additional information, such as a percentage and total number of profiles belonging to a bucket.
Distribution of Scores
The Distribution of Scores card gives you a visual summary of the population based on the score. The colors that you see in the Distribution of Scores card represent the type of propensity score generated. Hovering over any of the scoring distributions provides the exact count belonging to that distribution.
Influential factors
For each score bucket, a card is generated that shows the top 10 influential factors for that bucket. The influential factors give you additional details on why your customers belong to various score buckets.
Influential factor drilldowns
Hovering over any of the top influential factors further breaks down the data. You are provided an overview as to why certain profiles belong to a propensity bucket. Depending on the factor, you may be given number, categorical, or boolean values. The example below displays categorical values by region.
Additionally, using drilldowns, you are able to compare a distribution factor if it occurs in two or more propensity buckets and create more specific segments with these values. The following example illustrates the first use case:
You can see that profiles with low propensity to convert are less likely to have made a recent visit to the adobe.com webpages. The “Days since last webVisit” factor has only 8% coverage compared to 26% in medium propensity profiles. Using these numbers, you can compare the distribution within each bucket for the factor. This information can be used to infer that the recency in webvisit is not as influential in the low propensity bucket, as it is in medium propensity bucket.
Create a segment
Selecting the Create Segment button in any of the buckets for low, medium, and high propensity redirects you to the segment builder.
The segment builder is used to define a segment. When selecting Create Segment from the Insights page, Customer AI automatically adds the selected buckets information to the segment. To finish creating your segment, simply fill in the Name and Description containers located in the right rail of the segment builder user interface. After you have given the segment a name and description, select Save in the top-right.
To view your new segment in the Experience Platform UI, select Segments in the left navigation. The Browse page appears and displays all available segments.
Historical performance
The Performance summary tab shows the actual churn or conversion rates, separated into each of the propensity buckets scored by Customer AI.
Initially only expected rates (dotted lines) are displayed. Expected rates are displayed when a scoring run has not occurred and data is not yet available. However, once an outcome window has passed, the expected rate is replaced with an actual rate (solid line).
Hovering over the lines displays the date and actual/expected rate for that day in that bucket.
You can filter the timeframe for the expected and actual rates being displayed. Select the calendar icon
Individual scoring run rates
The bottom half of the Performance summary tab displays the results for each individual scoring run. Select the dropdown date in the top-right to display results for a different scoring run.
Depending on if you are predicting churn or conversion, the Distribution of Scores graph displays the distribution of profiles churned/converted and not churned/not converted in each increment.
Model evaluation
In addition to tracking the predicted and actual outcomes over time on the Historical Performance tab, marketers have even more transparency over model quality with the Model Evaluation tab. You can use the Lift and Gains charts to determine the differences in using a predictive model vs random targeting. Additionally, you are able to determine how many positive outcomes would be captured at each score cutoff. This is useful for segmentation and for aligning return on investment with marketing actions.
Lift chart
The lift chart measures the improvement of using a predictive model instead of random targeting.
High quality model indicators include:
- High lift values in the first few deciles. This means that the model is good at identifying the users with the highest propensity to take the action of interest.
- Descending lift values. This means that Customers with higher scores are more likely to take the action of interest than people with lower scores.
Gains chart
The cumulative gains chart measures the percentage of positive outcomes captured by targeting scores above a certain threshold. After sorting the customers by propensity score from high to low, the population is split into deciles - 10 equally sized groups. A perfect model would capture all of the positive outcomes in the highest score deciles. A baseline random targeting method captures positive outcomes proportionally to the size of the group - targeting 30% of the users would capture 30% of the outcomes.
High quality model indicators include:
- The cumulative gains approach 100% quickly.
- The cumulative gains curve for the model is closer to the upper left corner of the chart.
- The cumulative gains chart can be used to determine the score cutoffs for segmentation and targeting. For example, if the model captures 70% of the positive outcomes in the first 2 score deciles, targeting users with PercentileScore > 80 is expected to capture approximately 70% of the positive outcomes.