Service instance details

There are two ways to view service instance details: from the dashboard or within the service instance.

Service instance dashboard

To view an overview of the service instance details within the dashboard, select a service instance container, avoiding the hyperlink that is attached to the name. This opens a right rail that provides additional details. The controls contain the following:

  • Edit: Selecting Edit allows you to modify an existing service instance. You can edit the name, description, and scoring frequency of the instance.
  • Clone: Selecting Clone copies the currently selected service instance set up. You can then modify the workflow to make minor tweaks and rename it as a new instance.
  • Delete: You can delete a service instance, including any historical runs.
  • Data source: A link to the dataset used by this instance.
  • Run Frequency: How often a scoring run takes place and when.
  • Score definition: A quick overview of the goal you configured for this instance.

NOTE
In the event that a scoring run fails, an error message is provided. The error message is listed under Last run details in the right rail which is only visible to failed runs.

failed run message

Show more insights dropdown

The second way to view additional details for a service instance is located within the insights page. Select Show more in the top-right to populate a drop down. Details are listed such as the score definition, when it was created, the propensity type, and the datasets used. For more information on any of the properties listed, please visit Configuring a Customer AI instance.

show more

Customer AI dataset preview popover

If more than one dataset is used by Customer AI, a hyperlink labeled **Multiple ** followed by the number of datasets in brackets () is provided.

multiple datasets

Selecting the multiple datasets link opens the Customer AI dataset preview popover. Each color in the preview represents a dataset as shown by the color key to the left of the dataset columns. In this example, you can see that only Dataset 1 contains the PROP1 column.

show more

Edit an instance

To edit an instance, select Edit in the top-right navigation.

click the edit button

The edit dialog box appears, allowing you to edit the name, description, status, and scoring frequency of the instance. To confirm your changes and close the dialog, select Save in the bottom-right corner.

edit popover

More actions

The More actions button is located in the top-right navigation next to Edit. Selecting More actions opens a dropdown that allows you to select one of the following operations:

  • Clone: Selecting Clone copies the service instance set up. You can then modify the workflow to make minor tweaks and rename it as a new instance.
  • Delete: Deletes the instance.
  • Access scores: Selecting Access scores opens a dialog providing a link to the downloading scores for Customer AI tutorial, the dialog also provides the dataset id required for making API calls.
  • View run history: A dialog containing a list of all the scoring runs associated with the service instance appears.

more actions

Scoring summary

Scoring summary displays the total number of profiles scored and categorizes them into buckets containing high, medium, and low propensity. The propensity buckets are determined based on score range, low is less than 24, medium is 25 to 74, and high is above 74. Each bucket has a color corresponding to the legend.

NOTE
If it is a conversion propensity score, the high scores show in green and the low scores in red. If you are predicting churn propensity this is flipped, the high scores are in red and the low scores are green. The medium bucket remains yellow regardless of what propensity type you choose.

scoring summary

You can hover over any color on the ring to view additional information, such as a percentage and total number of profiles belonging to a bucket.

Distribution of Scores

The Distribution of Scores card gives you a visual summary of the population based on the score. The colors that you see in the Distribution of Scores card represent the type of propensity score generated. Hovering over any of the scoring distributions provides the exact count belonging to that distribution.

distribution of scores

Influential factors

For each score bucket, a card is generated that shows the top 10 influential factors for that bucket. The influential factors give you additional details on why your customers belong to various score buckets.

Influential factors

Influential factor drilldowns

Hovering over any of the top influential factors further breaks down the data. You are provided an overview as to why certain profiles belong to a propensity bucket. Depending on the factor, you may be given number, categorical, or boolean values. The example below displays categorical values by region.

drilldown screenshot

Additionally, using drilldowns, you are able to compare a distribution factor if it occurs in two or more propensity buckets and create more specific segments with these values. The following example illustrates the first use case:

You can see that profiles with low propensity to convert are less likely to have made a recent visit to the adobe.com webpages. The “Days since last webVisit” factor has only 8% coverage compared to 26% in medium propensity profiles. Using these numbers, you can compare the distribution within each bucket for the factor. This information can be used to infer that the recency in webvisit is not as influential in the low propensity bucket, as it is in medium propensity bucket.

Create a segment

Selecting the Create Segment button in any of the buckets for low, medium, and high propensity redirects you to the segment builder.

NOTE
The Create Segment button is only available if Real-Time Customer Profile is enabled for the dataset. For more information on how to enable Real-Time Customer Profile, visit the Real-Time Customer Profile overview.

Click create segment

Create a segment

The segment builder is used to define a segment. When selecting Create Segment from the Insights page, Customer AI automatically adds the selected buckets information to the segment. To finish creating your segment, simply fill in the Name and Description containers located in the right rail of the segment builder user interface. After you have given the segment a name and description, select Save in the top-right.

NOTE
Since the propensity scores are written to the individual profile, they are available in the Segment builder like any other profile attributes. When you navigate to the segment builder to create new segments you can see all the various propensity scores under your namespace Customer AI.

Segment fill in

To view your new segment in the Experience Platform UI, select Segments in the left navigation. The Browse page appears and displays all available segments.

All your Segments

Historical performance

The Performance summary tab shows the actual churn or conversion rates, separated into each of the propensity buckets scored by Customer AI.

Performance summary tab

Initially only expected rates (dotted lines) are displayed. Expected rates are displayed when a scoring run has not occurred and data is not yet available. However, once an outcome window has passed, the expected rate is replaced with an actual rate (solid line).

Hovering over the lines displays the date and actual/expected rate for that day in that bucket.

Bucket example

You can filter the timeframe for the expected and actual rates being displayed. Select the calendar icon icon then select a new date range. The results in each of the buckets are updated to display within the new date range.

Date selector

Individual scoring run rates

The bottom half of the Performance summary tab displays the results for each individual scoring run. Select the dropdown date in the top-right to display results for a different scoring run.

Depending on if you are predicting churn or conversion, the Distribution of Scores graph displays the distribution of profiles churned/converted and not churned/not converted in each increment.

individual scoring

Model evaluation

In addition to tracking the predicted and actual outcomes over time on the Historical Performance tab, marketers have even more transparency over model quality with the Model Evaluation tab. You can use the Lift and Gains charts to determine the differences in using a predictive model vs random targeting. Additionally, you are able to determine how many positive outcomes would be captured at each score cutoff. This is useful for segmentation and for aligning return on investment with marketing actions.

Lift chart

lift chart

The lift chart measures the improvement of using a predictive model instead of random targeting.

High quality model indicators include:

  • High lift values in the first few deciles. This means that the model is good at identifying the users with the highest propensity to take the action of interest.
  • Descending lift values. This means that Customers with higher scores are more likely to take the action of interest than people with lower scores.

Gains chart

gains chart

The cumulative gains chart measures the percentage of positive outcomes captured by targeting scores above a certain threshold. After sorting the customers by propensity score from high to low, the population is split into deciles - 10 equally sized groups. A perfect model would capture all of the positive outcomes in the highest score deciles. A baseline random targeting method captures positive outcomes proportionally to the size of the group - targeting 30% of the users would capture 30% of the outcomes.

High quality model indicators include:

  • The cumulative gains approach 100% quickly.
  • The cumulative gains curve for the model is closer to the upper left corner of the chart.
  • The cumulative gains chart can be used to determine the score cutoffs for segmentation and targeting. For example, if the model captures 70% of the positive outcomes in the first 2 score deciles, targeting users with PercentileScore > 80 is expected to capture approximately 70% of the positive outcomes.