Data management license entitlement best practices
Adobe Experience Platform is an open system that transforms your data into robust customer profiles that update in real time and uses AI-driven insights to help you to deliver the right experiences across every channel. You can ingress data of varying types, volumes, and histories to Experience Platform using sources and then cater that data to use cases ranging from segmentation and personalization to analytics and machine learning.
Experience Platform offers licenses that establish the number of profiles that you can create and the amount of data that you can bring in. Given the capacity to bring in any source, volume, or history of data, it is possible to exceed your licensing entitlements as your data volumes grow.
Read this guide for best practices to follow and tools you can use to better manage your license entitlements with Experience Platform.
Summary of features summary-of-features
Use the best practices and tools outlined in this document to better manage your license entitlement usage within Experience Platform. This document is updated as additional features are released to help provide visibility and control to all Experience Platform customers.
The following table outlines the list of currently available features at your disposal, to better manage your license usage entitlement.
Understanding Experience Platform data storage
Experience Platform is primarily composed of two data repositories: the data lake and the Profile store.
Data lake primarily serves the following purposes:
- Acts as the staging area for onboarding data into Experience Platform;
- Acts as the long-term data storage for all Experience Platform data;
- Enables use cases such as data analytics and data science.
The Profile store is where customer profiles are created and primarily serves the following purposes:
- Acts as a data storage for profiles that are used to support real-time experiences;
- Enables use cases such as segmentation, activation, and personalization.
License usage license-usage
When you license Experience Platform, you are provided with license usage entitlements that vary depending on SKU:
Addressable Audience: the total number of customer profiles that are contractually allowed in Experience Platform, including both known and pseudonymous profiles.
Total Data Volume: the total amount of data available for Real-Time Customer Profile to use in engagement workflows.
The availability of these metrics and the specific definition of each of these metrics varies depending on the licensing that your organization has purchased.
License usage dashboard
The Adobe Experience Platform UI provides a dashboard through which you can view a snapshot of your organization’s license-related data for Experience Platform. The data in the dashboard is displayed exactly as it appears at the specific point in time when the snapshot was taken. The snapshot is neither an approximation nor a sample of data, and the dashboard is not updating in real-time.
For more information, see the guide on using the license usage dashboard on Experience Platform UI.
Data management best practices
The following sections outline best practices to follow to better manage your data.
Understanding your data
Not all data is the same in Adobe Experience Platform. Some data may be dense, but low in value, while others may be sparse, but high in value. Some data may lose value as soon as its generated, while others may be valuable for months, if not years.
There are three dimensions to consider in understanding the value of your data:
Data Management tools data-management-tools
There are two central scenarios to consider when ensuring that your data usage remains within your license entitlement limits:
What data to bring into Experience Platform?
Data can be ingested into one or multiple systems in Experience Platform, namely the data lake and/or the Profile store. This means that different data can exist in both systems for a variety of different use cases. For example, you may want to hold historical data in the data lake, but not in the Profile store. You can select which data to send to the Profile store by enabling a dataset for Profile ingestion.
What data to keep?
You can apply both data ingestion filters and expiration rules to remove data that has become obsolete for your use cases. Typically, behavioral data (such as Analytics data) consumes significantly more storage than record data (such as CRM data). For example, many Experience Platform users have upwards of up to 90% of profiles being populated by behavioral data alone, in comparison to that of record data. Therefore, managing your behavioral data is critical in ensuring compliance within your license entitlements.
There are a number of tools that you can leverage to stay within your license usage entitlements:
Identity Service and addressable audience identity-service
Identity graphs do not count towards your total addressable audience entitlement because addressable audience refers to your total count of customer profiles.
However, identity graph limits can affect your addressable audience due to splitting identities. For example, if the oldest ECID is removed from the graph, ECID will continue to exist in Real-Time Customer Profile as a pseudonymous profile. You can set Pseudonymous profile data expirations to circumvent this behavior. For more information, read the guardrails for Identity Service data.
Ingestion filters ingestion-filters
Ingestion filters allow you to bring in only the data that is needed for your use cases and filters out all events that are not required.
Profile store profile-service
The Profile store is composed of the following components:
Profile store Composition Reports
There are a number of reports available to help you understand the composition of the Profile store. These reports help you make informed decisions about how and where to set your Experience Event expirations to better optimize your license usage:
- Dataset Overlap Report API: Exposes the datasets that contribute the most to your Addressable Audience. You can use this report to identify which ExperienceEvent datasets to set an expiration for. See the tutorial on generating the dataset overlap report for more information.
- Identity Overlap Report API: Exposes the identity namespaces that contribute the most to your Addressable Audience. See the tutorial on generating the identity overlap report for more information.
Pseudonymous Profile data expirations pseudonymous-profile-expirations
Use the Pseudonymous profiles data expiration capability to automatically remove data from that is no longer valid or useful for your use cases from the Profile store. Pseudonymous Profile data expiration removes both event and profile records. As a result, this setting will reduce Addressable Audience volumes. For more information on this feature, please read the Pseudonymous Profile data expiration overview.
Dataset UI - Experience Event Dataset retention data-retention
Configure dataset expiration and retention settings to enforce a fixed retention period for your data in data lake and Profile store. Once the retention period ends, data is deleted. Experience Event data expiration only removes events and does not remove profile class data, which will reduce total data volume in license usage metrics. For more information, read the guide on setting data retention policy.
Profile Experience Event expirations event-expirations
Configure expiration times to automatically remove behavioral data from your Profile-enabled dataset once they are no longer valuable for your use cases. Read the overview on Experience Event expirations for more information.
Summary of best practices for license usage compliancy best-practices
The following is a list of some recommended best practices that you can follow to ensure better adherence to your license usage entitlement:
- Use the license usage dashboard to track and monitor customer usage trends. This allows you to get ahead of any potential overages that may incur.
- Configure ingestion filters by identifying the events required for your segmentation and personalization use cases. This allows you to send only important events required for your use cases.
- Ensure that you have only enabled datasets for profile that are required for your segmentation and personalization use cases.
- Configure Experience Event expirations and Pseudonymous Profile data expirations for high-frequency data like web data.
- Configure Time-to-Live (TTL) retention policies for Experience Event datasets in the data lake to automatically remove outdated records and optimize storage usage in line with your license entitlements.
- Periodically check the Profile Composition Reports to understand your Profile store composition. This allows you to understand the data sources contributing most to your license usage consumption.
Use case: License Usage Compliance
Why consider this use case
By ensuring your compliance with license usage provisions for both data lake and Profile storage, you can confidently prevent overages, optimize costs, and align your data retention policies with your business requirements.
Prerequisites and planning
Consider the following prerequisites in your planning process:
-
Access and permissions:
- Ensure that you have the Manage Datasets permission to use Experience Event TTL.
- Ensure that you have the Manage Profile Settings to use Pseudonymous Profile TTL.
-
Understanding of data retention policy:
- Organizational policies regarding data retention and compliance
- Business needs for data analytics and segmentation lookback windows
UI functionality, Experience Platform components, and Experience Cloud products that you will use
To successfully implement this use case, you must use multiple areas of Adobe Experience Platform. Ensure you have the necessary attribute-based access control permissions for all these areas, or ask your system administrator to grant them.
- License usage dashboard - View your current entitlement usage at the sandbox level.
- Dataset management - Monitor and manage dataset-level retention policies.
- Audiences (Real-Time Customer Profile) - Ensure segmentation rules look back window aligns with data retention windows.
- Monitoring and alerts - Track updates and receive insights on dataset retention operations.
How to achieve the use case: Step-by-step instructions
Read through the sections below, which include links to further documentation, to complete each of the steps in the high-level overview above.
Check your current license usage
First, navigate to the license usage dashboard and review your entitlement usage at the sandbox level.
Use the Metrics interface to view your license usage metrics. The interface displays information for your production sandbox by default.
Select Development to view license usage metrics related to your development sandboxes.
For more information, read the documentation on using the license usage dashboard.
Analyze dataset-level storage usage
Use the Dataset browse view to review your dataset usage metrics for both data lake and Real-Time Customer Profile. Select the column headers for either Data Lake Storage or Profile Storage, then select Sort Descending from the pop up panel.
Your datasets in data lake are sorted by storage size. Use this feature to identify the largest consumers of storage in data lake.
Your datasets in Profile are sorted by storage size. Use this feature to identify the largest consumers of storage in Profile.
Evaluate and configure retention rule
Next, determine if your datasets have the appropriate retention policies based on license limits and business requirements for Analytics and Segmentation. To view a dataset’s retention policy, select the ellipses (...) beside your dataset, then select Set data retention policy.
The Set dataset retention interface appears. Use this interface to configure a retention policy for your dataset. You can also use it to view how much storage space your dataset is consuming in either the data lake or Profile.
You can further analyze the retention impact of your dataset using the impact forecaster. Select View ExperienceEvent data distribution to view a chart that displays your retention window and the total percentage of storage that is set to expire.
When finished, select Save
Validate retention changes
Once you have applied your retention policies, you can use the following tools to validate your changes:
- Dataset usage metrics in the dataset browse view.
- The monitoring dashboard to view and analyze impact of retention.
- The license usage dashboard to view daily snapshots, predictive trends, and sandbox-level insights.