Guardrails for Real-time Customer Profile data

Real-time Customer Profile provides individual profiles that enable you to deliver personalized cross-channel experiences based on behavioral insights and customer attributes. In order to achieve this targeting, Profile, and the segmentation engine within Adobe Experience Platform use a highly denormalized hybrid data model which offers a new approach to developing customer profiles. Use of this hybrid data model makes it important that the data being collected is modeled correctly. While the Profile data store maintaining profile data is not a relational store, Profile permits integration with small dimension entities in order to create segments in a simplified and intuitive manner. This integration is known as multi-entity segmentation.

Adobe Experience Platform provides a series of guardrails to help you avoid creating data models which Real-time Customer Profile cannot support. This document outlines these guardrails and best practices and constraints when using Profile data for segmentation.

NOTE

The guardrails and limits outlined in this document are constantly being improved. Please check back regularly for updates.

Getting started

It is recommended that you read the following Experience Platform services documentation before attempting to build data models for use in Real-time Customer Profile. Working with data models, and the guardrails outlined in this document, require an understanding of the various Experience Platform services involved with managing Real-time Customer Profile entities:

Entity types

The Profile store data model consists of two core entity types:

  • Primary entity: A primary entity, or profile entity, merges data together to form a “single source of truth” for an individual. This unified data is represented using what is known as a “union view”. A union view aggregates the fields of all schemas that implement the same class into a single union schema. The union schema for Real-time Customer Profile is a denormalized hybrid data model that acts as a container for all profile attributes and behavioral events.

    Time-independent attributes, also known as “record data” are modeled using XDM Individual Profile, while time-series data, also known as “event data” is modeled using XDM ExperienceEvent. As record and time-series data is ingested in Adobe Experience Platform, it triggers Real-time Customer Profile to begin ingesting data that has been enabled for its use. The more interactions and details that are ingested, the more robust individual profiles become.

  • Dimension entity: Your organization may also define XDM classes to describe things other than individuals, such as stores, products, or properties. These non-XDM Individual Profile schemas are known as “dimension entities” and do not contain time-series data. Dimension entities provide lookup data which aids and simplifies multi-entity segment definitions and must be small enough that the segmentation engine can load the entire data set into memory for optimal processing (fast point lookup).

Profile fragments

There are multiple guardrails in this document referring to “profile fragments.” The Real-time Customer Profile is composed of multiple profile fragments. Each fragment represents the data for the identity from a dataset where it is the primary identity. This means that a fragment may contain a primary ID and event data (time series) in an XDM ExperienceEvent dataset or it may be composed of a primary ID and record data (time-independent attributes) in an XDM Individual Profile dataset.

Limit types

When defining your data model, it is recommended to stay within the provided guardrails to ensure proper performance and avoid system errors.

The guardrails provided in this document include two limit types:

  • Soft limit: A soft limit provides a recommended maximum for optimal system performance. It is possible to go beyond a soft limit without breaking the system or receiving error messages, however going beyond a soft limit will result in performance degradation. It is recommended to stay within the soft limit to avoid decreases in overall performance.

  • Hard limit: A hard limit provides an absolute maximum for the system. Going beyond a hard limit will result in breakages and errors, preventing the system from functioning as expected.

Data model guardrails

Adhering to the following guardrails is recommended when creating a data model for use with Real-time Customer Profile.

Primary entity guardrails

Guardrail Limit Limit Type Description
Number of datasets recommended to contribute to the Profile union schema 20 Soft A maximum of 20 Profile-enabled datasets is recommended. To enable another dataset for Profile, an existing dataset should first be removed or disabled.
Number of multi-entity relationships recommended 5 Soft A maximum of 5 multi-entity relationships defined between primary entities and dimension entities is recommended. Additional relationship mappings should not be made until an existing relationship is removed or disabled.
Maximum JSON depth for ID field used in multi-entity relationship 4 Soft The recommended maximum JSON depth for an ID field used in multi-entity relationships is 4. This means that in a highly nested schema, fields that are nested more than 4 levels deep should not be used as an ID field in a relationship.
Array cardinality in a profile fragment <=500 Soft The optimal array cardinality in a profile fragment (time-independent data) is <=500.
Array cardinality in ExperienceEvent <=10 Soft The optimal array cardinality in an ExperienceEvent (time series data) is <=10.

Dimension entity guardrails

Guardrail Limit Limit Type Description
No time-series data permitted for non-XDM Individual Profile entities 0 Hard Time-series data is not permitted for non-XDM Individual Profile entities in Profile Service. If a time-series dataset is associated with a non-XDM Individual Profile ID, the dataset should not be enabled for Profile.
No nested relationships 0 Soft You should not create a relationship between two non-XDM Individual Profile schemas. The ability to create relationships is not recommended for any schemas which are not part of the Profile union schema.
Maximum JSON depth for primary ID field 4 Soft The recommended maximum JSON depth for the primary ID field is 4. This means that in a highly nested schema, you should not select a field as a primary ID if it is nested more than 4 levels deep. A field that is on the 4th nested level can be used as a primary ID.

Data size guardrails

The following guardrails refer to data size and are recommended to ensure that data can be ingested, stored, and queried as intended.

NOTE

Data size is measured as uncompressed data in JSON at time of ingestion.

Primary entity guardrails

Guardrail Limit Limit Type Description
Maximum ExperienceEvent size 10KB Hard The maximum size of an event is 10KB. Ingestion will continue, however any events larger than 10KB will be dropped.
Maximum profile record size 100KB Hard The maximum size of a profile record is 100KB. Ingestion will continue, however profile records larger than 100KB will be dropped.
Maximum profile fragment size 50MB Hard The maximum size of a profile fragment is 50MB. Segmentation, exports, and lookups may fail for any profile fragment that is larger than 50MB.
Maximum profile storage size 50MB Soft The maximum size of a stored profile is 50MB. Adding new profile fragments into a profile that is larger than 50MB will affect system performance.
Number of Profile or ExperienceEvent batches ingested per day 90 Soft The maximum number of Profile or ExperienceEvent batches ingested per day is 90. This means that the combined total of Profile and ExperienceEvent batches ingested each day cannot exceed 90. Ingesting additional batches will affect system performance.

Dimension entity guardrails

Guardrail Limit Limit Type Description
Maximum total size for all dimensional entities 5GB Soft The maximum recommended total size for all dimensional entities is 5GB. Ingesting large dimension entities will result in degraded system performance. For example, attempting to load a 10GB product catalog as a dimension entity is not recommended.
Datasets per dimensional entity schema 5 Soft A maximum of 5 datasets associated with each dimensional entity schema is recommended. For example, if you create a schema for “products” and add five contributing datasets, you should not create a sixth dataset tied to the products schema.
Number of dimension entity batches ingested per day 4 per entity Soft The maximum number of dimension entity batches ingested per day is 4 per entity. For example, you could ingest updates to a product catalog up to 4 times per day. Ingesting additional dimension entity batches for the same entity will affect system performance.

Segmentation guardrails

The guardrails outlined in this section refer to the number and nature of segments an organization can create within Experience Platform, as well as mapping and activating segments to destinations.

Guardrail Limit Limit Type Description
Maximum number of segments per sandbox 10K Soft The maximum number of segments an organization can create is 10K per sandbox. An organization can have more than 10K segments in total, as long as there are less than 10,000 segments in each individual sandbox. Attempting to create additional segments will result in degraded system performance.
Maximum number of streaming segments per sandbox 500 Soft The maximum number of streaming segments an organization can create is 500 per sandbox. An organization can have more than 500 streaming segments in total, as long as there are less than 500 streaming segments in each individual sandbox. Attempting to create additional streaming segments will result in degraded system performance.
Maximum number of batch segments per sandbox 10K Soft The maximum number of batch segments an organization can create is 10K per sandbox. An organization can have more than 10K batch segments in total, as long as there are less than 10,000 batch segments in each individual sandbox. Attempting to create additional batch segments will result in degraded system performance.

On this page