Guardrails for activation data

Last update: 2023-11-01
  • Created for:
  • Developer
    User
    Admin
    Leader

This page provides default usage and rate limits with regard to activation behavior. When reviewing the following guardrails, it is assumed that you have correctly connected to destinations.

NOTE
  • Most customers do not exceed these default limits. If you would like to learn about custom limits, please contact your customer care representative.
  • The limits outlined in this document are constantly being improved. Please check back regularly for updates.
  • Depending on individual downstream limitations, some destinations might have tighter guardrails than the ones documented on this page. Make sure to also check the catalog page of the destination you are connecting and activating data to.

Guardrail types

There are two types of default limits within this document:

Guardrail type Description
Performance guardrail (Soft limit) Performance guardrails are usage limits that relate to the scoping of your use cases. When exceeding performance guardrails, you may experience performance degradation and latency. Adobe is not responsible for such performance degradation. Customers who consistently exceed a performance guardrail may elect to license additional capacity to avoid performance degradation.
System-enforced guardrails (Hard limit) System-enforced guardrails are enforced by the Real-Time CDP UI or API. These are limits that you cannot exceed as the UI and API will block you from doing so or will return an error.

Activation limits

The following guardrails provide recommended limits when activating Real-Time Customer Profile data to destinations.

General activation guardrails

The guardrails below generally apply to activation through all destination types.

Guardrail Limit Limit Type Description
Maximum number of audiences to a single destination 250 Performance guardrail The recommendation is to map a maximum of 250 audiences to a single destination in a dataflow.

If you need to activate more than 250 audiences to a destination, you can either:
  • Unmap audiences that you don’t want to activate anymore, or
  • Create a new dataflow to the desired destination and map audiences to this new dataflow.

Note that in the case of some destinations, you may be limited to fewer than 250 audiences mapped to the destination. Those destinations are called out further below on the page, in their respective sections.
Maximum number of attributes mapped to a destination 50 Performance guardrail In the case of several destinations and destination types, you can select profile attributes and identities to map for export. For optimal performance, a maximum of 50 attributes should be mapped in a dataflow to a destination.
Maximum number of destinations 100 System-enforced guardrail You can create a maximum of 100 destinations that you can connect and activate data to, per sandbox. Edge personalization destinations (Custom personalization) can make up a maximum of 10 of the 100 recommended destinations.
Type of data activated to destinations Profile data, including identities and identity map System-enforced guardrail Currently, it is only possible to export profile record attributes to destinations. XDM attributes that describe event data are not supported for export at this time.
Type of data activated to destinations - array and map attributes support Not available System-enforced guardrail At this time, it is not possible to export array or map attributes to destinations. The exception to this rule is the identity map, which gets exported in both streaming and file-based activations.

Streaming activation

The guardrails below apply to activation through streaming destinations.

Guardrail Limit Limit Type Description
Number of activations (HTTP messages with profile exports) per second N/A - There is currently no limit to the number of messages per second sent from Experience Platform to partner destinations’ API endpoints.
Any limits or latencies are dictated by the endpoint where Experience Platform is sending data. Make sure to also check the catalog page of the destination you are connecting and activating data to.

Batch (file-based) activation

The guardrails below apply to activation through batch (file-based) destinations.

Guardrail Limit Limit Type Description
Activation frequency One daily full export or more frequent incremental exports every 3, 6, 8, or 12 hours. System-enforced guardrail Read the export full files and export incremental files documentation sections for more information about the frequency increments for batch exports.
Maximum number of audiences that can pe exported at a given hour 100 Performance guardrail The recommendation is to add a maximum of 100 audiences to batch destination dataflows.
Maximum number of rows (records) per file to activate 5 million System-enforced guardrail Adobe Experience Platform automatically splits the exported files at 5 million records (rows) per file. Each row represents one profile. Split file names are appended with a number that indicates the file is part of a larger export, as such: filename.csv, filename_2.csv, filename_3.csv. For more information, read the scheduling section of the activate batch destinations tutorial.

Ad-hoc activation

The guardrails below apply to the ad-hoc activation method.

Guardrail Limit Limit Type Description
Audiences activated per ad-hoc activation job 80 System-enforced guardrail Currently, each ad-hoc activation job can activate up to 80 audiences. Attempting to activate more than 80 audiences per job will cause the job to fail. This behavior is subject to change in future releases.
Concurrent ad-hoc activation jobs per audience 1 System-enforced guardrail Do not run more than one concurrent ad-hoc activation job per audience.

Edge personalization destinations activation

The guardrails below apply to activation through edge personalization destinations.

Guardrail Limit Limit Type Description
Maximum number of Custom personalization destinations 10 Performance guardrail You can set up dataflows to 10 Custom personalization destinations per sandbox.
Maximum number of attributes mapped to a personalization destination per sandbox 30 System-enforced guardrail A maximum of 30 attributes can be mapped in a dataflow to a personalization destination, per sandbox.
Maximum number of audiences mapped to a single Adobe Target destination 50 Performance guardrail You can activate a maximum of 50 audiences in an activation flow to a single Adobe Target destination.

Dataset exports

Dataset exports are currently supported in a First Full and then Incremental pattern. The guardrails described in this section apply to the first full export that occurs after a dataset export workflow is set up.

Dataset Types

The dataset export guardrails apply to two types of datasets exported from Experience Platform, as described below:

Datasets based on the XDM Experience Events schema
In the case of datasets based on the XDM Experience Events schema, the dataset schema includes a top level timestamp column. Data is ingested in an append-only fashion.

Datasets based on the XDM Individual Profile schema
In the case of datasets based on the XDM Individual Profile schema, the dataset schema does not include a top level timestamp column. Data is ingested in an upsert fashion.

The soft guardrail below applies to all datasets exported out of Experience Platform. Review also the hard guardrails further below, specific to different dataset and compression types.

Guardrail Limit Limit Type Description
Size of exported datasets 5 billion records Performance guardrail The limit described here for dataset exports is a soft guardrail. For example, while the user interface will not block you from exporting datasets larger than 5 billion records, the behavior is unpredictable and exports might either fail or have very long export latency.

Guardrails for scheduled dataset exports

For scheduled, or recurring dataset exports, the guardrails below are identical for the two formats of the exported file (JSON or parquet), and are grouped by dataset type.

WARNING

Exports to JSON files are supported in a compressed mode only.

Dataset type Guardrail Guardrail type Description
Datasets based on the XDM Experience Events schema Last 365 days of data System-enforced guardrail The data from the last calendar year is exported.
Datasets based on the XDM Individual Profile schema Ten billion records across all exported files in a dataflow System-enforced guardrail The record count of the dataset must be less than ten billion for compressed JSON or parquet files and one million for uncompressed parquet files, otherwise the export fails. Reduce the size of the dataset that you are trying to export if it is larger than the allowed threshold.

Read more about exporting datasets.

Destination SDK guardrails

Destination SDK is a suite of configuration APIs that allow you to configure destination integration patterns for Experience Platform to deliver audience and profile data to your endpoint, based on data and authentication formats of your choice. The guardrails below apply to the destinations you configure using Destination SDK.

Guardrail Limit Limit Type Description
Maximum number of private custom destinations 5 Performance guardrail You can create a maximum of 5 private custom streaming or batch destinations using Destination SDK. Reach out to a custom care representative if you need to create more than 5 such destinations.
Profile export policy for Destination SDK
  • maxBatchAgeInSecs (minimum 1.800 and maximum 3.600)
  • maxNumEventsInBatch (minimum 1.000, maximum 10.000)
 System-enforced guardrail When using the configurable aggregation option for your destination, be mindful of the minimum and maximum values that determine how often HTTP messages are sent to your API-based destination and how many profiles the messages should include.

Destination throttling and retry policy

Details on throttling thresholds or limitations for given destinations. This section also provides information regarding the retry policy for destinations.

Type of destination Description
Enterprise destinations (HTTP API, Amazon Kinesis, Azure EventHubs) In 95 percent of the time, Experience Platform attempts to offer a throughput latency of less than 10 minutes for successfully sent messages with a rate of less than 10 thousand requests per second for each dataflow to an enterprise destination.
In case of failed requests to your enterprise destination, Experience Platform stores the failed requests and retries twice to send the requests to your endpoint.

Next steps

See the following documentation for more information on other Experience Platform services guardrails, on end-to-end latency information, and licensing information from Real-Time CDP Product Description documents:

Previous page
Next page

On this page