Capping & throttling APIs
About Capping & Throttling APIs
When configuring a datasource or an action, you establish a connection to a system to either retrieve additional information to use in your journeys or send messages or API calls.
Journeys APIs support up to 5,000 event per second but some external systems or API may not have an equivalent throughput. To prevent overloading these systems, you can use the Capping and Throttling APIs to limit the number of events sent per second.
Every time an API call is performed by journeys, it passes through the API engine. If the limit set in the API is reached, the call is either rejected if you are using the Capping API, or queued for up to 6 hours and processed as soon as possible in the order they were received if you are using the Throttling API.
For example, let’s say that you have defined a capping or throttling rule of 200 calls per second for your external system. Your system is called by a custom action in 10 different journeys. If one journey receives 300 calls per second, it will use the 200 slots available and discard or queue the 100 remaining slots. Since the maximum rate has exceeded, the other 9 journeys will not have any slot left. This granularity helps to protect the external system from over-loading and crashing.
For more information on how to work with the APIs, refer to these sections:
A detailed description of the APIs is available in Adobe Journey Optimizer APIs documentation
Data sources & custom actions capacity
For external data sources, the maximum number of calls per second is limited to 15. If this limit is exceeded, any additional calls are either discarded or queued depending on the API in use. It is possible to increase this limit for private external data sources by contacting Adobe to include the endpoint in the allowlist, but this is not an option for public external data sources. * Learn how to configure data sources.
For custom actions, you need to evaluate the capacity of your external API. For example, if Journey Optimizer sends 1000 calls per second and your system can only support 200 calls per second, you need to define a capping or throtlling configuration so that your system does not saturate. Learn how to configure actions
Timeout and retries
If the capping or throttling rule is fulfilled, then the timeout rule is applied.
In each journey, you can define a timeout duration. This allows you to set a maximum duration when calling an external system. Timeout duration is configured in the properties of a journey. Refer to this page.
This timeout is global to all external calls (external API calls in custom actions and custom data sources). By default, it is set to 30 seconds.
During the defined timeout duration, Journey Optimizer tries to call the external system. After the first call, a maximum of three retries can be performed until the end of timeout duration is reached. The number of retries cannot be changed.
Each retry uses one slot. If you have a capping of 100 calls per second and each of your calls require two retries, the rate drops to 30 calls per second (each call uses 3 slots: the first call and two retries).
The timeout duration value depends on the use case. If you want to send your message quickly, for example when the client enters the store, then you do not want to set up a long timeout. Also, the longer the timeout is, the more items will be placed in queue. This can greatly impact performance. If Journey Optimizer performs 1000 calls per seconds, keeping 5 or 15 seconds of data can quickly overwhelm the system.
Let’s take an example for a timeout of 5 seconds.
-
The first call lasts less than 5 seconds: the call is successful, no retry is performed.
-
The first call lasts longer 5 seconds: the call is cancelled and there is no retry. It is counted as a timeout error in reporting.
-
The first call fails after 2 seconds (the external system returns an error): 3 seconds are left for retries, if capping slots are available.
- If one of the three retries is successful before the end of the 5 seconds, the call is performed, and there is no error.
- If the end of the timeout duration is reached during the retries, the call is cancelled and counted as a timeout error in reporting.