Destinations allow you to activate your data from Adobe Experience Platform to countless external partners. Platform makes the process of tracking the flow of data to your destinations easier by providing transparency with dataflows.
The monitoring dashboard provides you with a visual representation of the journey of a dataflow, including the destination the data is activated to. This tutorial provides instructions on how you can either monitor dataflows directly in the destinations workspace or use the monitoring dashboard to monitor dataflows for your destinations using the Experience Platform user interface.
This guide requires a working understanding of the following components of Adobe Experience Platform:
In the Destinations workspace within the Platform UI, navigate to the Browse tab and select the name of a destination that you want to view.
A list of existing dataflows appears. On this page is a list of viewable dataflows, including information about their destination, username, number of dataflows, and status.
See the following table for more information on statuses:
For streaming destinations, the Dataflow runs tab provides an hourly update for metric data on your dataflow runs. The most prominent statistics labelled are for identities.
Identities represent the different facets of a profile. For example, if a profile contains both a phone number and an email address, that profile will have two identities.
A list of individual runs and their particular metrics is displayed, along with the following totals for identities:
Each individual dataflow run shows the following details:
To view the details of a particular dataflow run, select the run’s start time from the list.
The details page for a dataflow run contains additional information such as the number of profiles received, the number of identities activated, the number of identities failed, and the number of identities excluded.
The details page also displays a list of identities that failed and identities that were excluded. Information for both the failed and excluded identities is displayed, including the error code, identity count, and description. By default, the list displays the failed identities. To show skipped identities, select the Identities excluded toggle.
For batch destinations, the Dataflow runs tab provides metric data on your dataflow runs. A list of individual runs and their particular metrics is displayed, along with the following totals for identities:
Each individual dataflow run shows the following details:
To view details of a specific dataflow run, select the run’s start time from the list.
Dataflow runs are generated based on the destination dataflow’s schedule frequency. A separate dataflow run is made for each merge policy applied to a segment.
The details page for a dataflow, in addition to the details shown on the dataflows list, displays more specific information about the dataflow:
The details page also displays a list of identities that failed and identities that were excluded. Information for both the failed and excluded identities is displayed, including the error code and description. By default, the list displays the failed identities. To show excluded identities, select the Identities excluded toggle.
To access the Monitoring dashboard, select Monitoring () in the left navigation. Once on the Monitoring page, select Destinations. The Monitoring dashboard contains metrics and information on the destination run jobs.
Use the Destinations dashboard to get an overall idea of the health of your activation flows. Start by getting insights on an aggregated level for all batch and streaming destinations and then drill down into detailed views for dataflows, dataflow runs, and activated segments for an in-depth look at your activation data. The screens in the Monitoring dashboard provide actionable insights through metrics and error descriptions to help you troubleshoot any problems that might arise in your activation scenarios.
At the center of the dashboard is the Activation panel, which contains metrics and graphs that display data on the activation rate of the data which is exported to streaming destinations, as well as on the failed batch dataflow runs to batch destinations.
By default, the data displayed contains the activation information from the last 24 hours. Select Last 24 hours to adjust the time frame of records displayed. Available options include Last 24 hours, Last 7 days, and Last 30 days. Alternatively, you can select the dates on the calendar pop-up window that appears. Once you have selected dates, select Apply to adjust the time frame of the information shown.
The following screenshot shows the activation rate and batch dataflow runs for the last 30 days instead of the last 24 hours. You can adjust the time frame by selecting Last 30 days.
Use the arrow icon () to expand or dismiss the cards at the top of the screen, which show at-a-glance information about the activation details, based on the destination type - streaming or batch:
The Activation graph is displayed by default and you can disable it to expand the list of destinations below. Select the Metrics and graphs toggle to disable the graphs.
The Activation panel displays a list of destinations that contain at least one existing account. This list also includes information on the profiles received, identities activated, identities failed, identities excluded, activation rate, total failed dataflows, and the last updated date for these destinations. Not all metrics are available for all destination types. The table below outlines which metrics are available per destination type, streaming or batch.
|Profiles received||Streaming and batch|
|Identities activated||Streaming and batch|
|Identities excluded||Streaming and batch|
|Total failed dataflows||Batch|
|Last updated||Streaming and batch|
You can also filter your list of destinations to only display the selected category of destinations. Select the My destinations dropdown, and select the destination category that you want to filter to.
Additionally, you can enter a destination into the search bar to isolate to a single destination. If you want to see the destination’s dataflows you can select the filter beside it to see a list of its active dataflows.
If you want to view all existing dataflows across all destinations, select Dataflows.
A list of dataflows appears, sorted by the last dataflow run. You can see additional details for a specific dataflow by locating the destination you want to monitor, selecting the filter beside it, and then subsequently selecting the filter beside the dataflow you want more information about.
Once you select a dataflow for further inspection, the dataflow details page contains a toggle which allows you to see the activated data in the dataflow, broken down by dataflow runs or segments.
When Dataflow runs is selected, you can see a list of dataflow runs for the selected dataflow and further information about each run.
For dataflows to streaming destinations, a dataflow run is broken down into hourly windows. Each hourly window generates a corresponding dataflow run ID.
For dataflows to batch destinations, each segment has a corresponding dataflow run generated, based on the segment activation scheduled frequency. For example, if you set up a daily scheduled activation for five segments in the same destination dataflow, there will be five separate dataflow runs generated every day.
Use the Show failures only toggle to display only the failed runs for a dataflow.
When Segments is selected, you see a list of the segments which were activated to the selected dataflow, within the selected time range. This screen includes segment-level information about the identities activated, identities excluded, as well as the status and the time of the last dataflow run. By reviewing the metrics for identities excluded and activated, you can verify if a segment has been successfully activated or not.
For example, you are activating a segment called “Loyalty Members in California” to an Amazon S3 destination “Loyalty Members California December”. Let’s assume that there are 100 profiles in the selected segment but only 80 out of 100 profiles contain Loyalty ID attributes and you have defined the export mapping rules as
loyalty.id is required. In this case, on a segment level, you will see 80 identities activated, and 20 identities excluded.
Note the current limitations related to segment-level metrics:
In the segment-level view, the metrics are aggregated across multiple dataflow runs within the selected time range. If there are multiple dataflow runs, you can drill down from the segment level to see the breakdown for each dataflow run, filtered by the selected segment.
Use the filter button to drill down into the dataflow runs view for each segment in the dataflow.
The dataflow runs page displays information on your dataflow runs, including the dataflow run start time, processing time, profiles received, identities activated, identities excluded, identities failed, activation rate, and status.
When you drill down into the dataflow runs page from the segment-level view, you have the option of filtering the dataflow runs by the following options:
To see more details about a specific dataflow run, select the filter beside the dataflow run start time to see the dataflow run details page.
The dataflow run details page, in addition to the details shown on the dataflow runs list, displays more specific information about the dataflow:
The details page also has a toggle to switch between dataflow run errors and segments. This option is only available for dataflow runs in batch destinations.
The dataflow run errors view displays a list of identities that failed and identities that were excluded. Information for both the failed and excluded identities is displayed, including the error code, identity count, and description. By default, the list displays the failed identities. To show skipped identities, select the Identities excluded toggle.
When Segments is selected, you see a list of the segments which were activated in the selected dataflow run. This screen includes segment-level information about the identities activated, identities excluded, as well as the status and the time of the last dataflow run.
By following this guide, you now know how to monitor dataflows for both batch and streaming destinations, including all the relevant information such as processing time, activation rate, and status. To learn more about dataflows in Platform, please read the dataflows overview. To learn more about destinations, please read the destinations overview.