This tutorial covers the steps for monitoring dataflows for streaming sources using the Sources workspace.
This tutorial requires a working understanding of the following components of Adobe Experience Platform:
In the Platform UI, select Sources from the left navigation bar to access the Sources workspace. The Catalog screen displays a variety of sources for which you can create an account with.
To view existing dataflows for streaming sources, select Dataflows from the top header.
The Dataflows page contains a list of all existing dataflows in your organization, including information about their source data, account name, and dataflow run status.
Select the name of the dataflow you want to view.
The following table contains more information on dataflow run statuses:
The Dataflow Activity page displays specific information on your streaming dataflow. The top banner contains the cumulative number of records ingested and records failed for all of your streaming dataflow runs in your selected date range.
The lower half of the page displays information on the number of records received, ingested, and failed, per flow run. Each flow run is recorded within an hourly window.
Each individual dataflow run shows the following details:
By default, the data displayed contains ingestion rates from the last seven days. Select Last 7 days to adjust the time frame of records displayed.
A calendar pop-up window appears, providing you options for alternative ingestion time frames. Select Last 30 days and then select Apply.
To view the details of a specific dataflow run, including its errors, select the run’s start time from the list.
The Dataflow run overview page contains additional information on your dataflow, such as its corresponding dataflow run ID, target dataset, and IMS organization ID.
A flow run with errors also contains the Dataflow run errors panel, which displays the particular error that led to the failure of the run, as well as the total count of records that failed.
By following this tutorial, you have successfully used the Sources workspace to monitor your streaming dataflows and identify the errors that led to any failed dataflows. See the following documents for more information: