Monitoring data ingestion

Data ingestion allows you to ingest your data to Adobe Experience Platform. You can either use batch ingestion, which allows you to insert your data using various file types (such as CSVs), or streaming ingestion, which allows you to ingest your data to Platform using streaming endpoints in real time.

This user guide provides steps on how to monitor your data within Adobe Experience Platform user interface. This guide requires you to have an Adobe ID and access to Adobe Experience Platform.

Monitor streaming end-to-end data ingestion monitor-streaming-end-to-end-data-ingestion

In the Experience Platform UI, select Monitoring on the left navigation menu, followed by Streaming end-to-end.

The Streaming end-to-end monitoring page appears. This workspace provides a graph that displays the rate of streamed events being received by Platform, a graph that displays the rate of streamed events which were successfully processed by Real-Time Customer Profile, as well as a detailed list of incoming data.

By default, the top graph shows the rate of ingestion over the past seven days. This date range can be adjusted to show various periods of time by selecting the highlighted button.

The bottom graph shows the rate of successfully processed streamed events by Profile over the past seven days. This date range can be adjusted to show various periods of time by selecting the highlighted button.

Underneath the graphs is a list of all the streaming ingestion records that correspond with the date range displayed above. Each listed batch displays its ID, dataset name, when it was last updated, the number of records in the batch, as well as the number of errors (if any exist). You can select any of the records for more detailed information about that record.

Viewing streaming records

When viewing the details of a successfully streamed record, information such as the number of records ingested, file size, and ingestion start and end times are shown.

The details of a failed streaming record displays the same information as a successful record.

In addition, failed records provide details on the errors that occurred while processing the batch. In the example below, there was a parsing error that occurred when converting or validating the data.

Monitor batch end-to-end data ingestion

In the Experience Platform UI, select Monitoring on the left navigation menu.

The Batch end-to-end monitoring page appears, displaying a list of the previously ingested batches. You can select any of the batches for more detailed information about that record.

Viewing batches

When viewing the details of a successful batch, information such as the number of records ingested, file size, and ingestion start and end times are shown.

The details of a failed batch displays the same information as a successful batch, with the addition of the number of records failed.

In addition, failed batches provide details on the errors that occurred while processing the batch. In the example below, there was an error with the ingested batch because it has the maximum number of identities for the person.