Resolve streaming dataflow UI discrepancies in Adobe Experience Platform

Last update: January 16, 2025

In Adobe Experience Platform (AEP), you may encounter discrepancies in the dataflow user interface, such as inaccurate processing durations and incorrect ingestion statuses. This article provides steps to verify data ingestion and understand processing times, ensuring accurate monitoring despite UI issues.

Description

Environment

  • Adobe Experience Platform (AEP)

Issue/Symptoms

In certain scenarios within the Adobe Experience Platform, you may observe discrepancies in the dataflow user interface. Specifically:

  • The UI does not accurately reflect the processing duration and shows an average of 1h 30m.
  • Ingestion percentages and status are not displayed correctly in the Monitoring section.

These inconsistencies can lead to confusion regarding whether data ingestion is functioning as expected. Despite these UI issues, using Query Service confirms that data is being ingested into both profile and event datasets after a delay of approximately 15-20 minutes.

Resolution

To address this issue:

  1. Understand processing times to help clarify discrepancies observed in the Adobe Experience Platform’s dataflow UI. The average processing duration for HTTP API streaming inlet via DIM (Data In Motion) service is approximately 1 hour and 30 minutes. It is important to note this timeframe pertains to metrics and status updates but does not directly correlate with actual data ingestion times.
  2. Confirm that despite UI discrepancies, the backend processes are ingesting data correctly by checking through Query Service or Flow Service APIs.
  3. Monitor specific dates for latency issues. There might be latency issues affecting how quickly metrics update on the UI even though ingestion times remain unaffected. These can be caused by maintenance windows or service interruptions, which can be tracked through our Adobe Status page at Adobe Status: Experience platform.
  4. Review batch details including start and end times to ensure that ingestion processing time remains consistent (e.g., for example, around 14 to 16 minutes).
  5. If clarification or assistance is required beyond verifying backend processes against UI displays, consider submitting a support ticket related to system latencies or other anomalies observed.

By following these steps, you can better understand discrepancies between what appears on the AEP’s user interface versus actual backend operations concerning dataflows across various environments.