Missing hits in Livestream

While comparing the amount of hits received by Live Stream API with hits in Data feeds, clients might come across certain discrepancies (with livestream API missing a few hits). Why does that happen?

Description description

Environment

Adobe Analytics (AA)

Issue

Check the following points for missing hits in Livestream output:

  1. If multiple users are connected to the livestream endpoint, ensure that the client is using outputs from all the connections to compare livestream data with data feeds
  2. Disconnecting and reconnecting from the livestream endpoint can result in data loss, as data in transit can be dropped when the client severs its connection from the API servers.
  3. Live Stream data isn’t the same as that available via Data Feeds, as it is sent mid-way through processing and therefore won’t have values based on profile/attribution (Live Stream API contains mid values). Specifically, if a value isn’t provided on the hit itself (For example: not part of the “pre-value” data), it won’t be included in Live Stream, but it could be in Data Feeds.
  4. Live Stream is considered part of the “speed layer” in a Lambda Architecture ( https://databricks.com/glossary/lambda-architecture ) and as such, has different fault-tolerances than our full reporting pipeline (For example: we prioritize speed over precise accuracy).
  5. Live Stream isn’t meant to be a replacement for our permanent data warehouse repository (which is the source of record), and a very small number of hits might not make it to the Live Stream feed.

Resolution resolution

A slight discrepancy between Live stream data and data feeds is expected due to the reasons stated above.

recommendation-more-help
3d58f420-19b5-47a0-a122-5c9dab55ec7f