Missing hits in Livestream
While comparing the amount of hits received by Live Stream API with hits in Data feeds, clients might come across certain discrepancies (with livestream API missing a few hits). Why does that happen?
Description description
Environment
Adobe Analytics (AA)
Issue
Check the following points for missing hits in Livestream output:
- If multiple users are connected to the livestream endpoint, ensure that the client is using outputs from all the connections to compare livestream data with data feeds
- Disconnecting and reconnecting from the livestream endpoint can result in data loss, as data in transit can be dropped when the client severs its connection from the API servers.
- Live Stream data isn’t the same as that available via Data Feeds, as it is sent mid-way through processing and therefore won’t have values based on profile/attribution (Live Stream API contains mid values). Specifically, if a value isn’t provided on the hit itself (For example: not part of the “pre-value” data), it won’t be included in Live Stream, but it could be in Data Feeds.
- Live Stream is considered part of the “speed layer” in a Lambda Architecture ( https://databricks.com/glossary/lambda-architecture ) and as such, has different fault-tolerances than our full reporting pipeline (For example: we prioritize speed over precise accuracy).
- Live Stream isn’t meant to be a replacement for our permanent data warehouse repository (which is the source of record), and a very small number of hits might not make it to the Live Stream feed.
Resolution resolution
A slight discrepancy between Live stream data and data feeds is expected due to the reasons stated above.
recommendation-more-help
3d58f420-19b5-47a0-a122-5c9dab55ec7f