Missing error logs when forwarding AEM logs to an HTTP endpoint via Nginx
Forwarding Adobe Experience Manager (AEM) logs to a custom HTTP endpoint using Nginx sometimes results in missing error exception logs at the destination, even though those logs exist in AEM Cloud Manager. This happens when the HTTP endpoint is temporarily unavailable or responds slowly, causing log batches to be dropped.
Description description
Environment
- Product: Adobe Experience Manager as a Cloud Service (AEMaaCS) – Sites
- Destination: Custom Nginx server (HTTP endpoint)
Issue/Symptoms
- Some error exception logs in AEM Cloud Manager aren’t found in the log files collected from the custom Nginx server.
- Observable gaps exist in the log archive on the destination side.
Resolution resolution
Note: Fluent Bit’s LogForwarding output plugin attempts up to five retries per batch if it can’t deliver logs to the configured endpoint. If all retries fail because the endpoint remains unavailable or responds slowly, those log batches are permanently dropped and won’t be delivered later.
To resolve this issue, follow these steps:
- Review the availability and responsiveness of your Nginx HTTP endpoint used for receiving forwarded logs.
- Ensure there are no periods of high latency, outages, network issues, or resource bottlenecks affecting your Nginx server.
- Check your Nginx error and access logs for any signs of downtime or resource contention during periods when expected logs are missing.
- Refer to Log Forwarding documentation for guidance on configuring log forwarding with supported endpoints.
- After stabilizing your Nginx server and network environment, verify that new batches of forwarded logs arrive consistently at your destination.
recommendation-more-help
3d58f420-19b5-47a0-a122-5c9dab55ec7f