Investigate requests with missing or empty User-Agent headers in AEM as a Cloud Service
This article explains how to investigate and address a high volume of requests with missing or empty User-Agent headers in Adobe Experience Manager (AEM) as a Cloud Service by analyzing CDN logs and reviewing CDN configuration.
Description description
Environment
Adobe Experience Manager as a Cloud Service
Issue/Symptoms
- CDN logs show a high volume of requests with a missing or empty
User-Agentheader. - You need to identify the request paths, volumes, and sources to determine whether the traffic belongs to an integration, monitoring tool, health check, or unwanted automation.
Cause
A request can have a missing or empty User-Agent when the client does not send the header or an intermediary does not forward it. This alone does not establish whether the request is legitimate or unwanted. Review the relevant CDN logs and configuration to identify the source.
Resolution resolution
Follow the steps below to resolve the issue:
- Access the CDN for the affected environment and download the logs for the relevant timeframe.
- Analyze the logs by using the AEM CDN Log Analysis Tooling. Refer to the Experience League documentation for guidance on processing and analyzing CDN log data.
- Filter the CDN logs for requests in which the
User-Agentheader is missing or empty. - Identify the URLs, endpoints, and request volumes associated with the filtered requests.
- Review the log entries to determine the originating IP addresses, clients, or services generating the requests, and look for patterns that indicate health checks, synthetic monitoring, or integration-related traffic.
- Review the CDN configuration and verify that no policy is removing the
User-Agentheader or preventing it from being forwarded. - Determine whether legitimate monitoring, health-check, or integration traffic is responsible for the missing
User-Agentheaders. - Do not block all requests that lack a
User-Agentheader until legitimate traffic sources have been identified and excluded. - After confirming the unwanted traffic sources, implement rate-limiting or blocking controls at the CDN layer or by using AEM CDN traffic-filter rules.
- Verify that legitimate traffic continues to function as expected and that any implemented rate-limiting or blocking rules affect only the intended traffic.
recommendation-more-help
experience-cloud-kcs-help-kbarticles