DDoS attack or bot overloading AEM with traffic
Debug this issue by enabling logging enabled at the dispatcher level, and take steps to improve dispatcher caching, which helps avoid outages caused by traffic spikes.
Description description
Environment
Adobe Experience Manager 6.x (AEM 6.x)
AEM Dispatcher
Adobe Communique 5.x (Adobe CQ5.x)
Issue
The site is overloaded by abnormal traffic.
Cause
Potential causes could be any of these:
- Denial of Service attack (DDoS)
- Search bot or scraping bot hitting expensive URLs
- Extra traffic spike due to a popular article, press release, etc.
Resolution resolution
To debug such an issue, it is best to have proper logging enabled at the dispatcher level.
Follow these steps:
Step 1: ENABLE LOGGING OF THE X-FORWARDED-FOR HEADER:
Apache:
In the Apache HTTP Server’s access_log, add this to the httpd.conf file in the dispatcher servers:
LogFormat "%{X-Forwarded-For}i %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
IIS:
In Microsoft IIS, add X-Forwarded-For in the “Advanced Logging” configuration.
That would log the correct end-user IP.
Use the output showing the IP and User-Agent to analyze whether it is a malicious attack, and block the offending IPs if it makes sense.
Step 2: SET DISPATCHER.LOG’S LOG LEVEL TO DEBUG:
Apache:
In the httpd configuration files, there should be a section for the dispatcher. Set DispatcherLogLevel to 3:
DispatcherLogLevel 3
IIS:
Modify the dis_iis.ini and set the loglevel to 3:
loglevel=3
Step 3: REVIEW THIS DOCUMENTATION AND WEBINAR ON DISPATCHER CACHING. TAKE STEPS TO IMPROVE DISPATCHER CACHING, THAT HELPS AVOID OUTAGES CAUSED BY TRAFFIC SPIKES: