AEM as a Cloud Service: How to isolate 503 errors?
If all pages return a 503 error check for an ongoing incident. If one page returns a 503 error check the cloud logs.
Description description
Environments
Adobe Experience Manager as a Cloud Service
Issue/Symptoms
This article explains how to isolate where the problem lies when you encounter 503 errors on AEM as a Cloud Service.
Resolution resolution
The content delivery flow in AEM as a Cloud Service is as follows:
Browser » CDN » Dispatcher » Publish
In case there is a service-wide incident, all pages will return the 503 error. When there is a problem between CDN - Dispatcher or Dispatcher - Publish for the requests with a particular condition, only particular pages would return the 503 error.
Case A - All pages return the error
When all pages return the error, there might be a service-wide incident. Check if there is an ongoing incident or a scheduled maintenance at Adobe System Status » Experience Cloud » Adobe Experience Manager as a Cloud Service.
Adobe System Status
https://status.adobe.com/
Case B - Only particular pages return the error
When the error occurs only on particular pages, the pages may have an inherent problem that prevents normal response at some point in the content delivery flow. In this case, try accessing the page and see the 503 error once again. Then isolate the problem using the logs downloaded from Cloud Manager.
Dispatcher’s httpdaccess and Publish’s aemrequest are especially important. Checking if each log contains the corresponding access record helps to isolate where the problem lies.
Here is a log sample for comparison, where both Dispatcher and Publish returned normal responses for access to /us/en.html.
Dispatcher’s httpdaccess:logged on responding
cm-p12345-e67890-aem-publish-55cf6bcc5-vxfcf - - 18/Oct/2022:10:20:11 +0000 "GET /us/en.html HTTP/1.1" 200 16263 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
Publish’s aemrequest: logged on receiving and responding
18/Oct/2022:10:20:11 +0000 [1063] -> GET /content/wknd/us/en.html HTTP/1.1 cm-p12345-e67890-aem-publish-55cf6bcc5-vxfcf
18/Oct/2022:10:20:14 +0000 [1063] <- 200 text/html 3355ms cm-p12345-e67890-aem-publish-55cf6bcc5-vxfcf
Refer to the manual for logging details.
https://experienceleague.adobe.com/docs/experience-manager-cloud-service/content/implementing/developing/logging.html?lang=en
The following four cases can be isolated from the combination of the presence/absence of access records.
Case B-1 - Both Dispatcher and Publish have the access record
The CDN might have timed out due to a long response time on Publish. Check if there is a response record in Publish’s aemrequest and how long the response time was. If the response time was long such as over several minutes, look for related messages in Publish’s aemerror.
Case B-2 - Dispatcher has the access record, but Publish does not
Either Dispatcher responded alone, or the request arrived at Publish, but something wrong might happen before logging the record. Check Dispatcher’s httpderror, aemdispatcher, and Publish’s aemerror for related messages.
Case B-3 - Dispatcher does not have the access record, but Publish does
Publish has accepted the request but has not yet returned a response. Check if there is a response record in Publish’s aemrequest. If there is no response record, look for related messages in Publish’s aemerror.
Case B-4 - Neither Dispatcher nor Publish has the access record
Dispatcher was unable to accept the requests due to some problem. Check Dispatcher’s httpderror and Dispatcher’s aemdispatcher for related messages.
In addition, the followings are also helpful when particular pages return the error.
- Try access with another browser or from another network
- Compare the component types and amount in the pages with the pages that returns a normal response
- Check if the error reproduces with the local SDK by creating a package of the pages