Performance Tree

Scope

The below diagram is meant to provide guidance on the steps that need to be taken in order to troubleshoot performance issues. It is split in 5 sections for easier reading.

Each step in the diagram is linked to a documentation resource or a recommendation.

Prerequisites and Assumptions

The assumption is that a performance issue is observed on a given page (either an AEM console or a web page) and can be reproduced consistently. Having a way to test or monitor the performance is a pre-requisite before starting the investigation.

The analysis starts at step 0. The goal is to determine which entity (dispatcher, external host or AEM) is responsible for the performance issue then determine which area (server or network) should be investigated.

Section 1

chlimage_1-103

Section 2

chlimage_1-104

Section 3

chlimage_1-105

Section 4

chlimage_1-106

Section 5

chlimage_1-107

Step Title Resources
Step 0 Analyze Request Flow

You can use standard HTTP request analysis in the browser to analyze the request flow. For more info on how to do this on Chrome, see:

https://developers.google.com/web/tools/chrome-devtools/profile/network-performance/resource-loading
https://developers.google.com/web/tools/chrome-devtools/profile/network-performance/understanding-resource-timing
Step 2 Are requests coming from external hosts? You can use standard HTTP request analysis in the browser to analyze the request flow. See the above links on how to do this on Chrome.
Step 3 Can the requests be cached? For more information on cacheable requests and general Dispatcher performance optimization advice, see Dispatcher Performance Optimization.
Step 4 Are requests coming from the Dispatcher?

Check the Dispatcher debugging documentation to see if the requests are cached properly.
Step 5 Is the Dispatcher trying to authenticate each request via AEM? Check if the dispatcher sends HEAD requests to AEM for authentication before delivering the cached resource. You can do this by looking for HEAD requests in the AEM access.log. For more information, see Logging.
Step 6 Is the geographic location of the Dispatcher far away from the users? Move the Dispatcher closer to the users.
Step 7 Is the network layer of the Dispatcher OK?
Investigate the network layer for saturation and latency issues.
Step 8 Is the slowness reproducible with a local instance?

Use Tough Day to replicate "real world" conditions from the production instances. If this is not realistic for the slace of your development, make sure to test the production instance (or an identical staging one) in a different network context.
Step 9 Is the geographical location of the server far away from the users? Move the server closer to the users.
Steps 10 and 29 Investigate network layer

Investigate the network layer for saturation and latency issues.

For the author tier, it is recommended that the latency does not surpass 100 milliseconds.

For more information on performance optimization tips, see this page.
Step 11 Move server closer or add one per region
Step 12 Troubleshoot AEM server Check the following sub steps in the diagram for more information.
Step 13 Check hardware requirements Check the documentation on Hardware Sizing Guidelines.
Step 14 Check for frequent causes of performance issues
Step 15 Find slow requests

You can check for slow requests by analysing the request.log or by using rlog.jar.

For more information about using rlog.jar, see this page.

See Using rlog.jar to find requests with long duration times.
Step 16 Profile server

For information about profiling tools you can use with AEM, see Tools for Monitoring and Analyzing Performance.
Step 17 Find slow methods in profiling
Step 18 Common scenarios of profiling See Analyzing Specific Scenarios in the Performance Optimization section.
Step 19 100% CPU https://helpx.adobe.com/experience-manager/6-3/sites-deploying/monitoring-and-maintaining.html#MonitoringPerformance
Step 20 Out of memory
  1. Out Of Memory
  2. My application throws out-of-memory errors
  3. Analyze Memory Problems on Helpx.
Step 21 Disk I/O

See the Disk I/O section in the Monitoring and Maintaining documentation.
Steps 22 and 22.1 Cache ratio See Calculating the Dispatcher Cache Ratio.
Step 23 Slow queries Best Practices for Queries and Indexing
Step 24 Repository tuning
Step 25 Workflows running

Step 26 MSM Infrastructure

Multi Site Manager Best Practices
Step 27 Assets tuning
  1. Assets Synchronization Service
  2. Multiple DAM Instances
  3. Performance tuning tips articles here and here.
Step 28 Unclosed sessions

Checking for unclosed JCR sessions
Step 30 Move dispatcher closer (add one per "region"?)
Step 31 Use CDN in front of dispatcher Using Dispatcher with a CDN
Step 32 Use session management at the dispatcher level to offload AEM server

Enabling Secure Sessions
Step 33 Make requests cacheable
  1. General Dispatcher Configuration
  2. Configuring the Dispatcher Cache

How to improve cache ratio; make requests cache-able (Dispatcher best practices)

Also, take into consideration the below settings in order to optimize your caching configurations

  1. Set a no-cache rule for HTTP requesrst that are not GET
  2. Configure query strings to not be cacheable
  3. Do not cache URLs with missing extensions
  4. Cache authentication headers (possible since Dispatcher version 4.1.10)
Step 34 Upgrade dispatcher version

You can download the latest Dispatcher version at this location:

Follow link
Step 35 Configure dispatcher Configuring the Dispatcher
Step 36 Check cache invalidation
Steps 37 and 38 Lazy-loading See the Gem Session on AEM Web Performance.
Step 39 Use pre-connect to reduce connection overhead See the Gem Session indicated above. Also, additional documentation preconnect on W3c: https://www.w3.org/TR/resource-hints/#dfn-preconnect
Steps 40 and 41
 External hosts latency and response time Investigate the latency and response time for the external hosts.
Steps 45
and 47
 Using HTTP/2 See the Gem Session for steps 37,38 and 39. Also, check out this forum post on HTTP/2 support.
Step 49 Shrink payload size Enable Gzip and shrink the image size.
Steps 42 and 43 Keep-Alive

Is the Keep-Alive header present in the different requests to re-use connections? Otherwise, it would mean that each requests leads to another connection establishment, which introduces unnecessary overhead. (Standard HTTP request analysis in the browser)

You can check the Proxy Server tool to check for Keep-Alive connections.
Step 44 How many requests are made? Perform standard HTTP request analysis in the browser.
Step 46 Reduce number of requests
  1. Concatenate resources (images, CSS sprites, JSON, etc.)
  2. Clientlibs embedding:
    1. Creating Client Library Folders - see heading Using embedding to minimize requests
Step 48 What is the size of the payload? Standard HTTP request analysis in the browser
Steps 50 and 51 JS code blocking https://experienceleague.adobe.com/docs/experience-manager-gems-events/gems/gems2016/aem-web-performance.html?lang=en

Business.Adobe.com resources

Docs links
Content Insights Style System Personalized Communications Managed Cloud Web Content Management Dynamic Forms

On this page