Set up local Dispatcher Tools

Adobe Experience Manager (AEM)'s Dispatcher is a Apache HTTP Web server module that provides a security and performance layer between the CDN and AEM Publish tier. Dispatcher is an integral part of the overall Experience Manager architecture and should be part of local development set up.

The AEM as a Cloud Service SDK includes the recommended Dispatcher Tools version, that facilitates configuring, validating and simulating Dispatcher locally. Dispatcher Tools is comprised of:

• a baseline set of Apache HTTP Web server and Dispatcher configuration files, located in .../dispatcher-sdk-x.x.x/src
• a configuration validator CLI tool, located at .../dispatcher-sdk-x.x.x/bin/validate
• a configuration generation CLI tool, located at .../dispatcher-sdk-x.x.x/bin/validator
• a configuration deployment CLI tool, located at .../dispatcher-sdk-x.x.x/bin/docker_run
• a Docker image that runs Apache HTTP Web server with the Dispatcher module

Note that ~ is used as shorthand for the User’s Directory. In Windows, this is the equivalent of %HOMEPATH%.

Prerequisites

1. Windows users must use Windows 10 Professional (or a version that supports Docker)
2. Install Experience Manager Publish Quickstart Jar on the local develop machine.
   • Optionally, install the latest AEM reference web site on the local AEM Publish service. This web site is used in this tutorial to visualize a working Dispatcher.
3. Install and start the latest version of Docker (Docker Desktop 2.2.0.5+ / Docker Engine v19.03.9+) on the local development machine.

Download the Dispatcher Tools (as part of the AEM SDK)

The AEM as a Cloud Service SDK, or AEM SDK, contains the Dispatcher Tools used to run Apache HTTP Web server with the Dispatcher module locally for development, as well as the compatible QuickStart Jar.

If the AEM as a Cloud Service SDK has already been downloaded to setup the local AEM runtime, it does not need to be re-downloaded.

1. Log in to experience.adobe.com/#/downloads with your Adobe ID
   • Your Adobe Organization must be provisioned for AEM as a Cloud Service to download the AEM as a Cloud Service SDK
2. Click on the latest AEM SDK result row to download

Extract the Dispatcher Tools from the AEM SDK zip

The version of Dispatcher Tools is different from that of the AEM SDK. Ensure the version of Dispatcher Tools is provided via the AEM SDK version matching the AEM as a Cloud Service version.

1. Unzip the downloaded aem-sdk-xxx.zip file
2. Unpack the Dispatcher Tools into ~/aem-sdk/dispatcher
   • Windows: Unzip aem-sdk-dispatcher-tools-x.x.x-windows.zip into C:\Users\<My User>\aem-sdk\dispatcher (creating missing folders as needed)
   • macOS / Linux: Execute the accompanying shell script aem-sdk-dispatcher-tools-x.x.x-unix.sh to unpack the Dispatcher Tools
     • chmod a+x aem-sdk-dispatcher-tools-x.x.x-unix.sh && ./aem-sdk-dispatcher-tools-x.x.x-unix.sh

Note that all commands issued below assume the current working directory contains the expanding Dispatcher Tools contents.

This video uses macOS for illustrative purposes. The equivalent Windows/Linux commands can be used to achieve similar results

Understand the Dispatcher configuration files

The Dispatcher Tools provides a set of Apache HTTP Web server and Dispatcher configuration files that define behavior for all environments, including local development.

These files are intended to be copied into an Experience Manager Maven project to the dispatcher/src folder, if they do not already exist in the Experience Manager Maven project.

A complete description of the configuration files is available in the unpacked Dispatcher Tools as dispatcher-sdk-x.x.x/docs/Config.html.

Validate configurations

Optionally, the Dispatcher and Apache Web server configurations (via httpd -t) can be validated using the validate script (not to be confused with the validator executable). The validate script provides a convenient way of running the 3 phases of the validator.

• Usage:
  • Windows: bin\validate src
  • macOS / Linux: ./bin/validate.sh ./src

Run Dispatcher locally

AEM Dispatcher is run locally using Docker against the src Dispatcher and Apache Web server configuration files.

• Usage:
  • Windows: bin\docker_run <src-folder> <aem-publish-host>:<aem-publish-port> <dispatcher-port>
  • macOS / Linux: ./bin/docker_run.sh <src-folder> <aem-publish-host>:<aem-publish-port> <dispatcher-port>

The <aem-publish-host> can be set to host.docker.internal, a special DNS name Docker provides in the container that resolves to the host machine’s IP. If he host.docker.internal does not resolve, please see the troubleshooting section below.

For example to start the Dispatcher Docker container using the default configuration files provided by the Dispatcher Tools:

Start Dispatcher Docker container providing the path to the Dispatcher configuration src folder:

• Windows: bin\docker_run src host.docker.internal:4503 8080
• macOS / Linux: ./bin/docker_run.sh ./src host.docker.internal:4503 8080

The AEM as a Cloud Service SDK’s Publish Service, running locally on port 4503 will be available through Dispatcher at http://localhost:8080.

To run Dispatcher Tools against an Experience Manager project’s Dispatcher configuration, point to your project’s dispatcher/src folder.

• Windows:

  $ bin\docker_run <User Directory>/code/my-project/dispatcher/src host.docker.internal:4503 8080
  

• macOS / Linux:

  $ ./bin/docker_run.sh ~/code/my-project/dispatcher/src host.docker.internal:4503 8080
  

Dispatcher Tools logs

Dispatcher logs are helpful during local development to understand if and why HTTP Requests are blocked. Log level can be set by prefixing the execution of docker_run with environment parameters.

Dispatcher Tools logs are emitted to the standard out when docker_run is run.

Useful parameters for debugging Dispatcher include:

• DISP_LOG_LEVEL=Debug sets Dispatcher module logging to Debug level
  • Default value is: Warn
• REWRITE_LOG_LEVEL=Debug sets Apache HTTP Web server rewrite module logging to Debug level
  • Default value is: Warn
• DISP_RUN_MODE sets the “run mode” of the Dispatcher environment, loading the corresponding run modes Dispatcher configuration files.
  • Defaults to dev
• Valid values: dev, stage, or prod

One or many parameters, can be passed to docker_run

• Windows:

  $ DISP_LOG_LEVEL=Debug REWRITE_LOG_LEVEL=Debug bin\docker_run <User Directory>/code/my-project/dispatcher/src host.docker.internal:4503 8080
  

• macOS / Linux:

  $ DISP_LOG_LEVEL=Debug REWRITE_LOG_LEVEL=Debug ./bin/docker_run.sh ~/code/my-project/dispatcher/src host.docker.internal:4503 8080
  

Log file access

Apache web server and AEM Dispatcher logs can be directly accessed in the Docker container:

• Accessing logs in the Docker container
• Copying the Docker logs to the local filesystem

When to update Dispatcher Tools

Dispatcher Tools versions increment less frequently than the Experience Manager, and thus Dispatcher Tools require fewer updates in the local development environment.

The recommended Dispatcher Tools version is that which is bundled with the AEM as a Cloud Service SDK that matches the Experience Manager as a Cloud Service version. The version of AEM as a Cloud Service can be found via Cloud Manager.

• Cloud Manager > Environments, per environment specified by the AEM Release label

Note that Dispatcher Tools version itself will not match the Experience Manager version.

Troubleshooting

docker_run results in ‘Waiting until host.docker.internal is available’ message

host.docker.internal is an hostname provided to the Docker contain that resolves to the host. Per docs.docker.com (macOS, Windows):

From Docker 18.03 onwards our recommendation is to connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host

If, when bin/docker_run src host.docker.internal:4503 8080 results in the message Waiting until host.docker.internal is available, then:

1. Ensure the installed version of Docker is 18.03 or greater
2. You may have an local machine set up that is preventing the registration/resolution of the host.docker.internal name. Instead use your local IP.
   • Windows:
     • From the Command Prompt, execute ipconfig, and record the host’s IPv4 Address of the host machine.
     • Then, execute docker_run using this IP address:
       bin\docker_run src <HOST IP>:4503 8080
   • macOS / Linux:
     • From Terminal, execute ifconfig and record the Host inet IP address, usually the en0 device.
     • Then execute docker_run using the host IP address:
       bin/docker_run.sh src <HOST IP>:4503 8080

Example error

$ docker_run src host.docker.internal:4503 8080

Running script /docker_entrypoint.d/10-check-environment.sh
Running script /docker_entrypoint.d/20-create-docroots.sh
Running script /docker_entrypoint.d/30-wait-for-backend.sh
Waiting until host.docker.internal is available

docker_run fails to start on Windows

Running docker_run on Windows can result in the following error, preventing Dispatcher from starting. This is a reported issue with Dispatcher on Windows and will be fixed in a future release.

Example error

$ \Users\MyUser\aem-sdk\dispatcher>bin\docker_run src host.docker.internal:4503 8080

Running script /docker_entrypoint.d/10-check-environment.sh
Running script /docker_entrypoint.d/20-create-docroots.sh
Running script /docker_entrypoint.d/30-wait-for-backend.sh
Waiting until host.docker.internal is available
host.docker.internal resolves to 192.168.65.2
Running script /docker_entrypoint.d/40-generate-allowed-clients.sh
Running script /docker_entrypoint.d/50-check-expiration.sh
Running script /docker_entrypoint.d/60-check-loglevel.sh
Running script /docker_entrypoint.d/70-check-forwarded-host-secret.sh
Starting httpd server
[Sun Feb 09 17:32:22.256020 2020] [dispatcher:warn] [pid 1:tid 140080096570248] Unable to obtain parent directory of /etc/httpd/conf.dispatcher.d/enabled_farms/farms.any: No such file or directory
[Sun Feb 09 17:32:22.256069 2020] [dispatcher:alert] [pid 1:tid 140080096570248] Unable to import config file: /etc/httpd/conf.dispatcher.d/dispatcher.any
[Sun Feb 09 17:32:22.256074 2020] [dispatcher:alert] [pid 1:tid 140080096570248] Dispatcher initialization failed.
AH00016: Configuration Failed

Additional Resources

• Download AEM SDK
• Adobe Cloud Manager
• Download Docker
• Download the AEM Reference Website (WKND)
• Experience Manager Dispatcher Documentation