UI Testing ui-testing

Custom UI testing is an optional feature that enables you to create and automatically run UI tests for your applications.

Overview custom-ui-testing

AEM provides an integrated suite of Cloud Manager quality gates to ensure smooth updates to custom applications. In particular, IT test gates already support the creation and automation of custom tests using AEM APIs.

UI tests are packaged in a Docker image to allow a wide choice in language and frameworks (such as Cypress, Selenium, Java and Maven, and JavaScript). Also, a UI tests project can easily be generated by using the AEM Project Archetype.

Adobe encourages the usage of Cypress, as it offers real-time reloading and automatic waiting, which helps save time and improves productivity during testing. Cypress also provides a simple and intuitive syntax, making it easy to learn and use, even for those who are new to testing.

UI tests are executed as part of a specific quality gate for each Cloud Manager pipeline with a Custom UI Testing step in production pipelines or optionally non-production pipelines. Any UI tests including regression and new functionalities enable errors to be detected and reported.

Unlike custom functional tests, which are HTTP tests written in Java, UI tests can be a Docker image with tests written in any language, as long as they follow the conventions defined in the section Building UI Tests.

Get Started with UI Tests get-started-ui-tests

This section describes the steps required to set up UI tests for execution in Cloud Manager.

Building UI Tests building-ui-tests

A Maven project generates a Docker build context. This Docker build context describes how to create a Docker image containing the UI tests, which Cloud Manager uses to generate a Docker image containing the actual UI tests.

This section describes the steps needed to add a UI tests project to your repository.

Generate a Docker Build Context generate-docker-build-context

To generate a Docker build context, you need a Maven module that:

The simplest way to do this is to configure the Maven Assembly Plugin to create the Docker build context archive and assign the right classifier to it.

You can build UI tests with different technologies and frameworks, but this section assumes that your project is laid out in a way similar to the following.

├── Dockerfile
├── assembly-ui-test-docker-context.xml
├── pom.xml
├── test-module
│   ├── package.json
│   ├── index.js
│   └── wdio.conf.js
└── wait-for-grid.sh

The pom.xml file takes care of the Maven build. Add an execution to the Maven Assembly Plugin similar to the following.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-assembly-plugin</artifactId>
    <configuration>
        <descriptors>
            <descriptor>${project.basedir}/assembly-ui-test-docker-context.xml</descriptor>
        </descriptors>
        <tarLongFileMode>gnu</tarLongFileMode>
    </configuration>
    <executions>
        <execution>
            <id>make-assembly</id>
            <phase>package</phase>
            <goals>
                <goal>single</goal>
            </goals>
        </execution>
    </executions>
</plugin>

This execution instructs the Maven Assembly Plugin to create an archive based on the instructions contained in assembly-ui-test-docker-context.xml, called an assembly descriptor in the plugin’s jargon. The assembly descriptor lists all the files that must be part of the archive.

<assembly>
    <id>ui-test-docker-context</id>
    <includeBaseDirectory>false</includeBaseDirectory>
    <formats>
        <format>tar.gz</format>
    </formats>
    <fileSets>
        <fileSet>
            <directory>${basedir}</directory>
            <includes>
                <include>Dockerfile</include>
                <include>wait-for-grid.sh</include>
            </includes>
        </fileSet>
        <fileSet>
            <directory>${basedir}/test-module</directory>
            <excludes>
                <exclude>node/**</exclude>
                <exclude>node_modules/**</exclude>
                <exclude>reports/**</exclude>
            </excludes>
        </fileSet>
    </fileSets>
</assembly>

The assembly descriptor instructs the plugin to create an archive of type .tar.gz and assigns the ui-test-docker-context classifier to it. Moreover, it lists the files that must be included in the archive, including the following:

The assembly descriptor also excludes some files that might be generated while running the UI tests locally. This guarantees a smaller archive and faster builds.

The archive containing the Docker build context is automatically picked up by Cloud Manager, which will build the Docker image containing your tests during its deployment pipelines. Eventually, Cloud Manager will run the Docker image to execute the UI tests against your application.

The build should produce either zero or one archive. If it produces zero archives, the test step passes by default. If the build produces more than one archive, which archive is selected is non-deterministic.

Customer Opt-In customer-opt-in

For Cloud Manager to build and execute your UI tests, you must opt into this feature by adding a file to your repository.

The UI tests build and executions are skipped if this file is not present.

To include a testing.properties file in the build artifact, add an include statement in the assembly-ui-test-docker-context.xml file.

[...]
<includes>
    <include>Dockerfile</include>
    <include>wait-for-grid.sh</include>
    <include>testing.properties</include> <!-- opt-in test module in Cloud Manager -->
</includes>
[...]

If you are using the samples provided by Adobe:

Writing UI Tests writing-ui-tests

This section describes the conventions that the Docker image containing your UI tests must follow. The Docker image is built out of the Docker build context described in the previous section.

Environment Variables environment-variables

The following environment variables are passed to your Docker image at run time, depending on your framework.

* these values will be empty if there is no publish instance

The Adobe test samples provide helper functions to access the configuration parameters:

Generate Test Reports generate-test-reports

The Docker image must generate test reports in the JUnit XML format and save them in the path specified by the environment variable REPORTS_PATH. The JUnit XML format is a widely used format for reporting the results of tests. If the Docker image uses Java and Maven, standard test modules such as Maven Surefire Plugin and Maven Failsafe Plugin can generate such reports out of the box.

If the Docker image is implemented with other programming languages or test runners, check the documentation for the chosen tools for how to generate JUnit XML reports.

Prerequisites prerequisites

Selenium-specific details

Waiting for Selenium to be Ready waiting-for-selenium

Before the tests start, it’s the responsibility of the Docker image to ensure that the Selenium server is up and running. Waiting for the Selenium service is a two-steps process.

Once the Selenium’s status endpoint answers with a positive response, the tests can start.

The Adobe UI test samples handle this with the script wait-for-grid.sh, which is executed upon Docker startup and starts the actual test execution only once the grid is ready.

Capture Screenshots and Videos capture-screenshots

The Docker image may generate additional test output (for example, screenshots or videos) and save them in the path specified by the environment variable REPORTS_PATH. Any file found below the REPORTS_PATH are included in the test result archive.

The test samples provided by Adobe by default create screenshots for any failed test.

You can use the helper functions to create screenshots through your tests.

If a test result archive is created during a UI test execution, you can download it from Cloud Manager using the Download Details button under the Custom UI Testing step.

Upload Files upload-files

Tests sometimes must upload files to the application being tested. To keep the deployment of Selenium flexible relative to your tests, it is not possible to directly upload an asset directly to Selenium. Instead, uploading a file requires the following steps.

Cypress-specific details

Set up HTTP proxy

The Docker container’s entrypoint needs to check the value of the PROXY_HOST environment variable.

If this value is empty, no additional steps are required and the tests should be executed without using HTTP proxy.

If it’s not empty, the entrypoint script needs to:

An example implementation can be found in the Cypress Sample Test Module’s Entrypoint on GitHub.

Playwright-specific details

Set up HTTP proxy

Similar to Cypress, tests need to use the HTTP proxy if a non-empty PROXY_HOST environment variable is provided.

To do this, the following modifications need to be made.

Dockerfile

Install cURL and libnss3-tools, which provides certutil.

RUN apt -y update \
    && apt -y --no-install-recommends install curl libnss3-tools \
    && rm -rf /var/lib/apt/lists/*

Entrypoint script

Include a bash script that, in case PROXY_HOST environment variable is provided, does the following:

Example implementation:

# setup proxy environment variables and CA certificate
if [ -n "${PROXY_HOST:-}" ]; then
  if [ -n "${PROXY_HTTPS_PORT:-}" ]; then
    export HTTP_PROXY="https://${PROXY_HOST}:${PROXY_HTTPS_PORT}"
  elif [ -n "${PROXY_HTTP_PORT:-}" ]; then
    export HTTP_PROXY="http://${PROXY_HOST}:${PROXY_HTTP_PORT}"
  fi
  if [ -n "${PROXY_CA_PATH:-}" ]; then
    echo "installing certificate"
    mkdir -p $HOME/.pki/nssdb
    certutil -d sql:$HOME/.pki/nssdb -A -t "CT,c,c" -n "EaaS Client Proxy Root" -i $PROXY_CA_PATH
    export NODE_EXTRA_CA_CERTS=${PROXY_CA_PATH}
  fi
  if [ -n "${PROXY_OBSERVABILITY_PORT:-}" ] && [ -n "${HTTP_PROXY:-}" ]; then
    echo "waiting for proxy"
    curl --silent  --retry ${PROXY_RETRY_ATTEMPTS:-3} --retry-connrefused --retry-delay ${PROXY_RETRY_DELAY:-10} \
      --proxy ${HTTP_PROXY} --proxy-cacert ${PROXY_CA_PATH:-""} \
      ${PROXY_HOST}:${PROXY_OBSERVABILITY_PORT}
    if [ $? -ne 0 ]; then
      echo "proxy is not ready"
      exit 1
    fi
  fi
fi

Playwright configuration

Modify the playwright configuration (for example in playwright.config.js) to use a proxy in case the HTTP_PROXY environment variable is set.

Example implementation:

const proxyServer = process.env.HTTP_PROXY || ''

// enable proxy if set
if (proxyServer !== '') {
 cfg.use.proxy = {
  server: proxyServer,
 }
}

Running UI Tests Locally run-ui-tests-locally

Before activating UI tests in a Cloud Manager pipeline, it’s recommended to run the UI tests locally against the AEM as a Cloud Service SDK or against an actual AEM as a Cloud Service instance.

Cypress Test Sample cypress-sample

JavaScript WebdriverIO Test Sample javascript-sample

Playwright Test Sample playwright-sample

Java Selenium WebDriver Test Sample java-sample