Functional Testing

Functional testing is categorized into three types:

  • Product Functional Testing
  • Custom Functional Testing
  • Custom UI Testing

Product Functional Testing

Product Functional Tests are a set of stable HTTP integration tests (ITs) around core functionality in AEM (for example, authoring and replication) that prevent customer changes to their application code from being deployed if it breaks this core functionality.

Product Functional Tests run automatically whenever a customer deploys new code to Cloud Manager and cannot be skipped.

Refer to Product Functional tests for sample tests.

Custom Functional Testing

The Custom Functional testing step in the pipeline is always present and cannot be skipped.

However, if no test JAR is produced by the build, the test passes by default.

NOTE

The Download Log button allows access to a ZIP file containing the logs for the test execution detailed form. These logs do not include the logs of the actual AEM runtime process – those can be accessed using the regular Download or Tail Logs functionality. Refer to Accessing and Managing Logs for more details.

Custom UI Testing

AEM provides its customers with an integrated suite of Cloud Manager quality gates to ensure smooth updates to their applications. In particular, IT test gates already allow customers to create and automate their own tests which use AEM APIs.

The Custom UI testing feature is an optional feature Customer Opt-in that enables our customers to create and automatically run UI tests for their applications. UI tests are Selenium-based tests packaged in a Docker image in order to allow a wide choice in language and frameworks (such as Java and Maven, Node and WebDriver.io, or any other framework and technology built upon Selenium). You can learn more about how to build UI and write UI tests from here. Additionally a UI Tests project can easily be generated by using the AEM Project Archetype.

The customers can create (via GIT) custom tests and test suite for UI. The UI test will be executed as part of a specific quality gate for each Cloud Manager pipeline with their specific step and feedback information. Any UI tests including regression and new functionalities, will enable errors to be detected and reported within the customer context.

The Customer UI tests run automatically on the Production pipeline under the “Custom UI Testing” step.

Unlike Custom Functional Testing which are HTTP tests written in java, the UI tests can be a docker image with tests written in any language, as long as they follow the conventions defined at Building UI Tests.

NOTE

It is recommended to follow the structure and language (js and wdio) conveniently provided in the AEM Project Archetype as a starting point.

Customer Opt-in

In order to have their UI tests built and executed, customers need to “opt-in” by adding a file to their code repository, under the maven sub-module for UI tests (next to the pom.xml file of the UI tests submodule) and ensure that this file is at the root of the built tar.gz file.

File name: testing.properties

Contents: one line: ui-tests.version=1

If this is not in the built tar.gz file, the UI tests build and executions will be skipped

NOTE

Production pipelines created before February 10, 2021 will need to be updated in order to use the UI tests as described in this section. This essentially means the User must edit Production pipeline and click Save from the UI even if no changes were made.
Refer to Configuring your CI-CD Pipeline to learn more about pipeline configuration.

Writing Functional Tests

Customer-written functional tests must be packaged as a separate JAR file produced by the same Maven build as the artifacts to be deployed to AEM. Generally this would be a separate Maven module. The resulting JAR file must contain all required dependencies and would generally be created using the maven-assembly-plugin using the jar-with-dependencies descriptor.

In addition, the JAR must have the Cloud-Manager-TestType manifest header set to integration-test. In the future, it is expected that additional header values will be supported. An example configuration for the maven-assembly-plugin is:

<build>
    <plugins>
        <!-- Create self-contained jar with dependencies -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-assembly-plugin</artifactId>
            <version>3.1.0</version>
            <configuration>
                <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
                </descriptorRefs>
                <archive>
                    <manifestEntries>
                        <Cloud-Manager-TestType>integration-test</Cloud-Manager-TestType>
                    </manifestEntries>
                </archive>
            </configuration>
            <executions>
                <execution>
                    <id>make-assembly</id>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>

Within this JAR file, the class names of the actual tests to be executed must end in IT.

For example, a class named com.myco.tests.aem.ExampleIT would be executed but a class named com.myco.tests.aem.ExampleTest would not.

The test classes need to be normal JUnit tests. The test infrastructure is designed and configured to be compatible with the conventions used by the aem-testing-clients test library. Developers are strongly encouraged to use this library and follow its best practices. Refer to Git Link for more details.

Local Test Execution

As the test classes are JUnit tests, they can be run from mainstream Java IDEs like Eclipse, IntelliJ, NetBeans, and so on.

However, when running these tests, it will be necessary to set a variety of system properties expected by the aem-testing-clients (and the underlying Sling Testing Clients).

The system properties are as follows:

  • sling.it.instances - should be set to 2
  • sling.it.instance.url.1 - should be set to the author URL, for example, http://localhost:4502
  • sling.it.instance.runmode.1 - should be set to author
  • sling.it.instance.adminUser.1 - should be set to the author admin user, e.g. admin
  • sling.it.instance.adminPassword.1 - should be set to the author admin password
  • sling.it.instance.url.2 - should be set to the author URL, for example, http://localhost:4503
  • sling.it.instance.runmode.2 - should be set to publish
  • sling.it.instance.adminUser.2 - should be set to the publish admin user, for example, admin
  • sling.it.instance.adminPassword.2 - should be set to the publish admin password

On this page