Functional testing is categorized into three types:
Product Functional Tests are a set of stable HTTP integration tests (ITs) around core functionality in AEM (for example, authoring and replication) that prevent customer changes to their application code from being deployed if it breaks this core functionality.
Product Functional Tests run automatically whenever a customer deploys new code to Cloud Manager and cannot be skipped.
Refer to Product Functional tests for sample tests.
The Custom Functional testing step in the pipeline is always present and cannot be skipped.
However, if no test JAR is produced by the build, the test passes by default.
The Download Log button allows access to a ZIP file containing the logs for the test execution detailed form. These logs do not include the logs of the actual AEM runtime process – those can be accessed using the regular Download or Tail Logs functionality. Refer to Accessing and Managing Logs for more details.
AEM provides its customers with an integrated suite of Cloud Manager quality gates to ensure smooth updates to their applications. In particular, IT test gates already allow customers to create and automate their own tests which use AEM APIs.
The Custom UI testing feature is an optional feature that enables our customers to create and automatically run UI tests for their applications. UI tests are Selenium-based tests packaged in a Docker image in order to allow a wide choice in language and frameworks (such as Java and Maven, Node and WebDriver.io, or any other framework and technology built upon Selenium). You can learn more about how to build UI and write UI tests from here. Additionally a UI Tests project can easily be generated by using the AEM Project Archetype.
The customers can create (via GIT) custom tests and test suite for UI. The UI test will be executed as part of a specific quality gate for each Cloud Manager pipeline with their specific step and feedback information. Any UI tests including regression and new functionalities, will enable errors to be detected and reported within the customer context.
The Customer UI tests run automatically on the Production pipeline under the “Custom UI Testing” step.
Unlike Custom Functional Testing which are HTTP tests written in java, the UI tests can be a docker image with tests written in any language, as long as they follow the conventions defined at Building UI Tests.
It is recommended to follow the structure and language (js and wdio) conveniently provided in the AEM Project Archetype as a starting point.
In order to have their UI tests built and executed, customers need to “opt-in” by adding a file to their code repository, under the maven sub-module for UI tests (next to the pom.xml file of the UI tests submodule) and ensure that this file is at the root of the built
If this is not in the built
tar.gz file, the UI tests build and executions will be skipped
testing.properties file in the built artifact, add an
include statement in
assembly-ui-test-docker-context.xml file (in the UI tests submodule):
``` [...] <includes> <include>Dockerfile</include> <include>wait-for-grid.sh</include> <include>testing.properties</include> <!- opt-in test module in Cloud Manager --> </includes> [...] ```
Production pipelines created before February 10, 2021 will need to be updated in order to use the UI tests as described in this section. This essentially means the User must edit Production pipeline and click Save from the UI even if no changes were made.
Refer to Configuring your CI-CD Pipeline to learn more about pipeline configuration.
Customer-written functional tests must be packaged as a separate JAR file produced by the same Maven build as the artifacts to be deployed to AEM. Generally this would be a separate Maven module. The resulting JAR file must contain all required dependencies and would generally be created using the maven-assembly-plugin using the jar-with-dependencies descriptor.
In addition, the JAR must have the Cloud-Manager-TestType manifest header set to integration-test. In the future, it is expected that additional header values will be supported. An example configuration for the maven-assembly-plugin is:
<build> <plugins> <!-- Create self-contained jar with dependencies --> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <version>3.1.0</version> <configuration> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> <archive> <manifestEntries> <Cloud-Manager-TestType>integration-test</Cloud-Manager-TestType> </manifestEntries> </archive> </configuration> <executions> <execution> <id>make-assembly</id> <phase>package</phase> <goals> <goal>single</goal> </goals> </execution> </executions> </plugin> </plugins>
Within this JAR file, the class names of the actual tests to be executed must end in IT.
For example, a class named
com.myco.tests.aem.ExampleIT would be executed but a class named
com.myco.tests.aem.ExampleTest would not.
The test classes need to be normal JUnit tests. The test infrastructure is designed and configured to be compatible with the conventions used by the aem-testing-clients test library. Developers are strongly encouraged to use this library and follow its best practices. Refer to Git Link for more details.
As the test classes are JUnit tests, they can be run from mainstream Java IDEs like Eclipse, IntelliJ, NetBeans, and so on.
However, when running these tests, it will be necessary to set a variety of system properties expected by the aem-testing-clients (and the underlying Sling Testing Clients).
The system properties are as follows:
sling.it.instances - should be set to 2
sling.it.instance.url.1 - should be set to the author URL, for example, http://localhost:4502
sling.it.instance.runmode.1 - should be set to author
sling.it.instance.adminUser.1 - should be set to the author admin user, e.g. admin
sling.it.instance.adminPassword.1 - should be set to the author admin password
sling.it.instance.url.2 - should be set to the author URL, for example, http://localhost:4503
sling.it.instance.runmode.2 - should be set to publish
sling.it.instance.adminUser.2 - should be set to the publish admin user, for example, admin
sling.it.instance.adminPassword.2 - should be set to the publish admin password