Introduction functional-testing-introduction
Discover the quality gates available in the AEM as a Cloud Service deployment process and the various types of built-in functional testing. Learn how you can contribute and optimize their use within the framework of a comprehensive testing strategy.
About functional testing
The following diagram provides a high-level overview of the available pipelines in the context of an overall testing strategy and the AEM as a Cloud Service deployment process.
Purpose of functional testing
The purpose of the AEM Cloud Service deployment pipelines is to facilitate robust and secure deployments at various stages of your development and AEM product release lifecycle. These pipelines incorporate multiple quality gates at different levels to ensure the integrity and safety of deployments for both your AEM application changes and AEM product updates.
Adobe provides several built-in quality gates, while others require your intervention for implementation and configuration. These quality gates are versatile, applying at various lifecycle stages and integrating directly into your development setup and CI/CD processes.
The built-in quality gates primarily validate the functionality of the AEM product within the context of your AEM application. In contrast, the custom quality gates you set up are designed to verify that your application’s critical features and user interactions perform as intended. Collectively, these two sets of quality gates work together to ensure robust and secure automated deployments for both your code modifications and AEM product updates.
It is important to note that these quality gates are not intended to be a comprehensive testing framework for your entire testing strategy. The AEM product is subjected to extensive testing before entering the AEM cloud service deployment process. Similarly, your application should already be of high quality before it reaches the deployment phase. This approach ensures that the quality gates focus on their primary objective of safeguarding the deployment process, rather than being a substitute for a full testing regimen.
Quality gates in testing
The following diagram provides a detailed view of available quality gates and their use in the overall testing strategy and the AEM as a Cloud Service deployment process.
Summary customer provided quality gates
Functional Tests
UI Tests
Validations
Testing
Blocking
Blocking
60m Timeout
Blocking
30m Timeout
Blocking
Blocking
60m Timeout
Blocking
30m Timeout
Blocking
Blocking
60m Timeout
Blocking
30m Timeout
Unit test
You are encouraged to provide the unit tests for your AEM application, which are the foundation of every testing strategy. They are intended to run fast and often and give early and fast feedback. They are tightly integrated into the developer workflows, your own CI/CD and the AEM cloud service deployment pipelines.
They are implemented using JUnit and are executed with Maven. See the core module of the AEM Project Archetype for an example unit test for AEM and getting started.
Code quality
This quality gate is configured out-of-the-box and runs static code analysis on your AEM application code.
See Code Quality Testing and Custom code quality rules for more information.
Product tests
Product functional tests are stable HTTP integration tests (ITs) for core AEM functionality, including authoring and replication tasks. Adobe provides and maintains them out-of-the-box. They are intended to prevent changes to custom application code from being deployed if it breaks core functionality in the AEM product.
They use JUnit for implementation, run with Maven, and rely on the official AEM Testing Clients. The product test suite is maintained as
an open-source project, follows best-practices and can be considered a good starting point for the implementation of your tests.
Custom functional tests
Similar to the product tests, customer functional tests are HTTP integration tests (ITs) implemented with JUnit, run using Maven, and built on top of the official AEM Testing Clients.
To maintain efficient pipeline runs, Adobe advises focusing on key features and primary user interaction flows, aiming for a functional test runtime of around 15 minutes or less. Full functional test suites that exceed this time should be executed as part of the general customer validation pipelines during the development process.
See open-sourced product tests or the it.tests module of the AEM Projects Archetype for examples.
See Java Functional Tests for more information.
Custom UI tests
To maximize risk control for your customer-specific development, Adobe encourages you to capture critical UI tests into AEM as a Cloud Service. Keep them limited but focused on maximizing their impact on the customer experience.
The tests are packaged in a Docker image - designed to be as volatile as possible (with support for Cypress, Playwright, Selenium, Java, and JavaScript). They follow the same characteristics and purposes like the custom functional tests.
To keep pipeline executions efficient, Adobe recommends focusing on key features and main user interaction flows. Full UI test suites that exceed this quality gate should be executed as part of the general customer validation pipelines. Incorporate them into the customer’s development process.
See open-sourced example tests or the ui.tests module of the AEM Projects Archetype for examples.
See Custom UI Testing for more information.
Experience audit
The experience audit quality gate is performing Google Lighthouse audits against the customer’s webpage.
This quality gate is provided by AEM out-of-the-box, but is not blocking the deployment pipelines. By default, an audit against the root page (/
) of the publish instance is performed. You can contribute by configuring up to 25 custom paths that are considered for audits.
See Experience Audit Testing for more information.
Customer validations
The customer validations quality gate is a placeholder for the customer’s own testing strategy and effort, executed before the customer’s application changes reach the AEM cloud deployment pipelines.
Here you can choose the tools and frameworks you prefer. In contrast to customer function tests and custom UI tests, there is no AEM as a Cloud Service-related limits. As such, Adobe recommends that you perform long-running functional and UI testing here.
While you can choose any tool and framework, Adobe suggests aligning HTTP-based integration and UI tests with the tools and frameworks used in the custom functional and UI test quality gates. Additionally, Adobe recommends incorporating Rapid Development Environments (RDE) into your local testing strategy to mirror AEM cloud environments closely.
Manual testing
The Manual testing quality gate is a placeholder for customers who do manual testing. Because AEM cloud pipelines do not support manual testing, it must be included in your local testing strategy.
For manual testing, it can be useful to integrate with an additional AEM Cloud Service development environment.