The Code Quality Testing evaluates the quality of your application code. It is the core objective of a Code-Quality only pipeline and is executed immediately following the build step in all non-production and production pipelines.
Refer to Configuring your CI-CD Pipeline to learn more about different types of pipelines.
In Code Quality Testing, the source code is scanned to ensure that it meets certain quality criteria. Currently, this is implemented by a combination of SonarQube and content package-level examination using OakPAL. There are over 100 rules combining generic Java rules and AEM-specific rules. Some of the AEM-specific rules are created based on best practices from AEM Engineering and are referred to as Custom Code Quality Rules.
You can download the complete list of rules here.
Three-tier Gate
There is a three-tier structure in this code quality testing step for the identified issues:
Critical: These are issues identified by the gate which cause an immediate failure of the pipeline.
Important: These are issues identified by the gate which cause the pipeline to enter a paused state. A deployment manager, project manager, or business owner can either override the issues, in which case the pipeline proceeds, or they can accept the issues, in which case the pipeline stops with a failure.
Info: These are issues identified by the gate which are provided purely for informational purposes and have no impact on the pipeline execution
The results of this step is delivered as Ratings.
The following table summarizes the ratings and failure thresholds for each of the Critical, Important and Information categories:
Name | Definition | Category | Failure Threshold |
---|---|---|---|
Security Rating | A = 0 Vulnerability B = at least 1 Minor Vulnerability C = at least 1 Major Vulnerability D = at least 1 Critical Vulnerability E = at least 1 Blocker Vulnerability |
Critical | < B |
Reliability Rating | A = 0 Bug B = at least 1 Minor Bug C = at least 1 Major Bug D = at least 1 Critical Bug E = at least 1 Blocker Bug |
Important | < C |
Maintainability Rating | Outstanding remediation cost for code smells is:
|
Important | < A |
Coverage | A mix of unit test line coverage and condition coverage using this formula: Coverage = (CT + CF + LC)/(2*B + EL) where: CT = conditions that have been evaluated to ‘true’ at least once while running unit tests CF = conditions that have been evaluated to ‘false’ at least once while running unit tests LC = covered lines = lines_to_cover - uncovered_lines B = total number of conditions EL = total number of executable lines (lines_to_cover) |
Important | < 50% |
Skipped Unit Tests | Number of skipped unit tests. | Info | > 1 |
Open Issues | Overall issue types - Vulnerabilities, Bugs, and Code Smells | Info | > 0 |
Duplicated Lines | Number of lines involved in duplicated blocks. For a block of code to be considered as duplicated:
Differences in indentation as well as in string literals are ignored while detecting duplications. |
Info | > 1% |
Cloud Service Compatibility | Number of identified Cloud Service Compatibility issues. | Info | > 0 |
Refer to Metric Definitions for more detailed definitions.
To learn more about the custom code quality rules executed by Cloud Manager, please refer to Custom Code Quality Rules.
The quality scanning process is not perfect and will sometimes incorrectly identify issues which are not actually problematic. This is referred to as a false positive.
In these cases, the source code can be annotated with the standard Java @SuppressWarnings
annotation specifying the rule ID as the annotation attribute. For example, one common problem is that the SonarQube rule to detect hardcoded passwords can be aggressive about how a hardcoded password is identified.
To look at a specific example, this code would be fairly common in an AEM project which has code to connect to some external service:
@Property(label = "Service Password")
private static final String PROP_SERVICE_PASSWORD = "password";
SonarQube will then raise a Blocker Vulnerability. After reviewing the code, you identify that this is not a vulnerability and can annotate this with the appropriate rule id.
@SuppressWarnings("squid:S2068")
@Property(label = "Service Password")
private static final String PROP_SERVICE_PASSWORD = "password";
However, on the other hand, if the code was actually this:
@Property(label = "Service Password", value = "mysecretpassword")
private static final String PROP_SERVICE_PASSWORD = "password";
Then the correct solution is to remove the hardcoded password.
While it is a best practice to make the @SuppressWarnings
annotation as specific as possible, i.e. annotate only the specific statement or block causing the issue, it is possible to annotate at a class level.
While there is no explicit Security Testing step, there are still security-related code quality rules evaluated during the code quality step. Refer to Security Overview for AEM as a Cloud Service to learn more about security in Cloud Service.