Code Quality Testing

The Code Quality Testing evaluates the quality of your application code. It is the core objective of a Code-Quality only pipeline and is executed immediately following the build step in all non-production and production pipelines.

Refer to Configuring your CI-CD Pipeline to learn more about different types of pipelines.

Understanding Code Quality Rules

In Code Quality Testing, the source code is scanned to ensure that it meets certain quality criteria. Currently, this is implemented by a combination of SonarQube and content package-level examination using OakPAL. There are over 100 rules combining generic Java rules and AEM-specific rules. Some of the AEM-specific rules are created based on best practices from AEM Engineering and are referred to as Custom Code Quality Rules.

NOTE

You can download the complete list of rules here.

Three-tier Gate

There is a three-tier structure in this code quality testing step for the identified issues:

  • Critical: These are issues identified by the gate which cause an immediate failure of the pipeline.

  • Important: These are issues identified by the gate which cause the pipeline to enter a paused state. A deployment manager, project manager, or business owner can either override the issues, in which case the pipeline proceeds, or they can accept the issues, in which case the pipeline stops with a failure.

  • Info: These are issues identified by the gate which are provided purely for informational purposes and have no impact on the pipeline execution

The results of this step is delivered as Ratings.

The following table summarizes the ratings and failure thresholds for each of the Critical, Important and Information categories:

Name Definition Category Failure Threshold
Security Rating A = 0 Vulnerability
B = at least 1 Minor Vulnerability
C = at least 1 Major Vulnerability
D = at least 1 Critical Vulnerability
E = at least 1 Blocker Vulnerability		 Critical < B
Reliability Rating A = 0 Bug
B = at least 1 Minor Bug
C = at least 1 Major Bug
D = at least 1 Critical Bug E = at least 1 Blocker Bug		 Important < C
Maintainability Rating Outstanding remediation cost for code smells is:
  • <=5% of the time that has already gone into the application, the rating is A
  • between 6 to 10% the rating is a B
  • between 11 to 20% the rating is a C
  • between 21 to 50% the rating is a D
  • anything over 50% is an E
 Important < A
Coverage A mix of unit test line coverage and condition coverage using this formula:
Coverage = (CT + CF + LC)/(2*B + EL)
where: CT = conditions that have been evaluated to ‘true’ at least once while running unit tests
CF = conditions that have been evaluated to ‘false’ at least once while running unit tests
LC = covered lines = lines_to_cover - uncovered_lines

B = total number of conditions
EL = total number of executable lines (lines_to_cover)		 Important < 50%
Skipped Unit Tests Number of skipped unit tests. Info > 1
Open Issues Overall issue types - Vulnerabilities, Bugs, and Code Smells Info > 0
Duplicated Lines Number of lines involved in duplicated blocks.
For a block of code to be considered as duplicated:
  • Non-Java projects:
  • There should be at least 100 successive and duplicated tokens.
  • Those tokens should be spread at least on:
  • 30 lines of code for COBOL
  • 20 lines of code for ABAP
  • 10 lines of code for other languages
  • Java projects:
  • There should be at least 10 successive and duplicated statements whatever the number of tokens and lines.

Differences in indentation as well as in string literals are ignored while detecting duplications.		 Info > 1%
Cloud Service Compatibility Number of identified Cloud Service Compatibility issues. Info > 0
NOTE

Refer to Metric Definitions for more detailed definitions.

NOTE

To learn more about the custom code quality rules executed by Cloud Manager, please refer to Custom Code Quality Rules.

Dealing with False Positives

The quality scanning process is not perfect and will sometimes incorrectly identify issues which are not actually problematic. This is referred to as a false positive.

In these cases, the source code can be annotated with the standard Java @SuppressWarnings annotation specifying the rule ID as the annotation attribute. For example, one common problem is that the SonarQube rule to detect hardcoded passwords can be aggressive about how a hardcoded password is identified.

To look at a specific example, this code would be fairly common in an AEM project which has code to connect to some external service:

@Property(label = "Service Password")
private static final String PROP_SERVICE_PASSWORD = "password";

SonarQube will then raise a Blocker Vulnerability. After reviewing the code, you identify that this is not a vulnerability and can annotate this with the appropriate rule id.

@SuppressWarnings("squid:S2068")
@Property(label = "Service Password")
private static final String PROP_SERVICE_PASSWORD = "password";

However, on the other hand, if the code was actually this:

@Property(label = "Service Password", value = "mysecretpassword")
private static final String PROP_SERVICE_PASSWORD = "password";

Then the correct solution is to remove the hardcoded password.

NOTE

While it is a best practice to make the @SuppressWarnings annotation as specific as possible, i.e. annotate only the specific statement or block causing the issue, it is possible to annotate at a class level.

NOTE

While there is no explicit Security Testing step, there are still security-related code quality rules evaluated during the code quality step. Refer to Security Overview for AEM as a Cloud Service to learn more about security in Cloud Service.

On this page