Use config pipelines config-pipelines

Learn how you can use config pipelines to deploy different configurations in AEM as a Cloud Service such as log forwarding settings, purge-related maintenance tasks, and various CDN configurations.

Overview overview

A Cloud Manager config pipeline deploys configurations files (created in YAML format) to a target environment. A number of features in AEM as a Cloud Service can be configured in this way, including log forwarding, purge-related maintenance tasks, and several CDN features.

For Publish Delivery Projects, config pipelines can be deployed via Cloud Manager to dev, stage, and production environment types. The configuration files can be deployed to Rapid Development Environments (RDEs) using command line tooling.

Config pipelines can also be deployed through Cloud Manager for Edge Delivery Projects.

This following sections of this document give an overview of important information regarding how config pipelines can be used and how configurations for them should be structured. It describes general concepts shared across either all or a subset of the features supported by config pipelines.

Supported configurations configurations

The following table offers a comprehensive list of such configurations with links to dedicated documentation describing its distinct configuration syntax and other information.

Create and manage config pipelines creating-and-managing

For information on how to create and configure Publish Delivery config pipelines, see CI/CD Pipelines. When creating a config pipeline in Cloud Manager, be sure to select a Targeted Deployment rather than Full Stack Code when configuring the pipeline. As noted earlier, configuration for RDEs is deployed using command line tooling rather than a pipeline.

For information on how to create and configure Edge Delivery config pipelines, see the Add an Edge Delivery Pipeline article.

Common syntax common-syntax

Each configuration file begins with properties resembling the following example snippet:

   kind: "LogForwarding"
   version: "1"
   metadata:
     envTypes: ["dev"]

You can use the yq utility to validate locally the YAML formatting of your configuration file (for example, yq cdn.yaml).

Folder structure folder-structure

A folder named /config or similar should be at the top of the tree, with one more YAML files somewhere in a tree below it.

For example:

/config
  cdn.yaml

Or

/config
  /dev
    cdn.yaml

The folder names and filenames below /config are arbitrary. The YAML file, however, must include a valid kind property value.

Typically, configurations are deployed to all environments. If all the property values are identical for each environment, a single YAML file is sufficient. However, it is common for property values to differ between environments, for example while testing a lower environment.

The following sections illustrate some strategies for structuring your files.

A single config file for all environments single-file

The file structure resembles the following:

/config
  cdn.yaml
  logForwarding.yaml

Use this structure when the same configuration is sufficient for all environments and for all types of configuration (CDN, log forwarding, and so on). In this scenario, the envTypes array property would include all environment types.

   kind: "cdn"
   version: "1"
   metadata:
     envTypes: ["dev", "stage", "prod"]

Using secret-type environment (or pipeline) variables, it is possible for secret properties to vary per environment, as illustrated by the following ${{SPLUNK_TOKEN}} reference.

kind: "LogForwarding"
version: "1"
metadata:
  envTypes: ["dev"]
data:
  splunk:
    default:
      enabled: true
      host: "splunk-host.example.com"
      token: "${{SPLUNK_TOKEN}}"
      index: "AEMaaCS"

A separate file per environment type file-per-env

The file structure resembles the following:

/config
  cdn-dev.yaml
  cdn-stage.yaml
  cdn-prod.yaml
  logForwarding-dev.yaml
  logForwarding-stage.yaml
  logForwarding-prod.yaml

Use this structure when there may be differences in property values. In the files, one would expect the envTypes array value to correspond with the suffix. For example,cdn-dev.yaml and logForwarding-dev.yaml with a value of ["dev"], cdn-stage.yaml and logForwarding-stage.yaml with a value of ["stage"], and so on.

A folder per environment folder-per-env

In this strategy, there is a separate config folder per environment, with a separate pipeline declared in Cloud Manager for each.

This approach is particularly useful if you have multiple dev environments, where each has unique property values.

The file structure resembles the following:

/config/dev1
  cdn.yaml
  logForwarding.yaml
/config/dev2
  cdn.yaml
  logForwarding.yaml
/config/prod
  cdn.yaml
  logForwarding.yaml

A variation of this approach is to maintain a separate branch per environment.

Edge Delivery Services yamls-for-eds

Edge Delivery config pipelines do not have separate development, staging, and production environments. In Publish Delivery environments, changes progress through dev, stage, and prod tiers. By contrast, an Edge Delivery config pipeline applies configuration directly to all domain mappings registered in Cloud Manager for an Edge Delivery site.

Thus, deploy a simple file structure like:

/config
  cdn.yaml
  logForwarding.yaml

If a rule needs to differ per Edge Delivery site, use the syntax when to distinguish the rules from each other. For example, notice that the domain matches dev.example.com in the snippet below, which can be distinguished from the domain www.example.com.

kind: "CDN"
version: "1"
data:
  trafficFilters:
    rules:
    # Block simple path
    - name: block-path
      when:
        allOf:
          - reqProperty: domain
            equals: "dev.example.com"
          - reqProperty: path
            equals: '/block/me'
      action: block

If including the envTypes metadata field, only the value prod should be used (omitting the envTypes metadata field is also fine). For the tier reqProperty, only the value publish should be used.

Secret environment variables secret-env-vars

So that sensitive information need not be stored in source control, configuration files support Cloud Manager environment variables of type secret. For some configurations, including log forwarding, secret environment variables are mandatory for certain properties.

Note that secret environment variables are used for Publish Delivery projects; see the Secret Pipeline Variables section for Edge Delivery Services projects.

The following snippet is an example of how the secret environment variable ${{SPLUNK_TOKEN}} is used in the configuration.

kind: "LogForwarding"
version: "1"
metadata:
  envTypes: ["dev"]
data:
  splunk:
    default:
      enabled: true
      host: "splunk-host.example.com"
      token: "${{SPLUNK_TOKEN}}"
      index: "AEMaaCS"

For details on how to use environment variables, see Cloud Manager Environment Variables.

Secret pipeline variables secret-pipeline-vars

For Edge Delivery Services Projects, use Cloud Manager pipeline variables of type secret so sensitive information need not be stored in source control. The Step Applied select box should use the deploy option.

The syntax is identical to the snippet shown in the previous section.

For details on how to use pipeline variables, see Pipeline Variables in Cloud Manager.