Traffic Filter Rules Including WAF Rules traffic-filter-rules-including-waf-rules
Traffic filter rules can be used to block or allow requests at the CDN layer, which may be useful in scenarios such as:
- Restricting access to specific domains to internal company traffic, before a new site goes live
- Establishing rate limits to be less susceptible to volumetric DoS attacks
- Preventing IP addresses known to be malicious from targeting your pages
Most of these traffic filter rules are available to all AEM as a Cloud Service Sites and Forms customers. They mainly operate on request properties and request headers, including IP, hostname, path, and user agent.
A subcategory of traffic filter rules requires either an Enhanced Security license or WAF-DDoS Protection license. These powerful rules are known as WAF (Web Application Firewall) traffic filter rules (or WAF rules for short) and have access to the WAF Flags described later in this article.
Traffic filter rules can be deployed via Cloud Manager config pipelines to dev, stage, and production environment types in production (non-sandbox) programs. Support for RDEs will come in the future.
Follow through a tutorial to quickly build concrete expertise on this feature.
How This Article is Organized how-organized
This article is organized into the following sections:
- Traffic protection overview: Learn how you are protected from malicious traffic.
- Suggested process for configuring rules: Read about a high-level methodology for protecting your website.
- Setup: Discover how to setup, configure, and deploy traffic filter rules, including the advanced WAF rules.
- Rules syntax: Read about how to declare traffic filter rules in the
cdn.yaml
configuration file. This includes both the traffic filter rules available to all Sites and Forms customers, and the subcategory of WAF rules for those who license that capability. - Rules examples: See examples of declared rules to get you on your way.
- Rate limit rules: Learn how to use rate limiting rules to protect your site from high volume attacks.
- Traffic Filter Rules Alerts Configure alerts to be notified when your rules are triggered.
- Default Traffic Spike at Origin Alert Get notified when there is a surge of traffic at the origin suggestive of a DDoS attack.
- CDN logs: See what declared rules and WAF Flags match your traffic.
- Dashboard Tooling: Analyze your CDN logs to come up with new traffic filter rules.
- Recommended Starter Rules: A set of rules to get started with.
- Tutorial: Practical knowledge about the feature, including how to use dashboard toolings to declare the right rules.
Adobe invites you to give feedback or ask questions about traffic filter rules by emailing aemcs-waf-adopter@adobe.com.
Traffic Protection Overview traffic-protection-overview
In the current digital landscape, malicious traffic is an ever-present threat. Adobe recognizes the gravity of the risk and offers several approaches to protect customer applications and mitigate attacks when they occur.
At the edge, the Adobe Managed CDN absorbs DoS attacks at the network layer (layers 3 and 4), including flood and reflection/amplification attacks.
By default, Adobe takes measures to prevent performance degradation due to bursts of unexpectedly high traffic beyond a certain threshold. If there is a DoS attack that impacts site availability, Adobe’s operations teams are alerted and take steps to mitigate.
Customers may take proactive measures to mitigate application layer attacks (layer 7) by configuring rules at various layers of the content delivery flow.
For example, at the Apache layer, customers may configure either the Dispatcher module or ModSecurity to limit access to certain content.
As this article describes, traffic filter rules may be deployed to the Adobe Managed CDN, using Cloud Manager’s config pipelines. In addition to traffic filter rules based on properties like IP address, path, and headers, or rules based on setting rate limits, customers may also license a powerful subcategory of traffic filter rules called WAF rules.
Suggested Process suggested-process
The following is a high-level recommended end-to-end process for coming up with the right traffic filter rules:
- Configure non-production and production config pipelines, as described in the Setup section.
- Customers who have licensed the subcategory of WAF traffic filter rules should enable them in Cloud Manager.
- Read and try out the tutorial to concretely understand how to use traffic filter rules, including WAF rules if they’ve been licensed. The tutorial walks you through deploying rules to a dev environment, simulating malicious traffic, downloading the CDN logs, and analyzing them in dashboard tooling.
- Copy the recommended starter rules to
cdn.yaml
and deploy the configuration to the production environment in log mode. - After collecting some traffic, analyze the results using dashboard tooling to see if there were any matches. Lookout for false positives, and make any necessary adjustments, ultimately enabling the starter rules in block mode.
- Add custom rules based on analysis of the CDN logs, first testing with simulated traffic on dev environments before deploying to stage and production environments in log mode, then block mode.
- Monitor traffic on an ongoing basis, changing the rules as the threat landscape evolves.
Setup setup
-
Create a file
cdn.yaml
with a set of traffic filter rules, including WAF rules.code language-none kind: "CDN" version: "1" metadata: envTypes: ["dev"] data: trafficFilters: rules: # Block simple path - name: block-path when: allOf: - reqProperty: tier matches: "author|publish" - reqProperty: path equals: '/block/me' action: block
See Using Config Pipelines for a description of the properties above the
data
node. Thekind
property value should be set to CDN and the version should be set to1
. -
If WAF rules are licensed, you should enable the feature in Cloud Manager, as described below for both the new and existing program scenarios.
-
To configure WAF on a new program, check the WAF-DDOS Protection check-box on the Security tab when you add a production program.
-
To configure WAF on an existing program, edit your program and on the Security tab uncheck or check the WAF-DDOS option at any time.
-
-
Create a config pipeline in Cloud Manager, as described in config pipeline article. The pipeline will reference a top level
config
folder with thecdn.yaml
file placed somewhere below, see Using Config Pipelines.
Traffic Filter Rules Syntax rules-syntax
You can configure traffic filter rules to match on patterns such as IPs, user agent, request headers, hostname, geo, and url.
Customers who license the Enhanced Security or WAF-DDoS Protection Security offering can also configure a special category of traffic filter rules called WAF traffic filter rules (or WAF rules for short) that reference one or more WAF flags.
Here’s an example of a set of traffic filter rules, which also includes a WAF rule.
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: "path-rule"
when:
allOf:
- { reqProperty: path, equals: /block-me }
- { reqProperty: tier, equals: publish }
action:
type: block
- name: "Enable-SQL-Injection-and-XSS-waf-rules-globally"
when: { reqProperty: path, like: "*" }
action:
type: block
wafFlags: [ SQLI, XSS]
The format of the traffic filter rules in the cdn.yaml
file is described below. See some other examples in a later section, and a separate section on Rate Limit Rules.
string
Condition
{ <getter>: <value>, <predicate>: <value> }
See Condition Structure syntax below, which describes the getters, predicates, and how to combine multiple conditions.
Action
RateLimit
There is a separate section further below describing the rateLimit syntax, along with examples.
Condition Structure condition-structure
A Condition can be either a simple Condition or a group of Conditions.
Simple Condition
A Simple Condition is composed of a getter and a predicate.
{ <getter>: <value>, <predicate>: <value> }
Group Conditions
A Group of Conditions is composed of multiple Simple and/or Group Conditions.
<allOf|anyOf>:
- { <getter>: <value>, <predicate>: <value> }
- { <getter>: <value>, <predicate>: <value> }
- <allOf|anyOf>:
- { <getter>: <value>, <predicate>: <value> }
array[Condition]
array[Condition]
Getter
string
Request property.
One of:
path
: Returns the full path of a URL without the query parameters.queryString
: Returns the query part of a URLmethod
: Returns the HTTP method used in the request.tier
: Returns one ofauthor
,preview
, orpublish
.domain
: Returns the domain property (as defined in theHost
header) in lower-caseclientIp
: Returns the client IP.clientCountry
: Returns a two letter code (Regional indicator symbol) that identify in which country the client is located.
string
string
string
string
application/x-www-form-urlencoded
Predicate
string
string
string
string
string
string
array[string]
array[string]
boolean
Notes
- The request property
clientIp
can only be used with the following predicates:equals
,doesNotEqual
,in
,notIn
.clientIp
can also be compared against IP ranges when usingin
andnotIn
predicates. The following example implements a condition to evaluate if a client IP is in the IP range of 192.168.0.0/24 (so from 192.168.0.0 to 192.168.0.255):
when:
reqProperty: clientIp
in: [ "192.168.0.0/24" ]
- Adobe recommends the use of regex101 and Fastly Fiddle when working with regex. You can also learn more about how Fastly handles regex from fastly documentation - Regular expressions in Fastly VCL.
Action Structure action-structure
An action
can either be a string specifying the action (allow, block or log), or an object composed of both the action type (allow, block or log) and options like wafFlags and/or status.
Action Types
Actions are prioritized according to their types in the following table, which is ordered to reflect the order actions are executed:
wafFlags
(optional), alert
(optional)If alert is specified, an Actions Center notification is sent if the rule is triggered 10 times in a 5-minute window. Once an alert is triggered for a particular rule, it will not fire off again until the next day (UTC).
status, wafFlags
(optional and mutually exclusive), alert
(optional)If alert is specified, an Actions Center notification is sent if the rule is triggered 10 times in a 5-minute window. Once an alert is triggered for a particular rule, it will not fire off again until the next day (UTC).
wafFlags
(optional), alert
(optional)If alert is specified, an Actions Center notification is sent if the rule is triggered 10 times in a 5-minute window. Once an alert is triggered for a particular rule, it will not fire off again until the next day (UTC).
WAF Flags List waf-flags-list
The wafFlags
property, which can be used in the licensable WAF traffic filter rules, may reference the following:
/bin/
CMDEXE
while disabling false-positive on /bin
due to AEM architecture./foo/./bar
is normalized to /foo/bar
).htaccess
file, or a configuration file which could leak sensitive informationConsiderations considerations
-
When two conflicting rules are created, the allowed rules always take precedence over the block rules. For example, if you create a rule to block a specific path and a rule to allow one specific IP address, requests from that IP address on the blocked path is allowed.
-
If a rule is matched and blocked, the CDN responds with a
406
return code. -
The configuration files should not contain secrets since they would be readable by anyone who has access to the git repository.
-
IP allowlists defined in Cloud Manager take precedence over Traffic Filters Rules.
-
WAF rule matches only appear in CDN logs for CDN misses and passes, not hits.
Rules Examples examples
Some rule examples follow. See the rate limit section further down for examples of rate limit rules.
Example 1
This rule blocks requests coming from IP 192.168.1.1:
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: "block-request-from-ip"
when: { reqProperty: clientIp, equals: "192.168.1.1" }
action:
type: block
Example 2
This rule blocks request on path /helloworld
on publish with a User-Agent that contains Chrome:
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: "block-request-from-chrome-on-path-helloworld-for-publish-tier"
when:
allOf:
- { reqProperty: path, equals: /helloworld }
- { reqProperty: tier, equals: publish }
- { reqHeader: user-agent, matches: '.*Chrome.*' }
action:
type: block
Example 3
This rule blocks requests on publish that contain the query parameter foo
, but allows every request coming from IP 192.168.1.1:
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: "block-request-that-contains-query-parameter-foo"
when:
allOf:
- { queryParam: url-param, equals: foo }
- { reqProperty: tier, equals: publish }
action:
type: block
- name: "allow-all-requests-from-ip"
when: { reqProperty: clientIp, equals: 192.168.1.1 }
action:
type: allow
Example 4
This rule blocks requests to path /block-me
on publish, and blocks every request that matches a SQLI
or XSS
pattern. This example includes a WAF traffic filter rules, which references the SQLI
and XSS
WAF Flags, and thus requires a separate license.
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: "path-rule"
when:
allOf:
- { reqProperty: path, equals: /block-me }
- { reqProperty: tier, equals: publish }
action:
type: block
- name: "Enable-SQL-Injection-and-XSS-waf-rules-globally"
when: { reqProperty: path, like: "*" }
action:
type: block
wafFlags: [ SQLI, XSS]
Example 5
This rule blocks access to OFAC countries:
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: block-ofac-countries
when:
allOf:
- reqProperty: tier
matches: "author|publish"
- reqProperty: clientCountry
in:
- SY
- BY
- MM
- KP
- IQ
- CD
- SD
- IR
- LR
- ZW
- CU
- CI
action: block
Rate Limit Rules
Sometimes it is desirable to block traffic if it exceeds a certain rate of incoming requests, based on a specific condition. Setting a value for the rateLimit
property limits the rate of those requests that match the rule condition.
Rate limit rules cannot reference WAF flags. They are available to all Sites and Forms customers.
Rate limits are calculated per CDN POP. As an example, assume that POPs in Montreal, Miami, and Dublin experience traffic rates of 80, 90, and 120 requests per second respectively. And, the rate limit rule is set to a limit of 100. In that case, only the traffic to Dublin would be rate limited.
Rate limits are evaluated based on either traffic hitting the edge, traffic hitting the origin, or the number of errors.
rateLimit Structure ratelimit-structure
Examples ratelimiting-examples
Example 1
This rule blocks a client for 5 milliseconds when it exceeds an average of 60 req/sec (per CDN POP) in the last 10 sec:
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: limit-requests-client-ip
when:
reqProperty: tier
matches: "author|publish"
rateLimit:
limit: 60
window: 10
penalty: 300
count: all
groupBy:
- reqProperty: clientIp
action: block
Example 2
Block requests on path /critical/resource for 60 seconds when it exceeds an average of 100 requests to origin per second (per CDN POP) on a time window of ten seconds:
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: rate-limit-example
when:
allOf:
- { reqProperty: path, equals: /critical/resource }
- { reqProperty: tier, equals: publish }
action:
type: block
rateLimit: { limit: 100, window: 10, penalty: 60, count: fetches }
CVE Rules cve-rules
If WAF is licensed, Adobe automatically applies blocking rules to protect against many known CVEs (Common Vulnerabilities and Exposures) and new CVEs may be added soon after being discovered. Customers should not and need not configure CVE rules themselves.
If a traffic request matches a CVE, it will appear in the corresponding CDN log entry.
Please contact Adobe support if there are questions about a particular CVE or if there is a particular CVE rule that your organization would like to disable.
Traffic Filter Rules Alerts traffic-filter-rules-alerts
A rule can be configured to send an Actions Center notification if it is triggered ten times within a 5-minute window. Such a rule alerts you when certain traffic patterns occur so that you can take any necessary measures. Once an alert is triggered for a particular rule, it will not fire off again until the next day (UTC).
Learn more about Actions Center, including how to set up the required Notification Profiles to receive emails.
The alert property can be applied to the action node for all action types (allow, block, log).
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: "path-rule"
when:
allOf:
- { reqProperty: path, equals: /block-me }
- { reqProperty: tier, equals: publish }
action:
type: block
alert: true
Default Traffic Spike at Origin Alert traffic-spike-at-origin-alert
An Actions Center email notification will be sent when there is a significant amount of traffic sent to the origin, where a high threshold of requests are coming from the same IP address, thus suggestive of a DDoS attack.
If this theshold is met, Adobe will block traffic from that IP address, but it is recommended to take additional measures to protect your origin, including configuring rate limit traffic filter rules to block traffic spikes at lower thresholds. See the Blocking DoS and DDoS attacks using traffic rules tutorial for a guided walk-through.
This alert is enabled by default, but it can be disabled using the defaultTrafficAlerts property, set to false. Once the alert is triggered, it will not fire off again until the next day (UTC).
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
defaultTrafficAlerts: false
CDN Logs cdn-logs
AEM as a Cloud Service provides access to CDN logs, which are useful for use cases including cache hit ratio optimization, and configuring traffic filter rules. CDN logs appear in the Cloud Manager Download Logs dialog, when selecting the Author or Publish service.
CDN logs may be delayed up to five minutes.
The rules
property describes what traffic filter rules are matched, and has the following pattern:
"rules": "match=<matching-customer-named-rules-that-are-matched>,waf=<matching-WAF-rules>,action=<action_type>"
For example:
"rules": "match=Block-Traffic-under-private-folder,Enable-SQL-injection-everywhere,waf="SQLI,SANS",action=block"
The rules behave in the following manner:
- The customer-declared rule name of any matching rules is listed in the
match
attribute. - The
action
attribute determines whether the rules block, allow, or log. - If the WAF is licensed and enabled, the
waf
attribute lists any WAF flags (for example, SQLI) that were detected. This is true regardless of whether the WAF flags were listed in any rules. This is to provide insight into potential new rules to declare. - If no customer-declared rules match and no waf rules match, the
rules
property is blank.
As noted earlier, WAF rule matches only appear in CDN logs for CDN misses and passes, not hits.
The example below shows a sample cdn.yaml
and two CDN log entries:
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev"]
data:
trafficFilters:
rules:
- name: "path-rule"
when: { reqProperty: path, equals: /block-me }
action: block
- name: "Enable-SQL-Injection-and-XSS-waf-rules-globally"
when: { reqProperty: path, like: "*" }
action:
type: block
wafFlags: [ SQLI, XSS ]
{
"timestamp": "2023-05-26T09:20:01+0000",
"ttfb": 19,
"cli_ip": "147.160.230.112",
"cli_country": "CH",
"rid": "974e67f6",
"req_ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",
"host": "example.com",
"url": "/block-me",
"method": "GET",
"res_ctype": "",
"cache": "PASS",
"status": 406,
"res_age": 0,
"pop": "PAR",
"rules": "match=path-rule,action=blocked"
}
{
"timestamp": "2023-05-26T09:20:01+0000",
"ttfb": 19,
"cli_ip": "147.160.230.112",
"cli_country": "CH",
"req_ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",
"rid": "974e67f6",
"host": "example.com",
"url": "/?sqli=%27%29%20UNION%20ALL%20SELECT%20NULL%2CNULL%2CNULL%2CNULL%2CNULL%2CNULL%2CNULL%2CNULL%2CNULL%2CNULL--%20fAPK",
"method": "GET",
"res_ctype": "image/png",
"cache": "PASS",
"status": 406,
"res_age": 0,
"pop": "PAR",
"rules": "match=Enable-SQL-Injection-and-XSS-waf-rules-globally,waf=SQLI,action=blocked"
}
Log Format cdn-log-format
Below is a list of the field names used in CDN logs, along with a brief description.
Also indicates if the match resulted in a block.
For example, “
match=Enable-SQL-Injection-and-XSS-waf-rules-globally,waf=SQLI,action=blocked
”Empty if no rules matched.
Dashboard Tooling dashboard-tooling
Adobe provides a mechanism to download dashboard tooling onto your computer to ingest CDN logs downloaded via Cloud Manager. With this tooling, you can analyze your traffic to help come up with the appropriate traffic filter rules to declare, including WAF rules.
Dashboard tooling can be cloned directly from the AEMCS-CDN-Log-Analysis-Tooling GitHub repository.
Tutorials are available for concrete instructions on how to use the dashboard tooling.
Recommended starter rules recommended-starter-rules
You can copy the recommended rules below into your cdn.yaml
to get started. Start in log mode, analyze your traffic, and when satisfied, change to block mode. You may want to modify the rules based on the unique characteristics of your website’s live traffic.
kind: "CDN"
version: "1"
metadata:
envTypes: ["dev", "stage", "prod"]
data:
trafficFilters:
rules:
# Block client for 5m when it exceeds an average of 100 req/sec to origin on a time window of 10sec
- name: limit-origin-requests-client-ip
when:
reqProperty: tier
equals: 'publish'
rateLimit:
limit: 100
window: 10
count: fetches
penalty: 300
groupBy:
- reqProperty: clientIp
action: log
# Block client for 5m when it exceeds an average of 500 req/sec on a time window of 10sec
- name: limit-requests-client-ip
when:
reqProperty: tier
equals: 'publish'
rateLimit:
limit: 500
window: 10
count: all
penalty: 300
groupBy:
- reqProperty: clientIp
action: log
# Block requests coming from OFAC countries
- name: block-ofac-countries
when:
allOf:
- { reqProperty: tier, in: ["author", "publish"] }
- reqProperty: clientCountry
in:
- SY
- BY
- MM
- KP
- IQ
- CD
- SD
- IR
- LR
- ZW
- CU
- CI
action: log
# Enable recommended WAF protections (only works if WAF is licensed enabled for your environment)
- name: block-waf-flags-globally
when:
reqProperty: tier
in: ["author", "publish"]
action:
type: log
wafFlags:
- TRAVERSAL
- CMDEXE-NO-BIN
- XSS
- LOG4J-JNDI
- BACKDOOR
- USERAGENT
- SQLI
- SANS
- TORNODE
- NOUA
- SCANNER
- PRIVATEFILE
- NULLBYTE
Tutorials tutorial
Two tutorials are available.
Protecting websites with traffic filter rules (including WAF rules) tutorial-protecting-websites
Work through a tutorial to gain general, practical knowledge and experience around traffic filter rules, including WAF rules.
The tutorial walks you through:
- Setting up the Cloud Manager config pipeline
- Using tools to simulate malicious traffic
- Declaring traffic filter rules, including WAF rules
- Analyzing results with dashboard tooling
- Best practices
Blocking DoS and DDoS attacks using traffic filter rules tutorial-blocking-DDoS-with-rules
Deep-dive on how to block Denial of Service (DoS) and Distributed Denial of Service (DDoS) attacks using rate limit traffic filter rules and other strategies.
The tutorial walks you through:
- understanding protection
- receiving alerts when rate limits are exceeded
- analyzing traffic patterns using dashboard tooling to configure thresholds for rate limit traffic filter rules