Documentation AEM as a Cloud Service User Guide

Understand Cloud Service content requests

Last update: Fri Oct 04 2024 00:00:00 GMT+0000 (Coordinated Universal Time)

CREATED FOR:

Admin
Developer

Introduction introduction

Content requests refer to requests made to AEM Sites, including those requests related to Edge Delivery Services or customer-provided caching systems like a Content Delivery Network. These requests deliver content or data in HTML format through page views (for example, pages and Experience Fragments) or in JSON format through API calls in a headless manner. Content requests are counted either as a page view or five API calls, and are measured at the ingress of the first caching system to receive a content request. Certain HTTP requests are included or excluded for purposes of counting content requests. The full list of such included and excluded HTTP requests, as well as their technical definitions, are available in the documentation.

About Cloud Service content requests understanding-cloud-service-content-requests

For customers using the out-of-the-box CDN, Cloud Service content requests are measured via server-side collection of data. This collection is enabled via CDN log analysis. AEM (Adobe Experience Manager) as a Cloud Service automatically collects content requests server-side at the edge. It analyzes log files generated by the AEM as a Cloud Service CDN. This process is done by isolating the requests returning HTML (text/html) or JSON (application/json) content from the CDN, and is based on several inclusion and exclusion rules detailed below. A content request occurs regardless of whether the content is served from the CDN caches or returned to the CDN origin (AEM’s dispatchers).

Variances of Cloud Service content requests content-requests-variances

Content requests can have variances within an organization’s Analytics reporting tools as summarized in the following table. In general, avoid using analytics tools that rely on client-side instrumentation to report the number of content requests for a site. These tools often miss a large portion of traffic because they depend on user consent to be activated. Analytics tools gathering data server-side in log files, or CDN reports for customers adding their own CDN on top of AEM as a Cloud Service, provide better counts.

Reason For Variance

Explanation

End user consent

Analytics tools relying on client-side instrumentation often depend on user consent to be triggered. This workflow could represent the majority of the traffic not being tracked. For customers who want to measure content requests on their own, it is recommended to rely on analytics tools gathering data server-side or CDN reports.

Tagging

All pages or API calls that are tracked as Adobe Experience Manager content requests may not be tagged with Analytics tracking.

Tag Management Rules

Tag management rule settings may result in various data collection configurations on a page, resulting in some combination of discrepancies with content request tracking.

Bots

Unknown bots that AEM has not pre-identified and removed may cause tracking discrepancies.

Report Suites

Pages that are part of the same AEM instance and domain may send data to different Analytics report suites.

Third-Party Monitoring and Security Tools

Monitoring and security scanning tools may generate content requests for AEM that are not tracked in Analytics reports.

API Access

Programmatic access to pages or to Adobe Experience Manager APIs may generate content requests for AEM that are not tracked in Analytics reports.

Prefetch Requests

Using a prefetch service to pre-load pages to increase speed can cause significant content request traffic increases.

DDOS

While Adobe makes attempts to detect and filter out traffic automatically from DDOS attacks, there is no guarantee that all possible DDOS attacks are detected.

Traffic Blockers

Using a tracker blocker in a browser may opt out some requests from being tracked.

Firewalls

Firewalls may block Analytics tracking. This scenario is more frequent with corporate firewalls.

Server-side collection rules serverside-collection

There are rules in place to exclude well-known bots, including well-known services visiting the site regularly to refresh their search index or service.

Types of included content requests included-content-requests

Request Type

Content Request

Description

HTTP Code 100-299

Included

Regular requests that deliver all or partial content.

HTTP libraries for automation

Included

Examples:
• Amazon CloudFront
• Apache Http Client
• Asynchronous HTTP Client
• Axios
• Azureus
• Curl
• GitHub Node Fetch
• Guzzle
• Go-http-client
• Headless Chrome
• Java™ Client
• Jersey
• Node Oembed
• okhttp
• Python Requests
• Reactor Netty
• Wget
• WinHTTP
• Fast HTTP
• GitHub Node Fetch
• Reactor Netty

Monitoring and Health Check tools

Included

Set up by the customer to monitor a certain aspect of the site. For example, availability or real-world user performance. If they are targeting specific endpoints like /system/probes/health for health checks, Adobe recommends that you use /system/probes/health endpoint and not the actual HTML pages from the site. See below
Examples:
• Amazon-Route53-Health-Check-Service
• EyeMonIT_bot_version_0.1_(https://eyemonit.com/)
• Investis-Site24x7
• Mozilla/5.0+(compatible; UptimeRobot/2.0; https://uptimerobot.com/)
• ThousandEyes-Dragonfly-x1
• OmtrBot/1.0
• WebMon/2.0.0

<link rel="prefetch"> requests

Included

To increase the speed of loading the next page, customers can have the browser load a set of pages before the user clicks the link—so they are already in the cache. Mind: This approach increases the traffic significantly—depending on how many of these pages are prefetched.

Traffic that blocks Adobe Analytics or Google Analytics reporting

Included

It is more common that visitors of sites have privacy software installed (Ad-blockers, and so on) that impact the accuracy of Google Analytics or Adobe Analytics. AEM as a Cloud Service counts requests on the first entry-point into the Adobe operated infrastructure and not the client-side.

Types of excluded content requests excluded-content-request

Request Type

Content Request

Description

HTTP Code 500+

Excluded

Errors returned to the visitor when something goes wrong on AEM as a Cloud Service or the customer custom code.

HTTP Code 400-499

Excluded

Errors returned to the visitor when the content does not exist (404) or there are other content or request-related issues.

HTTP Code 300-399

Excluded

Good requests that either check if something has changed on the server, or redirect the request to another resource. They do not contain content itself, therefore they are not billable.

Requests going to /libs/*

Excluded

AEM internal JSON requests, such as the CSRF token that is not billable.

Traffic from DDOS attacks

Excluded

DDOS protection. AEM does auto-detect some of the DDOS attacks and blocks them. DDOS attacks if detected are not billable.

AEM as a Cloud Service New Relic Monitoring

Excluded

AEM as a Cloud Service global monitoring.

URL for customers to monitor their Cloud Service program

Excluded

Adobe recommends that you use the URL to monitor the availability or health check externally.

/system/probes/health

AEM as a Cloud Service Pod Warm-up Service

Excluded

Agent: skyline-service-warmup/1.*

Well-known search engines, social networks, and HTTP libraries (tagged by Fastly)

Excluded

Well-known services visiting the site regularly to refresh their search index or service:

Examples:
• AddSearchBot
• AhrefsBot
• Applebot
• Ask Jeeves Corporate Spider
• Bingbot
• BingPreview
• BLEXBot
• BuiltWith
• Bytespider
• CrawlerKengo
• Facebookexternalhit
• Google AdsBot
• Google AdsBot Mobile
• Googlebot
• Googlebot Mobile
• lmspider
• LucidWorks
• MJ12bot
• Pinterest
• SemrushBot
• SiteImprove
• StashBot
• StatusCake
• YandexBot
• Claudebot

Exclude Commerce Integration Framework calls

Excluded

Requests made to AEM that gets forwarded to the Commerce Integration Framework—the URL starts with /api/graphql—to avoid double counting, they are not billable for Cloud Service.

Exclude manifest.json

Excluded

Manifest is not an API call. It is here to provide information on how to install web sites on a desktop or mobile phone. Adobe should not count JSON request to /etc.clientlibs/*/manifest.json

Exclude favicon.ico

Excluded

Although the returned content should not be HTML or JSON, certain scenarios like SAML authentication flows have been observed to return favicons as HTML. As a result, favicons are explicitly excluded from the count.

CDN proxy to a different backend

Excluded

Requests routed to different non-AEM backends using the CDN Origin Selectors technique are excluded as they do not hit AEM.

recommendation-more-help

fbcff2a9-b6fe-4574-b04a-21e75df764ab