Cloudflare (BYOCDN)
This configuration routes agentic traffic (requests from AI bots and LLM user agents) to the Edge Optimize backend service (live.edgeoptimize.net). Human visitors and SEO bots continue to be served from your origin as usual. To test the configuration, after the setup is complete, look for the header x-edgeoptimize-request-id in the response.
Prerequisites
Before setting up the Cloudflare Worker routing rules, ensure you have:
- A Cloudflare account with Workers enabled on your domain.
- Access to your domain’s DNS settings in Cloudflare.
- Completed the LLM Optimizer onboarding process.
- Completed CDN log forwarding to LLM Optimizer.
- An Edge Optimize API key retrieved from the LLM Optimizer UI.
Steps to retrieve your API key:
-
Navigate to Customer Configuration and select the CDN Configuration tab.
-
Under AI Traffic Routing to Deploy Optimizations, tick the Deploy Optimizations to AI Agents checkbox.
-
Copy the API key and proceed with the routing configuration steps below.
note note NOTE At this stage, the status may show a red cross indicating that the setup is not yet completed. This is expected — once you complete the routing configuration below and AI bot traffic starts flowing, the status will update to a green checkmark confirming that routing is successfully enabled.
Additionally, if you require any help with the above steps, reach out to your Adobe account team or llmo-at-edge@adobe.com.
How routing works
When configured correctly, a request to your domain (for example, www.example.com/page.html) from an agentic user agent is intercepted by the Cloudflare Worker and routed to the Edge Optimize backend. The backend request includes the required headers.
Testing the backend request
You can verify the routing by making a direct request to the Edge Optimize backend.
curl -svo /dev/null https://live.edgeoptimize.net/page.html \
-H 'x-forwarded-host: www.example.com' \
-H 'x-edgeoptimize-url: /page.html' \
-H 'x-edgeoptimize-api-key: $EDGE_OPTIMIZE_API_KEY' \
-H 'x-edgeoptimize-config: LLMCLIENT=TRUE;'
Required Headers
The following headers must be set on requests to the Edge Optimize backend:
x-forwarded-hostwww.example.comx-edgeoptimize-url/page.html or /products?id=123x-edgeoptimize-api-keyyour-api-key-herex-edgeoptimize-configLLMCLIENT=TRUE;Step 1: Create the Cloudflare Worker
- Log in to your Cloudflare dashboard.
- Navigate to Workers & Pages in the sidebar.
- Click Create application and then Create Worker.
- Name your worker (for example,
edge-optimize-router). - Click Deploy to create the worker with the default code.
Step 2: Add the Worker code
After creating the worker, click Edit code and replace the default code with the following:
/**
* Edge Optimize BYOCDN - Cloudflare Worker
*
* This worker routes requests from agentic bots (AI/LLM user agents) to the
* Edge Optimize backend service for optimized content delivery.
*
* Features:
* - Routes agentic bot traffic to Edge Optimize backend
* - Failover to origin on Edge Optimize errors (any 4XX or 5XX errors)
* - Loop protection to prevent infinite redirects
* - Human visitors and SEO bots are served from the origin as usual
*/
// List of agentic bot user agents to route to Edge Optimize
const AGENTIC_BOTS = [
'AdobeEdgeOptimize-AI',
'ChatGPT-User',
'GPTBot',
'OAI-SearchBot',
'PerplexityBot',
'Perplexity-User'
];
// Targeted paths for Edge Optimize routing
// Set to null to route all HTML pages, or specify an array of paths
const TARGETED_PATHS = null; // e.g., ['/', '/page.html', '/products']
// Failover configuration
// Failover on any 4XX client error or 5XX server error from Edge Optimize
const FAILOVER_ON_4XX = true; // Failover on any 4XX error (400-499)
const FAILOVER_ON_5XX = true; // Failover on any 5XX error (500-599)
export default {
async fetch(request, env, ctx) {
return await handleRequest(request, env, ctx);
},
};
async function handleRequest(request, env, ctx) {
const url = new URL(request.url);
const userAgent = request.headers.get("user-agent")?.toLowerCase() || "";
// Check if request is already processed (loop protection)
const isEdgeOptimizeRequest = !!request.headers.get("x-edgeoptimize-request");
// Construct the original path and query string
const pathAndQuery = `${url.pathname}${url.search}`;
// Check if the path matches HTML pages (no extension or .html extension)
const isHtmlPage = /(?:\/[^./]+|\.html|\/)$/.test(url.pathname);
// Check if path is in targeted paths (if specified)
const isTargetedPath = TARGETED_PATHS === null
? isHtmlPage
: (isHtmlPage && TARGETED_PATHS.includes(url.pathname));
// Check if user agent is an agentic bot
const isAgenticBot = AGENTIC_BOTS.some((ua) => userAgent.includes(ua.toLowerCase()));
// Route to Edge Optimize if:
// 1. Request is NOT already from Edge Optimize (prevents infinite loops)
// 2. User agent matches one of the agentic bots
// 3. Path is targeted for optimization
if (!isEdgeOptimizeRequest && isAgenticBot && isTargetedPath) {
// Build the Edge Optimize request URL
const edgeOptimizeURL = `https://live.edgeoptimize.net${pathAndQuery}`;
// Clone and modify headers for the Edge Optimize request
const edgeOptimizeHeaders = new Headers(request.headers);
// Remove any existing Edge Optimize headers for security
edgeOptimizeHeaders.delete("x-edgeoptimize-api-key");
edgeOptimizeHeaders.delete("x-edgeoptimize-url");
edgeOptimizeHeaders.delete("x-edgeoptimize-config");
// x-forwarded-host: The original site domain
// Use environment variable if set, otherwise use the request host
edgeOptimizeHeaders.set("x-forwarded-host", env.EDGE_OPTIMIZE_TARGET_HOST ?? url.host);
// x-edgeoptimize-api-key: Your Adobe-provided API key
edgeOptimizeHeaders.set("x-edgeoptimize-api-key", env.EDGE_OPTIMIZE_API_KEY);
// x-edgeoptimize-url: The original request URL path and query
edgeOptimizeHeaders.set("x-edgeoptimize-url", pathAndQuery);
// x-edgeoptimize-config: Configuration for cache key differentiation
edgeOptimizeHeaders.set("x-edgeoptimize-config", "LLMCLIENT=TRUE;");
try {
// Send request to Edge Optimize backend
const edgeOptimizeResponse = await fetch(new Request(edgeOptimizeURL, {
headers: edgeOptimizeHeaders,
redirect: "manual", // Preserve redirect responses from Edge Optimize
}), {
cf: {
cacheEverything: true, // Enable caching based on origin's cache-control headers
},
});
// Check if we need to failover to origin
const status = edgeOptimizeResponse.status;
const is4xxError = FAILOVER_ON_4XX && status >= 400 && status < 500;
const is5xxError = FAILOVER_ON_5XX && status >= 500 && status < 600;
if (is4xxError || is5xxError) {
console.log(`Edge Optimize returned ${status}, failing over to origin`);
return await failoverToOrigin(request, env, url);
}
// Return the Edge Optimize response
return edgeOptimizeResponse;
} catch (error) {
// Network error or timeout - failover to origin
console.log(`Edge Optimize request failed: ${error.message}, failing over to origin`);
return await failoverToOrigin(request, env, url);
}
}
// For all other requests (human users, SEO bots), pass through unchanged
return fetch(request);
}
/**
* Failover to origin server when Edge Optimize returns an error
* @param {Request} request - The original request
* @param {Object} env - Environment variables
* @param {URL} url - Parsed URL object
*/
async function failoverToOrigin(request, env, url) {
// Build origin URL
const originURL = `${url.protocol}//${env.EDGE_OPTIMIZE_TARGET_HOST}${url.pathname}${url.search}`;
// Prepare headers - clean Edge Optimize headers and add loop protection
const originHeaders = new Headers(request.headers);
originHeaders.set("Host", env.EDGE_OPTIMIZE_TARGET_HOST);
originHeaders.delete("x-edgeoptimize-api-key");
originHeaders.delete("x-edgeoptimize-url");
originHeaders.delete("x-edgeoptimize-config");
originHeaders.delete("x-forwarded-host");
originHeaders.set("x-edgeoptimize-request", "fo");
// Create and send origin request
const originRequest = new Request(originURL, {
method: request.method,
headers: originHeaders,
body: request.body,
redirect: "manual",
});
const originResponse = await fetch(originRequest);
// Add failover marker header to response
const modifiedResponse = new Response(originResponse.body, {
status: originResponse.status,
statusText: originResponse.statusText,
headers: originResponse.headers,
});
modifiedResponse.headers.set("x-edgeoptimize-fo", "1");
return modifiedResponse;
}
Click Save and deploy to publish the worker.
Step 3: Configure environment variables
Environment variables store sensitive configuration like your API key securely.
-
In your worker’s settings, navigate to Settings > Variables.
-
Under Environment Variables, click Add variable.
-
Add the following variables:
table 0-row-3 1-row-3 2-row-3 Variable Name Description Required EDGE_OPTIMIZE_API_KEYYour Adobe-provided Edge Optimize API key. Yes EDGE_OPTIMIZE_TARGET_HOSTThe target host for Edge Optimize requests (sent as x-forwarded-hostheader) and the origin domain for failover. Must be the domain only without protocol (for example,www.example.com, nothttps://www.example.com).Yes -
For the API key, click Encrypt to store it securely.
-
Click Save and deploy.
Step 4: Add a route to your domain
To activate the worker on your domain:
- Go to your worker’s Settings > Triggers.
- Under Routes, click Add route.
- Enter your domain pattern (for example,
www.example.com/*orexample.com/*). - Select your zone from the dropdown.
- Click Save.
Alternatively, you can configure routes at the zone level:
- Navigate to your domain in Cloudflare.
- Go to Workers Routes.
- Click Add route and specify the pattern and worker.
Verifying failover behavior
If Edge Optimize is unavailable or returns an error, the worker automatically fails over to your origin. Failover responses include the x-edgeoptimize-fo header:
< HTTP/2 200
< x-edgeoptimize-fo: 1
You can monitor failover events in Cloudflare Workers logs to troubleshoot issues.
Understanding the Worker logic
The Cloudflare Worker implements the following logic:
-
User Agent detection: Checks if the incoming request’s user agent matches any of the defined agentic bots (case-insensitive).
-
Path targeting: Optionally filters requests based on targeted paths. By default, all HTML pages (URLs ending with
/, no extension, or.html) are routed. You can specify specific paths using theTARGETED_PATHSarray. -
Loop protection: The
x-edgeoptimize-requestheader prevents infinite loops. When Edge Optimize makes requests back to your origin, this header is set to"1", and the worker passes the request through without routing it back to Edge Optimize. -
Header security: Before setting Edge Optimize headers, the worker removes any existing
x-edgeoptimize-*headers from the incoming request to prevent header injection attacks. -
Header mapping: The worker sets the required headers for Edge Optimize:
x-forwarded-host– Identifies the original site domain.x-edgeoptimize-url– Preserves the original request path and query string.x-edgeoptimize-api-key– Authenticates the request with Edge Optimize.x-edgeoptimize-config– Provides cache key configuration.
-
Failover logic: If Edge Optimize returns any error status code (4XX client errors or 5XX server errors) or the request fails due to a network error, the worker automatically fails over to your origin using
EDGE_OPTIMIZE_TARGET_HOST. The failover response includes thex-edgeoptimize-fo: 1header to indicate that failover occurred. -
Redirect handling: The
redirect: "manual"option ensures that redirect responses from Edge Optimize are passed through to the client without the worker following them.
Customizing the configuration
You can customize the worker behavior by modifying the configuration constants at the top of the code:
Agentic bot list
Modify the AGENTIC_BOTS array to add or remove user agents:
const AGENTIC_BOTS = [
'AdobeEdgeOptimize-AI',
'ChatGPT-User',
'GPTBot',
'OAI-SearchBot',
'PerplexityBot',
'Perplexity-User',
// Add additional user agents as needed
'ClaudeBot',
'Anthropic-AI'
];
Targeted paths
By default, all HTML pages are routed to Edge Optimize. To limit routing to specific paths, modify the TARGETED_PATHS array:
// Route all HTML pages (default)
const TARGETED_PATHS = null;
// Or specify exact paths to route
const TARGETED_PATHS = ['/', '/page.html', '/products', '/about-us'];
Failover configuration
By default, the worker fails over on any 4XX or 5XX error from Edge Optimize. Customize this behavior:
// Default: failover on any 4XX or 5XX error
const FAILOVER_ON_4XX = true;
const FAILOVER_ON_5XX = true;
// Failover only on 5XX server errors (not 4XX client errors)
const FAILOVER_ON_4XX = false;
const FAILOVER_ON_5XX = true;
// Disable automatic failover (not recommended)
const FAILOVER_ON_4XX = false;
const FAILOVER_ON_5XX = false;
Important considerations
-
Failover behavior: The worker automatically fails over to your origin if Edge Optimize returns any error (4XX or 5XX status codes) or if the request fails due to a network error. Failover uses
EDGE_OPTIMIZE_TARGET_HOSTas the origin domain (similar to Fastly’sF_Default_Originor CloudFront’sDefault_Origin). Failover responses include thex-edgeoptimize-fo: 1header, which you can use for monitoring and debugging. -
Caching: Cloudflare caches responses based on the URL by default. Because agentic traffic receives different content than human traffic, ensure your cache configuration accounts for this. Consider using Cache API or cache headers to differentiate cached content. The
x-edgeoptimize-configheader should be included in your cache key. -
Rate limiting: Monitor your Edge Optimize usage and consider implementing rate limiting for agentic traffic if needed.
-
Testing: Always test the configuration in a staging environment before deploying to production. Verify that both agentic and human traffic behave as expected. Test failover behavior by simulating Edge Optimize errors.
-
Logging: Enable Cloudflare Workers logging to monitor requests and troubleshoot issues. Navigate to Workers > your worker > Logs to view real-time logs. The worker logs failover events for debugging purposes.
Troubleshooting
x-edgeoptimize-request-id header in responseAGENTIC_BOTS array.EDGE_OPTIMIZE_API_KEY is correctly set in environment variables. Contact Adobe to confirm your API key is active.x-edgeoptimize-request header check is in place.TARGETED_PATHS is configured correctly.x-edgeoptimize-fo: 1 header in responseFAILOVER_ON_4XX and FAILOVER_ON_5XX are set to true. Check worker logs for error messages.TARGETED_PATHS (if specified) and matches the HTML page regex pattern.EDGE_OPTIMIZE_TARGET_HOST includes protocol (for example, https://).example.com, not https://example.com).Verify the setup
After completing the setup, verify that bot traffic is being routed to Edge Optimize and that human traffic remains unaffected.
1. Test bot traffic (should be optimized)
Simulate an AI bot request using an agentic user-agent:
curl -svo /dev/null https://www.example.com/page.html \
--header "user-agent: chatgpt-user"
A successful response includes the x-edgeoptimize-request-id header, confirming that the request was routed through Edge Optimize:
< HTTP/2 200
< x-edgeoptimize-request-id: 50fce12d-0519-4fc6-af78-d928785c1b85
2. Test human traffic (should NOT be affected)
Simulate a regular human browser request:
curl -svo /dev/null https://www.example.com/page.html \
--header "user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36"
The response should not contain the x-edgeoptimize-request-id header. The page content and response time should remain identical to before enabling Optimize at Edge.
3. How to differentiate between the two scenarios
x-edgeoptimize-request-idx-edgeoptimize-fo1)The status of the traffic routing can also be checked in the LLM Optimizer UI. Navigate to Customer Configuration and select the CDN Configuration tab.
To learn more about Optimize at Edge, including available opportunities, auto-optimization workflows, and FAQs, return to the Optimize at Edge overview.