Resolving SSE streaming delays for /bin/bit/chat in AEM as a Cloud Service

This article explains how to resolve delays in Server-Sent Events (SSE) streaming for the /bin/bit/chat endpoint in Adobe Experience Manager as a Cloud Service, where CDN buffering causes responses to be delivered in a single burst instead of progressively.

Description description

Environment

Adobe Experience Manager as a Cloud Service (AEMaaCS) — Dev and Stage environments, Author and Publish tiers

Symptoms

SSE/text streaming for the chatbot feature works as expected on local AEM and Author environments, but on Publish environments, the response is delayed and delivered in a single burst after a significant wait. This affects the /bin/bit/chat endpoint with Content-Type: text/event-stream. Modifying the Dispatcher configuration doesn’t resolve the issue. The problem persists even when bypassing customer-managed CDN layers, indicating the buffering occurs at the CDN layer provided by Adobe.

Cause

The issue is caused by the CDN buffering the entire SSE response for the /bin/bit/chat endpoint before releasing it to the client, due to default proxy or cache configuration. This prevents real-time streaming of events required by the chatbot feature.

Resolution resolution

Follow the steps below to resolve the issue:

  1. Review and update your Dispatcher configuration:

    • Ensure that the ProxyPass directive for /bin/bit/chat is placed inside a <Location> block, not a <LocationMatch> block, in your Dispatcher vhost configuration. For example:

    ProxyPass and other required configuration

    • Deploy the configuration change and restart the Dispatcher, if necessary.
  2. Adjust CDN configuration for SSE streaming:

    • For the path /bin/bit/chat*, ensure that response buffering is disabled. In Fastly, this can be done by enabling streaming for this path (such as setting Stream miss or Streaming Miss in the Fastly UI, or setting beresp.do_stream = true in VCL for this path).

    • Disable Edge Side Includes (ESI) processing for this path, if it is enabled.

    • Ensure the response isn’t cached by the CDN for this path (For example: by configuring the CDN to bypass cache for /bin/bit/chat).

  3. Test and validate:

    • After making the above changes, test the /bin/bit/chat endpoint in the affected environments.

    • Use a tool, such as curl, with the --no-buffer option to verify that SSE events are delivered progressively. For example:

    code language-none
    curl --http1.1 --no-buffer -N -i \
    -X POST "https:///bin/bit/chat" \
    -H "Accept: text/event-stream" \
    -H "Content-Type: application/json" \
    --data '{"action":"message","conversationId":"","userContent":""}'
    
    • Confirm that SSE events are received incrementally and not in a single burst at the end of the response.
recommendation-more-help
experience-cloud-kcs-help-kbarticles