Streaming ingestion fails due to an invalid schema reference

In Adobe Experience Platform, streaming ingestion can fail when using a streaming endpoint. The error states that the message cannot be validated because it doesn’t match the dataset’s schema reference. This issue occurs when the schema reference in the payload is incorrect or malformed, often due to an extra character introduced during copy-paste operations (using the copy schema payload from the UI). Correcting the schema reference resolves the validation error and allows data ingestion to proceed. To fix this, review the schemaRef.id in your payload, remove any unwanted characters, and resend the corrected request.

Description description

Environment

Adobe Experience Platform

Issue/Symptoms

  • Streaming ingestion attempts fail with validation errors.
  • The error message indicates the message doesn’t match the dataset’s schema reference.
  • Failed records show that schemaRef contains an unexpected URL-encoded double quote (%22) at the end of the schema URL.

Resolution resolution

To fix this issue, follow these steps:

  1. Review the payload sent for streaming ingestion in AEP.
  2. Inspect the header section and locate the schemaRef.id value.
  3. Remove any extraneous characters from the end of schemaRef.id, such as a URL-encoded double quote (%22).
  4. Ensure that the schemaRef.id matches the expected schema URL for your dataset without additional characters.
  5. Resend the streaming payload with the corrected header.
  6. Verify that data ingestion succeeds and no validation errors occur.

Note: This issue often occurs when copying values from other sources, which may inadvertently introduce unwanted characters into configuration fields. Always confirm that URLs and identifiers are free from encoding artifacts before using them in API requests or configuration files.

recommendation-more-help
3d58f420-19b5-47a0-a122-5c9dab55ec7f