AEM Content Transfer Tool ingestion fails due to corrupted TAR files

This article explains how to resolve ingestion failures in the AEM Content Transfer Tool caused by corrupted TAR files in the source segment store during migration from AEM on-premise to AEM as a Cloud Service.

Description description

Environment

  • Adobe Experience Manager (AEM) on-premise
  • AEM as a Cloud Service
  • Content Transfer Tool (CTT)

Issue/Symptoms

  • The ingestion process fails after several hours during migration.
  • Logs show repeated InvalidBinaryReferencesIndexException: Unrecognized magic number errors while processing segment archive files.
  • Extraction may complete with warnings about corrupted TAR files and report a FINISHED status, but ingestion fails repeatedly on different TAR files with the same error.
  • The issue is not related to network or proxy configuration.
  • The problem can occur even with small migration sets because the segment store copy phase transfers the full compacted segment store from the source environment.

Resolution resolution

Follow the steps below to resolve the issue:

  1. Run a full repository integrity check on the source segment store using the oak-run check --deep command.

  2. Review the output for any reported corruption in TAR files (for example, data02490a.tar and data69802a.tar).

  3. If corruption is detected, remediate the source repository by either:

    • Performing a journal rollback and offline compaction if a last-known good revision exists that predates all corruption.
    • Restoring from a known-good backup if no clean revision is available or if the corruption is old.
  4. After remediation, create a fresh extraction of the migration set from the repaired source environment.

  5. Ensure that the new extraction package is used for the next ingestion attempt.

  6. During ingestion, disable the Use pre-copy to speed up ingestion option.

  7. Start the ingestion and monitor for errors.

  8. If remediation is not feasible (for example, if no clean backup or revision exists), the only option is to address the source repository corruption before proceeding with migration. Migration cannot succeed while the source segment store contains corrupted TAR files.

To verify resolution, confirm that extraction completes without warnings about corrupted TAR files and that ingestion completes successfully without InvalidBinaryReferencesIndexException errors.

recommendation-more-help
experience-cloud-kcs-help-kbarticles