Best practices for SEO and indexing when migrating public PDFs from AEM Sites DAM to AEM Assets delivery URLs
This article describes the recommended approach for migrating public PDF assets from AEM Sites DAM URLs to AEM Assets delivery URLs while preserving search engine indexing, SEO signals, and the relationship between referring webpages and PDFs.
Description description
Environment
Adobe Experience Manager (AEM) as a Cloud Service (all versions)
Issue/Symptoms
Organizations using Adobe Experience Manager (AEM) as a Cloud Service may need to migrate public PDF assets from their existing AEM Sites DAM URLs to AEM Assets delivery URLs. This migration can impact search engine indexing and SEO if not handled carefully.
- Existing public PDFs are already indexed by search engines under the original domain.
- Migrated PDFs delivered from the default Adobe delivery domain are not indexed due to restrictive
robots.txtand response headers. - The relationship between the referring webpage and the PDF is disrupted if the PDF moves to a new domain.
Cause
The default Adobe delivery domain uses a robots.txt file and response headers that prevent search engine crawling and indexing. Migrating PDFs without proper redirects and domain configuration can result in loss of SEO signals and discoverability.
Resolution resolution
Follow the steps to resolve the issue:
-
Assess your current PDF delivery and indexing:
- Identify all public PDFs currently delivered from your AEM Sites/local DAM domain.
- Determine which PDFs are already indexed by search engines and referenced from public webpages.
-
Plan and configure a custom delivery domain:
- Do not use the default Adobe delivery domain for SEO-important assets, as its default
robots.txtblocks search engine indexing. - Configure a customer-owned custom domain for the Dynamic Media Delivery tier to serve public, indexable PDFs. See Adobe documentation for configuring a custom domain for the delivery tier.
- Do not use the default Adobe delivery domain for SEO-important assets, as its default
-
Implement 301 redirects for migrated assets:
- For PDFs already indexed at the original domain, implement HTTP
301(permanent) redirects from the old URL to the new custom delivery domain URL. - This preserves SEO signals and search engine rankings for migrated assets.
- For PDFs already indexed at the original domain, implement HTTP
-
Validate
robots.txtand response headers:- Ensure the custom delivery domain’s
/robots.txtallows crawling of public PDFs. - Verify that PDF responses do not include
X-Robots-Tag: noindex, nofollowheaders unless intentional. - Confirm that anonymous access to the PDF returns HTTP
200andContent-Type: application/pdf.
- Ensure the custom delivery domain’s
-
Update metadata and sitemap references:
- Update referring webpages to link to the new PDF URLs on the custom domain.
- Include the new PDF URLs in your website’s sitemap for discoverability.
- Ensure PDF metadata (title, description, author, etc.) is accurate and descriptive for search engines and LLMs.
-
Preserve attribution and semantic relationships:
- Where possible, maintain attribution to your website within the PDF content and metadata.
- Use canonical link headers or HTML tags on referring pages to indicate the preferred URL if duplicates exist.
-
Use an interim approach during the transition:
- While the custom delivery domain is being implemented, continue serving new SEO-important PDFs from the existing AEM Sites/local DAM domain.
- Manage assets centrally in AEM Assets as the source of truth, but delay public delivery from the new domain until it is fully validated.
-
Coordinate DNS and CDN updates:
- Coordinate with your DNS and CDN administrators to configure CNAME records and ensure the custom domain points to Adobe’s CDN as required.
- Complete CNAME verification and SSL certificate setup in AEM Cloud Manager.
-
Verification:
- Test access to a migrated PDF through the new custom domain.
- Confirm that the PDF is accessible, returns the correct headers, and is discoverable by search engines.