Documentation

Prevent search engines from indexing AEM-managed URLs

Last update: Fri Aug 22 2025 00:00:00 GMT+0000 (Coordinated Universal Time)

URLs associated with Adobe Experience Manager (AEM) -managed domains appear in search engine results, leading to visibility of pages that should remain unindexed. To resolve the issue, configure exclusion rules and request removals via search tools.

Description description

Environment

Adobe Experience Manager 6.5 with Managed Services

Issue/Symptoms

  • AEM-managed URLs appear in Bing and other search engine results.
  • Sensitive or unwanted pages are indexed despite internal restrictions.
  • robots.txt and sitemap.xml lack proper exclusion rules.

Resolution resolution

Follow these steps to remove unwanted URLs from search engine indexing:

  1. Disallow crawling of specific URLs or directories in your robots.txt file.
  2. Exclude sensitive or unwanted pages from your sitemap.xml configuration.
  3. Request URL removal or update indexing preferences using tools like Bing Webmaster Tools.
  4. Verify server settings and ensure X-Robots-Tag headers are correctly set for excluded pages.
  5. Monitor search engine results periodically to confirm successful removal of targeted URLs.

AEM — SEO | robots.txt | sitemap.xml |Troubleshoot URL Indexing | Block Lower Environment URLs from Indexing through Server Configuration

recommendation-more-help
3d58f420-19b5-47a0-a122-5c9dab55ec7f