AEM Cloud Manager pipeline stuck during Deploy to Stage due to OSGi bundle activation conflict
This article explains how to resolve an issue where an Adobe Experience Manager (AEM) Cloud Manager pipeline becomes stuck during the Deploy to Stage phase, resulting in unavailable instances and missing-class errors due to OSGi bundle activation conflicts.
Description description
Environment
Adobe Experience Manager (AEM) as a Cloud Service (AEMaaCS) (all versions)
Symptoms
- Cloud Manager pipeline execution remains stuck in the Deploy to Stage phase for an extended period (over an hour) with no visible errors or progress.
- Affected environment(s) become unavailable after deployment.
- Attempts to access affected instance(s) result in a blank login page.
- Logs show missing-class errors, HTTP 500 or 404 responses for
/libs/granite/core/content/login.html, and health-check failures. - Restarting AEM doesn’t resolve the issue; errors persist after restart.
- The issue may only affect a subset of instances, indicating a timing or race condition.
Cause
A deployment updated a foundational shared library, triggering an OSGi framework refresh. A custom component’s activation method performed a blocking repository (JCR) login during this refresh, causing lock contention and leaving dependent bundles in a half-started state. This resulted in the pipeline stalling and affected instances becoming unavailable.
Resolution resolution
Follow the steps below to resolve the issue:
-
Identify affected environment(s):
-
Review Cloud Manager pipeline execution logs and AEM instance logs for missing-class errors and bundle activation failures.
-
Confirm which instances are unavailable after deployment.
-
-
Restore affected environment(s):
-
Use available environment backups to restore affected instances to a known good state.
-
Validate that all instances return to an operational state and the login page loads correctly.
-
-
Review custom OSGi bundle activation logic:
-
Inspect custom OSGi components, especially those with
onActivate()methods that perform repository (JCR) logins or other blocking operations during activation. -
Refactor activation logic to avoid blocking calls or repository logins during OSGi bundle activation, as these can cause deadlocks or stalls during framework refreshes.
-
Look for code patterns where repository login or long-running operations are performed directly in the activation method.
-
-
Test deployment in a lower environment:
- Before deploying to production, test the same package in a lower environment to verify that the issue doesn’t recur.
-
Monitor future deployments:
- If a similar stall occurs, collect logs and pipeline execution details for further analysis.
Verification:
- Confirm that all affected instances are operational and that the login page loads as expected after restoration and code changes.