Pre-Upgrade Maintenance Tasks pre-upgrade-maintenance-tasks
Before beginning your upgrade, it is important to follow these maintenance tasks to ensure that the system is ready and can be rolled back should issues occur:
- Ensure Sufficient Disk Space
- Fully Back Up AEM
- Back Up Changes to /etc
- Generate The quickstart.properties File
- Configure Workflow and Audit Log Purging
- Install, Configure, and Run The Pre-Upgrade Tasks
- Disable Custom Login Modules
- Remove Updates From The /install Directory
- Stop Any Cold Standby Instances
- Disable Custom Scheduled Jobs
- Execute Offline Revision Cleanup
- Execute Datastore Garbage Collection
- Upgrade the Database Schema If Needed
- Delete Users that Might Hinder the Upgrade
- Rotate Log Files
Ensure Sufficient Disk Space ensure-sufficient-disk-space
When executing the upgrade, in addition to the content and code upgrade activities, a repository migration will need to be performed. The migration will create a copy of the repository in the new Segment Tar format. As a result, you will need enough disk space to retain a second, potentially larger, version of your repository.
Fully Back Up AEM fully-back-up-aem
AEM should be fully backed up before beginning the upgrade. Make sure to back up your repository, application installation, datastore, and Mongo instances if applicable. For more information on backing up and restoring an AEM instance, see Backup and Restore.
Back Up Changes to /etc backup-changes-etc
The upgrade process does a good job of maintaining and merging existing content and configurations from under the /apps
and /libs
paths in the repository. For changes made to the /etc
path, including Context Hub configurations, it is often necessary to re-apply these changes after the upgrade. While the upgrade will make a backup copy of any changes that it cannot merge under /var
, we recommend backing up these changes manually before beginning the upgrade.
Generate The quickstart.properties File generate-quickstart-properties
When starting AEM from the jar file, a quickstart.properties
file will be generated under crx-quickstart/conf
. If AEM has only been started with the start script in the past, this file will not be present and the upgrade will fail. Make sure to check for the existence of this file and restart AEM from the jar file if it is not present.
Configure Workflow and Audit Log Purging configure-wf-audit-purging
The WorkflowPurgeTask
and com.day.cq.audit.impl.AuditLogMaintenanceTask
tasks require separate OSGi configurations and will not work without them. If they fail during pre-upgrade task execution, missing configurations is the most likely reason. Therefore, make sure to add OSGi configurations for these tasks or remove them altogether from the pre-upgrade optimization tasks list if you do not wish to run them. Documentation for configuring workflow purging tasks can be found at Administering Workflow Instances and audit log maintenance task configuration can be found at Audit Log Maintenance in AEM 6.
For workflow and audit log purging on CQ 5.6 as well as audit log purging on AEM 6.0, see Purge workflow and audit nodes.
Install, Configure, and Run The Pre-Upgrade Tasks install-configure-run-pre-upgrade-tasks
Because of the level of customization AEM allows, environments usually do not adhere to a uniform way of performing upgrades. This makes creating a standardized procedure for upgrades a difficult process.
In previous versions, it was also difficult for AEM upgrades that were stopped or that have failed to be safely resumed. This led to situations where restarting the full upgrade procedure was necessary or where defective upgrades were carried out without triggering any warnings.
In order to address these issues, Adobe has added several enhancements to the upgrade process, making it more resilient and user friendly. Pre-upgrade maintenance tasks that before had to be performed manually are being optimized and automated. Also, post-upgrade reports have been added so that the process can be fully scrutinized in the hope that any issues are found more easily.
Pre-upgrade maintenance tasks are currently spread over various interfaces which are partially or completely performed manually. The pre-upgrade maintenance optimization introduced in AEM 6.3 enables a unified way to trigger these tasks and be able to inspect their result on demand.
All tasks included in the pre-upgrade optimization step are compatible with all versions from AEM 6.0 onwards.
How to Set It Up how-to-set-it-up
In AEM 6.3 and later, the pre-upgrade maintenance optimization tasks come included in the quickstart jar. If you are upgrading from an older version of AEM 6, they are made available through separate packages that you can download from the Package Manager.
You can find the packages at these locations:
How to Use It how-to-use-it
The PreUpgradeTasksMBean
OSGI component comes preconfigured with a list of pre-upgrade maintenance tasks that can be run all at once. You can configure the tasks by following the below procedure:
-
Go to the Web Console by browsing to
https://serveraddress:serverport/system/console/configMgr
-
Search for “preupgradetasks”, then click on the first matching component. The full name of the component is
com.adobe.aem.upgrade.prechecks.mbean.impl.PreUpgradeTasksMBeanImpl
-
Modify the list of maintenance tasks that need to run as shown below:
The task list differs depending on the run mode that is being used to start the instance. Below is a description of the run mode each maintenance task is designed for.
DataStoreGarbageCollectionTask
is calling a Datastore Garbage Collection operation with the mark and sweep phase if used. For deployments that use a shared datastore make sure to either reconfigure it or properly or prepare the instance to avoid deletion of items referenced by another instance. This might require running the mark phase manually on all instances before triggering this pre-upgrade task.Default Configuration of the Pre-Upgrade Health Checks default-configuration-of-the-pre-upgrade-health-checks
The PreUpgradeTasksMBeanImpl
OSGI component comes pre-configured with a list of pre-upgrade health check tags to execute when the runAllPreUpgradeHealthChecks
method is called:
-
system - the tag used by the granite maintenance health checks
-
pre-upgrade - this is a custom tag that could be added to all the health checks that you can set to run before an upgrade
The list is editable. You can use the plus (+) and minus (-) buttons besides the tags to add more custom tags, or remove the default ones.
MBean Methods
The managed bean functionality can be accessed using the JMX Console.
You can access the MBeans by:
-
Going to the JMX Console at https://serveraddress:serverport/system/console/jmx
-
Search for PreUpgradeTasks and click the result
-
Select any method from the Operations section and select Invoke in the following window.
Below is a list of all the available methods that the PreUpgradeTasksMBeanImpl
exposes:
- The JMX Console
- Any external application that connects to JMX
- cURL
Disable Custom Login Modules disable-custom-login-modules
The way custom LoginModules
are configured for authentication at the repository level has fundamentally changed in Apache Oak.
In AEM versions that used CRX2 configuration was placed in the repository.xml
file, while from AEM 6 onwards it is done in the Apache Felix JAAS Configuration Factory service via the Web Console.
Therefore, any existing configurations will have to be disabled and re-created for Apache Oak after the upgrade.
To disable the custom modules defined in the JAAS configuration of repository.xml
, you need to modify the configuration to make use of default LoginModule
, as in this example:
<Security >
....
<!--
Use LoginModule authenticating against repository itself
-->
<LoginModule class = "com.day.crx.core.CRXLoginModule" >
<param name = "anonymousId" value = "anonymous" />
<param name = "adminId" value ="admin" />
<param name = "disableNTLMAuth" value = "true" />
<param name = "tokenExpiration" value = "43200000" />
<!-- param name="trust_credentials_attribute" value="d5b9167e95dad6e7d3b5d6fa8df48af8"/
-->
</LoginModule >
</ Security>
LoginModule
configuration in AEM 6, see Configuring LDAP with AEM 6.Remove Updates From The /install Directory remove-updates-install-directory
Remove any service packs, feature packs or hotfixes that have been deployed through the crx-quickstart/install
directory on the local file system. This will prevent the inadvertent installation of old hotfixes and service packs on top of the new AEM version after the update has completed.
Stop Any Cold Standby Instances stop-tarmk-coldstandby-instance
If using TarMK cold standby, stop any cold standby instances. These will guarantee an efficient way to come back online in case of issues in the upgrade. After the upgrade has completed successfully, the cold standby instances will need to be rebuilt from the upgraded primary instances.
Disable Custom Scheduled Jobs disable-custom-scheduled-jobs
Disable any OSGi scheduled jobs that are included in your application code.
Execute Offline Revision Cleanup execute-offline-revision-cleanup
If using TarMK, you should execute Offline Revision Cleanup before upgrading. This will make the repository migration step and subsequent upgrade tasks execute much faster and will help to ensure that Online Revision Cleanup can execute successfully after the upgrade has completed. For information on running Offline Revision Cleanup, see Performing Offline Revision Cleanup.
Execute Datastore Garbage Collection execute-datastore-garbage-collection
After running revision cleanup on CRX3 instances, you should run Datastore Garbage Collection to remove any unreferenced blobs in the data store. For instructions, see the documentation on Data Store Garbage Collection.
Delete Users that Might Hinder the Upgrade delete-users-that-might-hinder-the-upgrade
- You are upgrading from AEM versions older than AEM 6.3
- You encounter any of the errors mentioned below during the upgrade.
There are exceptional cases when service users might end up in an older AEM versions being improperly tagged as regular users.
If this happens, the upgrade will fail with a message like this one:
ERROR [Apache Sling Repository Startup Thread] com.adobe.granite.repository.impl.SlingRepositoryManager Exception in a SlingRepositoryInitializer, SlingRepository service registration aborted
java.lang.RuntimeException: Unable to create service user [communities-utility-reader]:java.lang.RuntimeException: Existing user communities-utility-reader is not a service user.
In order to work around this issue, make sure you do the following:
In order to work around this issue, make sure you do the following:
-
Detach the instance from production traffic
-
Create a backup of the user(s) causing the problem. You can do this via Package Manager. For more information, see How to Work with Packages.
-
Delete the user(s) causing the problem. Below is a list of users that might fall under this category:
- dynamic-media-replication
- communities-ugc-writer
- communities-utility-reader
- communities-user-admin
- oauthservice
- sling-scripting
Upgrade the Database Schema If Needed upgrade-the-database-schema-if-needed
Usually, the underlying Apache Oak stack AEM uses for persistence will take care of upgrading the database schema if needed.
However, cases might arise when the schema cannot be upgraded automatically. These are mostly high security environments where the database is running under a user with very limited priviledges. If this happens, AEM will continue to use the old schema.
In order to prevent this from happening, you need to upgrade the schema by following the below procedure:
-
Shut down the AEM instance that needs to be upgraded.
-
Upgrade the database schema. Please consult the documentation for your database type in order to see what is the tooling you need to use in order to achieve this.
For more information on how Oak handles schema upgrades, see this page on the Apache website.
-
Proceed with upgrading AEM.
Rotate Log Files rotate-log-files
We recommend archiving your current log files before beginning your upgrade. This will make it easier to monitor and scan your log files during and after the upgrade to identify and resolve any issues that may occur.