Observed SegmentNotFound Exceptions in the logs. For example:
*ERROR* FelixStartLevel org.apache.sling.event org.apache.sling.event.impl.jobs.queues.QueueManager(1431) The activate method has thrown an exception (org.apache.jackrabbit.oak.plugins.segment.SegmentNotFoundException: Segment da5bcb95-d00a-4c04-a9d9-0f10f2b14e5e not found)
*ERROR* pool-6-thread-3 org.apache.sling.commons.scheduler.impl.QuartzScheduler Exception during job execution of org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate@1dc173f9 : Segment e669f30b-e886-4b7a-b161-56432601ec6b not found
org.apache.jackrabbit.oak.plugins.segment.SegmentNotFoundException: Segment e669f30b-e886-4b7a-b161-56432601ec6b not found
Environment
AEM 6.*
Cause
Due to some older issues in Oak or some inconsistencies in the repository, a segment can go missing and repository might be inconsistent.
Run consistency check on the repository and find the last good revision healthy state and revert back to it. Please do the following:
Download a version of oak-run that matches your oak core version from https://mvnrepository.com/artifact/org.apache.jackrabbit/oak-run
To revert a corrupt segment store to its latest good state change into CQ’s working directory (the one containing the crx-quickstartfolder) and backup all files in ./crx-quickstart/repository/segmentstore/.
Run the consistency check,
java -Xmx6000m -jar oak-run-*.jar check --bin=-1 /path/to/crx-quickstart/repository/segmentstore
This searches backwards through the revisions until it finds a consistent one:
Look for message like below:
main INFO o.a.j.o.p.s.f.t.ConsistencyChecker - Found latest good revision afdb922d-ba53-4a1b-aa1b-1cb044b535cf:234880
Revert the repository to this revision by editing ./crx-quickstart/repository/segmentstore/journal.log and deleting all lines after the line containing the latest good revision.
Remove all ./crx-quickstart/repository/segmentstore/*.bak files.
Run checkpoint clean-up to remove orphaned checkpoints:
java -Xmx6000m -jar oak-run-*.jar checkpoints /path/to/crx-quickstart/repository/segmentstore rm-unreferenced
Finally compact the repository:
java -Xmx6000m -jar oak-run-*.jar compact /path/to/crx-quickstart/repository/segmentstore/