Performance tuning for S3 Datastore - Asynchronous mode



Having an external S3 Datastore.

Performing lots of intensive upload activities:

  • multiple and large assets ingestions
  • importing large index definition (generated out-of-the-band)

By default, S3 uploads are synchronous and mono-threaded, therefore huge operations can take some time to perform


The documentation specifies, however, that S3 uploads can be configured to perform asynchronously and multi-threaded (10 threads by default):

In order to greatly improve uploads time, it is possible to enable and test the asynchronous mode by adding a Java System parameter “oak.lucene.ds.async” to the startup command line:


To change the number of upload threads, you need to specify the uploadThreadsparameter in your S3Datastore.config file.

These settings need to be tested first in a lower environment, performing some benchmarks, before pushing the change to production.

On this page