Before you can start tagging your assets using Smart Content Services, integrate Experience Manager Assets with Adobe Developer Console to leverage smart service of Adobe Sensei. Once configured train the service using a few images and a tag.
Before you use the Smart Content Service, ensure the following:
Install the latest Experience Manager Service Pack.
When you integrate with Adobe Developer Console, the Experience Manager server authenticates your service credentials with the Adobe Developer Console gateway before forwarding your request to the Smart Content Service. To integrate, you need an Adobe ID account that has administrator privileges for the organization and Smart Content Service license purchased and enabled for your organization.
To configure the Smart Content Service, follow these top-level steps:
Create an integration in Adobe Developer Console and upload the generated public key.
Configure your deployment using the API key and other credentials from Adobe Developer Console.
Optionally, enable auto-tagging on asset upload.
A public certificate allows you to authenticate your profile on Adobe Developer Console.
In the Experience Manager user interface, access Tools > Cloud Services > Legacy Cloud Services.
In the Cloud Services page, click Configure Now under Assets Smart Tags.
In the Create Configuration dialog, specify a title and name for the Smart Tags configuration. Click Create.
In the AEM Smart Content Service dialog, use the following values:
https://smartcontent.adobe.io/<region where your Experience Manager author instance is hosted>
https://smartcontent.adobe.io/apac. You can specify
apac as the regions where your Experience Manager author instance is hosted.
If the Experience Manager Managed Service is provisioned before September 01, 2022, use the following Service URL:
Leave the other fields blank for now (to be provided later). Click OK.
Figure: Smart Content Service dialog to provide content service URL
The URL provided as Service URL is not accessible via browser and generates a 404 error. The configuration works OK with the same value of the Service URL parameter. For the overall service status and maintenance schedule, see https://status.adobe.com.
Click Download Public Certificate for OAuth Integration, and download the public certificate file
Figure: Settings for smart tagging service.
After a certificate expires, it is no longer trusted. You cannot renew an expired certificate. To add a certificate, follow these steps.
Log in your Experience Manager deployment as an administrator. Click Tools > Security > Users.
Locate and click dam-update-service user. Click Keystore tab.
Delete the existing similaritysearch keystore with the expired certificate. Click Save & Close.
Figure: Delete the existing
similaritysearch entry in Keystore to add a security certificate.
Navigate to Tools > Cloud Services > Legacy Cloud Services. Click Asset Smart Tags > Show Configuration > Available Configurations. Click the required configuration.
To download a public certificate, click Download Public Certificate for OAuth Integration.
Access https://console.adobe.io and navigate to the existing Smart Content Services on the Integrations page. Upload the new certificate. For more information, see the instructions in Create Adobe Developer Console integration.
To use Smart Content Service APIs, create an integration in Adobe Developer Console to obtain API Key (generated in CLIENT ID field of Adobe Developer Console integration), TECHNICAL ACCOUNT ID, ORGANIZATION ID, and CLIENT SECRET for Assets Smart Tagging Service Settings of cloud configuration in Experience Manager.
Access https://console.adobe.io in a browser. Select the appropriate account and verify that the associated organization role is system administrator.
Create a project with any desired name. Click Add API.
On the Add an API page, select Experience Cloud and select Smart Content. Click Next.
Select Upload your public key. Provide the certificate file downloaded from Experience Manager. A message Public key(s) uploaded successfully is displayed. Click Next.
Create a new Service Account (JWT) credential page displays the public key for the service account.
On the Select product profiles page, select Smart Content Services. Click Save configured API.
A page displays more information about the configuration. Keep this page open to copy and add these values in Assets Smart Tagging Service Settings of cloud configuration in Experience Manager to configure smart tags.
Figure: Details of integration in Adobe Developer Console
To configure the integration, use the values of TECHNICAL ACCOUNT ID, ORGANIZATION ID, CLIENT SECRET, and CLIENT ID fields from the Adobe Developer Console integration. Creating a Smart Tags cloud configuration allows authentication of API requests from the Experience Manager deployment.
In Experience Manager, navigate to Tools > Cloud Service > Legacy Cloud Services to open the Cloud Services console.
Under the Assets Smart Tags, open the configuration created above. On the service settings page, click Edit.
In the AEM Smart Content Service dialog, use the pre-populated values for the Service URL and Authorization Server fields.
For the fields Api Key, Technical Account ID, Organization ID, and Client Secret, copy and use the following values generated in Adobe Developer Console integration.
|Assets Smart Tagging Service Settings||Adobe Developer Console integration fields|
|Api Key||CLIENT ID|
|Technical Account ID||TECHNICAL ACCOUNT ID|
|Organization ID||ORGANIZATION ID|
|Client Secret||CLIENT SECRET|
After you have completed the configuration, you can use a JMX MBean to validate the configuration. To validate, follow these steps.
Access your Experience Manager server at
Go to Tools > Operations > Web Console to open the OSGi console. Click Main > JMX.
com.day.cq.dam.similaritysearch.internal.impl. It opens SimilaritySearch Miscellaneous Tasks.
validateConfigs(). In the Validate Configurations dialog, click Invoke.
The validation results are displayed in the same dialog.
In Experience Manager, go to Tools > Workflow > Models.
On Workflow Models page, select the DAM Update Asset workflow model.
Click Edit from the toolbar.
Expand the Side Panel to display the steps. Drag Smart Tag Asset step that is available in the DAM Workflow section and place it after the Process Thumbnails step.
Figure: Add smart tag asset step after the process thumbnail step in the DAM Update Asset workflow.
Open the step in edit mode. Under Advanced Settings, ensure that the Handler Advance option is selected.
Figure: Configure DAM Update Asset workflow and add smart tag step
In the Arguments tab, select Ignore Errors if you want the workflow to complete even if the automatic tagging step fails.
Figure: Configure DAM Update Asset workflow to add smart tag step and select handler advance
To tag assets when they are uploaded irrespective of whether smart tagging is enabled on folders, select Ignore Smart Tag Flag.
Figure: Configure DAM Update Asset workflow to add smart tag step and select ignore Smart Tag flag.
Click OK to close the process step, and then save the workflow.
For the Smart Content Service to recognize your business taxonomy, run it on a set of assets that already include tags that are relevant to your business. To effectively tag your brand images, the Smart Content Service requires that the training images conform to certain guidelines. After training, the service can apply the same taxonomy on a similar set of assets.
You can train the service multiple times to improve its ability to apply relevant tags. After each training cycle, run a tagging workflow and check whether your assets are tagged appropriately.
You can train the Smart Content Service periodically or on requirement basis.
The training workflow runs on folders only.
For best results, images in your training set conform to the following guidelines:
Quantity and size: Minimum 30 images per tag. Minimum of 500 pixels on the longer side.
Coherence: Images used for a specific tag are visually similar.
For example, it is not a good idea to tag all of these images as
my-party (for training) because they are not visually similar.
Coverage: Use sufficient variety in the images in the training. The idea is to supply a few but reasonably diverse examples so that Experience Manager learns to focus on the right things. If you’re applying the same tag on visually dissimilar images, include at least five examples of each kind.
For example, for the tag model-down-pose, include more training images similar to the highlighted image below for the service to identify similar images more accurately during tagging.
Distraction/obstruction: The service trains better on images that have less distraction (prominent backgrounds, unrelated accompaniments, such as objects/persons with the main subject).
For example, for the tag casual-shoe, the second image is not a good training candidate.
Completeness: If an image qualifies for more than one tag, add all applicable tags before including the image for training. For example, for tags, such as
model-side-view, add both the tags on the eligible asset before including it for training.
The ability of the Smart Content Service to train on your tags and apply them on other images depends on the quality of images you use for training. For best results, Adobe recommends that you use visually similar images to train the service for each tag.
You can enable the Smart Content Service to train periodically on the assets and associated tags within a folder. Open the Properties page of your asset folder, select Enable Smart Tags under the Details tab, and save the changes.
Once this option is selected for a folder, Experience Manager runs a training workflow automatically to train the Smart Content Service on the folder assets and their tags. By default, the training workflow runs on a weekly basis at 12:30 AM on Saturdays.
You can train the Smart Content Service whenever required from the Workflow console.
In Experience Manager interface, go to Tools > Workflow > Models.
From the Workflow Models page, select the Smart Tags Training workflow and then click Start Workflow from the toolbar.
In the Run Workflow dialog, browse to the payload folder that includes the tagged assets for training the service.
Specify a title for the workflow and add a comment. Then, click Run. The assets and tags are submitted for training.
Once the assets in a folder are processed for training, only the modified assets are processed in subsequent training cycles.
To check whether the Smart Content Service is trained on your tags in the training set of assets, review the training workflow report from the Reports console.
In Experience Manager interface, go to Tools > Assets > Reports.
In the Asset Reports page, click Create.
Select the Smart Tags Training report, and then click Next from the toolbar.
Specify a title and description for the report. Under Schedule Report, leave the Now option selected. If you want to schedule the report for later, select Later and specify a date and time. Then, click Create from the toolbar.
In the Asset Reports page, select the report you generated. To view the report, click View from the toolbar.
Review the details of the report.
The report displays the training status for the tags you trained. The green color in the Training Status column indicates that the Smart Content Service is trained for the tag. Yellow color indicates that the service is not completely trained for a particular tag. In this case, add more images with the particular tag and run the training workflow to train the service completely on the tag.
If you do not see your tags in this report, run the training workflow again for these tags.
To download the report, select it from the list, and click Download from the toolbar. The report downloads as a Microsoft Excel spreadsheet.
Enhanced smart tags are based on learning models of images and their tags. These models are not always perfect at identifying tags. The current version of the Smart Content Service has the following limitations:
To search for assets with smart tags (regular or enhanced), use the Assets Omnisearch (full-text search). There is no separate search predicate for smart tags.