Prepare Assets for smart tagging

Before you can start tagging your assets using Smart Content Services, integrate Experience Manager Assets with Adobe Developer Console to use smart service of Adobe Sensei. Once configured train the service using a few images and a tag.

Before you use the Smart Content Service, ensure the following:

• Integrate with Adobe Developer Console.

• Train the Smart Content Service.

• Install the latest Experience Manager Service Pack.

Integrate with Adobe Developer Console

When you integrate with Adobe Developer Console, the Experience Manager server authenticates your service credentials with the Adobe Developer Console gateway before forwarding your request to the Smart Content Service. To integrate, you need an Adobe ID account that has administrator privileges for the organization and Smart Content Service license purchased and enabled for your organization.

To configure the Smart Content Service, follow these top-level steps:

1. To generate a public key, Create a Smart Content Service configuration in Experience Manager. Obtain public certificate for OAuth integration.

2. Create an integration in Adobe Developer Console and upload the generated public key.

3. Configure your deployment using the API key and other credentials from Adobe Developer Console.

4. Test the configuration.

5. Optionally, enable auto-tagging on asset upload.

Obtain public certificate by creating Smart Content Service configuration

A public certificate lets you authenticate your profile on Adobe Developer Console.

1. In the Experience Manager user interface, access Tools > Cloud Services > Legacy Cloud Services.

2. In the Cloud Services page, click Configure Now under Assets Smart Tags.

3. In the Create Configuration dialog, specify a title and name for the Smart Tags configuration. Click Create.

4. In the AEM Smart Content Service dialog, use the following values:

   Service URL: https://smartcontent.adobe.io/<region where your Experience Manager author instance is hosted>

   For example, https://smartcontent.adobe.io/apac. You can specify na, emea, or, apac as the regions where your Experience Manager author instance is hosted.

   NOTE

   If the Experience Manager Managed Service is provisioned before September 01, 2022, use the following Service URL:
   https://mc.adobe.io/marketingcloud/smartcontent

   Authorization Server: https://ims-na1.adobelogin.com

   Leave the other fields blank for now (to be provided later). Click OK.

   

   Figure: Smart Content Service dialog to provide content service URL

   NOTE

   The URL provided as Service URL is not accessible via browser and generates a 404 error. The configuration works OK with the same value of the Service URL parameter. For the overall service status and maintenance schedule, see https://status.adobe.com.

5. Click Download Public Certificate for OAuth Integration, and download the public certificate file AEM-SmartTags.crt.

   

   Figure: Settings for smart tagging service.

Reconfigure when a certificate expires

After a certificate expires, it is no longer trusted. You cannot renew an expired certificate. To add a certificate, follow these steps.

1. Log in your Experience Manager deployment as an administrator. Click Tools > Security > Users.

2. Locate and click dam-update-service user. Click Keystore tab.

3. Delete the existing similaritysearch keystore with the expired certificate. Click Save & Close.

   

   Figure: Delete the existing similaritysearch entry in Keystore to add a security certificate.

4. Navigate to Tools > Cloud Services > Legacy Cloud Services. Click Asset Smart Tags > Show Configuration > Available Configurations. Click the required configuration.

5. To download a public certificate, click Download Public Certificate for OAuth Integration.

6. Access https://console.adobe.io and navigate to the existing Smart Content Services on the Integrations page. Upload the new certificate. For more information, see the instructions in Create Adobe Developer Console integration.

Create Adobe Developer Console integration

To use Smart Content Service APIs, create an integration in Adobe Developer Console to obtain API Key (generated in CLIENT ID field of Adobe Developer Console integration), TECHNICAL ACCOUNT ID, ORGANIZATION ID, and CLIENT SECRET for Assets Smart Tagging Service Settings of cloud configuration in Experience Manager.

1. Access https://console.adobe.io in a browser. Select the appropriate account and verify that the associated organization role is system administrator.

2. Create a project with any desired name. Click Add API.

3. On the Add an API page, select Experience Cloud and select Smart Content. Click Next.

4. Select Upload your public key. Provide the certificate file downloaded from Experience Manager. A message Public key(s) uploaded successfully is displayed. Click Next.

   Create a new Service Account (JWT) credential page displays the public key for the service account.

5. Click Next.

6. On the Select product profiles page, select Smart Content Services. Click Save configured API.

   A page displays more information about the configuration. Keep this page open to copy and add these values in Assets Smart Tagging Service Settings of cloud configuration in Experience Manager to configure smart tags.

   

   Figure: Details of integration in Adobe Developer Console

Configure Smart Content Service

To configure the integration, use the values of TECHNICAL ACCOUNT ID, ORGANIZATION ID, CLIENT SECRET, and CLIENT ID fields from the Adobe Developer Console integration. Creating a Smart Tags cloud configuration allows authentication of API requests from the Experience Manager deployment.

1. In Experience Manager, navigate to Tools > Cloud Service > Legacy Cloud Services to open the Cloud Services console.

2. Under the Assets Smart Tags, open the configuration created above. On the service settings page, click Edit.

3. In the AEM Smart Content Service dialog, use the pre-populated values for the Service URL and Authorization Server fields.

4. For the fields Api Key, Technical Account ID, Organization ID, and Client Secret, copy and use the following values generated in Adobe Developer Console integration.

   Assets Smart Tagging Service Settings	Adobe Developer Console integration fields
   Api Key	CLIENT ID
   Technical Account ID	TECHNICAL ACCOUNT ID
   Organization ID	ORGANIZATION ID
   Client Secret	CLIENT SECRET

Validate the configuration

After you have completed the configuration, you can use a JMX MBean to validate the configuration. To validate, follow these steps.

1. Access your Experience Manager server at https://[aem_server]:[port].

2. Go to Tools > Operations > Web Console to open the OSGi console. Click Main > JMX.

3. Click com.day.cq.dam.similaritysearch.internal.impl. It opens SimilaritySearch Miscellaneous Tasks.

4. Click validateConfigs(). In the Validate Configurations dialog, click Invoke.

The validation results are displayed in the same dialog.

Enable smart tagging in the DAM Update Asset workflow (Optional)

1. In Experience Manager, go to Tools > Workflow > Models.

2. On Workflow Models page, select the DAM Update Asset workflow model.

3. Click Edit from the toolbar.

4. Expand the Side Panel to display the steps. Drag Smart Tag Asset step that is available in the DAM Workflow section and place it after the Process Thumbnails step.

   

   Figure: Add smart tag asset step after the process thumbnail step in the DAM Update Asset workflow.

5. Open the step in edit mode. Under Advanced Settings, ensure that the Handler Advance option is selected.

   

   Figure: Configure DAM Update Asset workflow and add smart tag step

6. In the Arguments tab, select Ignore Errors if you want the workflow to complete even if the automatic tagging step fails.

   

   Figure: Configure DAM Update Asset workflow to add smart tag step and select handler advance

   To tag assets when they are uploaded irrespective of whether smart tagging is enabled on folders, select Ignore Smart Tag Flag.

   

   Figure: Configure DAM Update Asset workflow to add smart tag step and select ignore Smart Tag flag.

7. Click OK to close the process step, and then save the workflow.

Train the Smart Content Service

For the Smart Content Service to recognize your business taxonomy, run it on a set of assets that already include tags that are relevant to your business. To effectively tag your brand images, the Smart Content Service requires that the training images conform to certain guidelines. After training, the service can apply the same taxonomy on a similar set of assets.

You can train the service multiple times to improve its ability to apply relevant tags. After each training cycle, run a tagging workflow and check whether your assets are tagged appropriately.

You can train the Smart Content Service periodically or on requirement basis.

Guidelines for training

For best results, images in your training set conform to the following guidelines:

Quantity and size: Minimum 30 images per tag. Minimum of 500 pixels on the longer side.

Coherence: Images used for a specific tag are visually similar.

For example, it is not a good idea to tag all of these images as my-party (for training) because they are not visually similar.

Coverage: Use sufficient variety in the images in the training. The idea is to supply a few but reasonably diverse examples so that Experience Manager learns to focus on the right things. If you’re applying the same tag on visually dissimilar images, include at least five examples of each kind.

For example, for the tag model-down-pose, include more training images similar to the highlighted image below for the service to identify similar images more accurately during tagging.

Distraction/obstruction: The service trains better on images that have less distraction (prominent backgrounds, unrelated accompaniments, such as objects/persons with the main subject).

For example, for the tag casual-shoe, the second image is not a good training candidate.

Completeness: If an image qualifies for more than one tag, add all applicable tags before including the image for training. For example, for tags, such as raincoat and model-side-view, add both the tags on the eligible asset before including it for training.

Periodic training

You can enable the Smart Content Service to train periodically on the assets and associated tags within a folder. Open the Properties page of your asset folder, select Enable Smart Tags under the Details tab, and save the changes.

Once this option is selected for a folder, Experience Manager runs a training workflow automatically to train the Smart Content Service on the folder assets and their tags. By default, the training workflow runs on a weekly basis at 12:30 AM on Saturdays.

On-demand training

You can train the Smart Content Service whenever required from the Workflow console.

1. In Experience Manager interface, go to Tools > Workflow > Models.

2. From the Workflow Models page, select the Smart Tags Training workflow and then click Start Workflow from the toolbar.

3. In the Run Workflow dialog, browse to the payload folder that includes the tagged assets for training the service.

4. Specify a title for the workflow and add a comment. Then, click Run. The assets and tags are submitted for training.

   

View training reports

To check whether the Smart Content Service is trained on your tags in the training set of assets, review the training workflow report from the Reports console.

1. In Experience Manager interface, go to Tools > Assets > Reports.

2. In the Asset Reports page, click Create.

3. Select the Smart Tags Training report, and then click Next from the toolbar.

4. Specify a title and description for the report. Under Schedule Report, leave the Now option selected. If you want to schedule the report for later, select Later and specify a date and time. Then, click Create from the toolbar.

5. In the Asset Reports page, select the report you generated. To view the report, click View from the toolbar.

6. Review the details of the report.

   The report displays the training status for the tags you trained. The green color in the Training Status column indicates that the Smart Content Service is trained for the tag. Yellow color indicates that the service is not completely trained for a particular tag. In this case, add more images with the particular tag and run the training workflow to train the service completely on the tag.

   If you do not see your tags in this report, run the training workflow again for these tags.

7. To download the report, select it from the list, and click Download from the toolbar. The report downloads as a Microsoft Excel spreadsheet.

Limitations

• Enhanced smart tags are based on learning models of images and their tags. These models are not always perfect at identifying tags. The current version of the Smart Content Service has the following limitations:

  • Inability to recognize subtle differences in images. For example, slim versus regular fitted shirts.
  • Inability to identify tags based on tiny patterns/parts of an image. For example, logos on T-shirts.
  • Tagging is supported in the locales that Experience Manager is supported in.

• To search for assets with smart tags (regular or enhanced), use the Assets Omnisearch (full-text search). There is no separate search predicate for smart tags.