Prepare Assets for smart tagging

Last update: November 21, 2024

Topics:
Tagging,Smart Tags

CREATED FOR:

Admin

Before you can start tagging your assets using Smart Content Services, integrate Experience Manager Assets with Adobe Developer Console to use smart service of Adobe Sensei. Once configured, train the service using a few images and a tag.

Before you use the Smart Content Service, ensure the following:

Integrate with Adobe Developer Console.
Train the Smart Content Service.
Install the latest Experience Manager Service Pack.

SCS upgrade to support Oauth for Adobe Managed Services

New Users

Install Service Pack 22. To support Oauth integration on Service Pack 22, you need to install Hotfix for Service Pack 22.

Follow the instructions mentioned in this article to set up Smart Content Services.

Existing users

If you have upgraded to Service Pack 21, install the Hotfix for Service Pack 21 to support Oauth integration. Any existing configuration is automatically deleted. Follow the instructions mentioned in this article to set up Smart Content Services. If you upgrade to Service Pack 22, you must install this Hotfix for Service Pack 22.

For Service Pack 20 and older, you need to perform the workaround steps for SCS to support Oauth integration. See Troubleshooting smart tags for OAuth credentials.

SCS upgrade to support Oauth for On-premise users

New Users

Smart Content Services is no longer available to new Experience Manager Assets On-Premise users.

Existing users

Existing On-Premise users, who already have this capability enabled, can continue using Smart Content Services.

If you have upgraded to Service Pack 21, install the Hotfix for Service Pack 21 to support Oauth integration. Any existing configuration is automatically deleted. Follow the instructions mentioned in this article to set up Smart Content Services. If you upgrade to Service Pack 22, you must install this Hotfix for Service Pack 22.

For Service Pack 20 and older, you need to perform the workaround steps for SCS to support Oauth integration. See Troubleshooting smart tags for OAuth credentials.

Integrate with Adobe Developer Console

When you integrate with Adobe Developer Console, the Experience Manager server authenticates your service credentials with the Adobe Developer Console gateway before forwarding your request to the Smart Content Service. To integrate, you need an Adobe ID account that has administrator privileges for the organization and Smart Content Service license purchased and enabled for your organization.

To configure the Smart Content Service, follow these top-level steps:

Create an integration in Adobe Developer Console.
Create IMS technical account configuration using the API key and other credentials from Adobe Developer Console.
Configure the Smart Content Service.
Test the configuration.

Create Adobe Developer Console integration

To use Smart Content Service APIs, create an integration in Adobe Developer Console to obtain API Key (generated in CLIENT ID field of Adobe Developer Console integration), ORGANIZATION ID, and CLIENT SECRET for Assets Smart Tagging Service Settings of cloud configuration in Experience Manager.

Access https://developer.adobe.com in a browser. Select the appropriate account and verify that the associated organization role is system administrator.
Create a project with any desired name. Click Add API.
On the Add an API page, select Experience Cloud and select Smart Content. Click Next.
Select OAuth Server-to-Server. Click Next.
For details on how to do this configuration, see the Developer Console documentation, depending on your requirements:
- Overview:
  - Server to Server authentication
- Creating a new OAuth credential:
  - OAuth Server-to-Server credential implementation guide
- Migrating an existing JWT credential to an OAuth credential:
  - Migrating from Service Account (JWT) credential to OAuth Server-to-Server credential
In the Select product profiles page, select Smart Content Services. Click Save configured API.

A page displays more information about the configuration. Keep this page open to copy and add these values in Assets Smart Tagging Service Settings of cloud configuration in Experience Manager to configure smart tags.

Create IMS technical account configuration

You need to create IMS technical account configuration using the steps below:

In the Experience Manager user interface, access Tools > Security > Adobe IMS Configurations.
Click Create.
In the IMS Technical Account Configuration dialog, use the following values:

Field
Description
Cloud Solution
Choose Smart Tags from the drop down.
Title
Add title of the configuring IMS account.
Authorization Server
Add https://ims-na1.adobelogin.com
Client ID
To be provided through Adobe Developer console.
Client Secret
To be provided through Adobe Developer console.
Scope
To be provided through Adobe Developer console.
Org ID
To be provided through Adobe Developer console.
Select the configuration that you have created and click Check Health.
Confirm the check health dialog box and click close once the configuration is in the healthy state.

Create a new configuration

To configure the integration, use the values of TECHNICAL ACCOUNT ID, ORGANIZATION ID, CLIENT SECRET, and CLIENT ID fields from the Adobe Developer Console integration. Creating a Smart Tags cloud configuration allows authentication of API requests from the Experience Manager deployment.

In Experience Manager, navigate to Tools > Cloud Service > Smart Tag to open the Smart Tag Configurations.
Click Create to create a new configuration. Otherwise, Click Properties to update the existing configuration.
Fill up the following fields:

Field
Description
Title
Add title of the configuring IMS account.
Associated Adobe IMS Configuration
Choose configuration from the drop down.
Service URL
https://smartcontent.adobe.io/<region where your Experience Manager author instance is hosted>. For example, https://smartcontent.adobe.io/apac. You can specify na, emea, or, apac as the regions where your Experience Manager author instance is hosted.

NOTE
If the Experience Manager Managed Service is provisioned before September 01, 2022, use the following Service URL:
https://mc.adobe.io/marketingcloud/smartcontent
Click Save & Close.

Validate the configuration

After you have completed the configuration, you can use a JMX MBean to validate the configuration. To validate, follow these steps.

Access your Experience Manager server at https://[aem_server]:[port].
Go to Tools > Operations > Web Console to open the OSGi console. Click Main > JMX.
Click com.day.cq.dam.similaritysearch.internal.impl (SCS).
Click validateConfigs(). In the Validate Configurations dialog, click Invoke.

The validation results are displayed in the same dialog.

Enable smart tagging in the DAM Update Asset workflow (Optional)

In Experience Manager, go to Tools > Workflow > Models.
On Workflow Models page, select the DAM Update Asset workflow model.
Click Edit from the toolbar.
Expand the Side Panel to display the steps. Drag Smart Tag Asset step that is available in the DAM Workflow section and place it after the Process Thumbnails step.
Open the properties of the step to modify the details. Under Advanced Settings, ensure that the Handler Advance option is selected.
In the Arguments tab, select Ignore Errors if you want the workflow to complete even if the automatic tagging step fails.

Moreover, to tag assets when they are uploaded irrespective of whether smart tagging is enabled on folders, select Ignore Smart Tag Flag.
Click done to close the process step.
Click Sync to save the workflow.

Train the Smart Content Service

For the Smart Content Service to recognize your business taxonomy, run it on a set of assets that already include tags that are relevant to your business. To effectively tag your brand images, the Smart Content Service requires that the training images conform to certain guidelines. After training, the service can apply the same taxonomy on a similar set of assets.

You can train the service multiple times to improve its ability to apply relevant tags. After each training cycle, run a tagging workflow and check whether your assets are tagged appropriately.

You can train the Smart Content Service periodically or on requirement basis.

NOTE

The training workflow runs on folders only.

Guidelines for training

For best results, images in your training set conform to the following guidelines:

Quantity and size: Minimum 30 images per tag. Minimum of 500 pixels on the longer side.

Coherence: Images used for a specific tag are visually similar.

For example, it is not a good idea to tag all of these images as my-party (for training) because they are not visually similar.

Illustrative images to exemplify the guidelines for training

Coverage: Use sufficient variety in the images in the training. The idea is to supply a few but reasonably diverse examples so that Experience Manager learns to focus on the right things. If you’re applying the same tag on visually dissimilar images, include at least five examples of each kind.

For example, for the tag model-down-pose, include more training images similar to the highlighted image below for the service to identify similar images more accurately during tagging.

Illustrative images to exemplify the guidelines for training

Distraction/obstruction: The service trains better on images that have less distraction (prominent backgrounds, unrelated accompaniments, such as objects/persons with the main subject).

For example, for the tag casual-shoe, the second image is not a good training candidate.

Illustrative images to exemplify the guidelines for training

Completeness: If an image qualifies for more than one tag, add all applicable tags before including the image for training. For example, for tags, such as raincoat and model-side-view, add both the tags on the eligible asset before including it for training.

Illustrative images to exemplify the guidelines for training

NOTE

The ability of the Smart Content Service to train on your tags and apply them on other images depends on the quality of images you use for training. For best results, Adobe recommends that you use visually similar images to train the service for each tag.

Periodic training

You can enable the Smart Content Service to train periodically on the assets and associated tags within a folder. Open the Properties page of your asset folder, select Enable Smart Tags under the Details tab, and save the changes.

enable_smart_tags

Once this option is selected for a folder, Experience Manager runs a training workflow automatically to train the Smart Content Service on the folder assets and their tags. By default, the training workflow runs on a weekly basis at 12:30 AM on Saturdays.

On-demand training

You can train the Smart Content Service whenever required from the Workflow console.

In Experience Manager interface, go to Tools > Workflow > Models.
From the Workflow Models page, select the Smart Tags Training workflow and then click Start Workflow from the toolbar.
In the Run Workflow dialog, browse to the payload folder that includes the tagged assets for training the service.
Specify a title for the workflow and add a comment. Then, click Run. The assets and tags are submitted for training.

NOTE

Once the assets in a folder are processed for training, only the modified assets are processed in subsequent training cycles.

View training reports

To check whether the Smart Content Service is trained on your tags in the training set of assets, review the training workflow report from the Reports console.

In Experience Manager interface, go to Tools > Assets > Reports.
In the Asset Reports page, click Create.
Select the Smart Tags Training report, and then click Next from the toolbar.
Specify a title and description for the report. Under Schedule Report, leave the Now option selected. If you want to schedule the report for later, select Later and specify a date and time. Then, click Create from the toolbar.
In the Asset Reports page, select the report you generated. To view the report, click View from the toolbar.
Review the details of the report.

The report displays the training status for the tags you trained. The green color in the Training Status column indicates that the Smart Content Service is trained for the tag. Yellow color indicates that the service is not completely trained for a particular tag. In this case, add more images with the particular tag and run the training workflow to train the service completely on the tag.

If you do not see your tags in this report, run the training workflow again for these tags.
To download the report, select it from the list, and click Download from the toolbar. The report downloads as a Microsoft Excel spreadsheet.

Limitations

Enhanced smart tags are based on learning models of images and their tags. These models are not always perfect at identifying tags. The current version of the Smart Content Service has the following limitations:
- Inability to recognize subtle differences in images. For example, slim versus regular fitted shirts.
- Inability to identify tags based on tiny patterns/parts of an image. For example, logos on T-shirts.
- Tagging is supported in the locales that Experience Manager is supported in.
To search for assets with smart tags (regular or enhanced), use the Assets Omnisearch (full-text search). There is no separate search predicate for smart tags.

recommendation-more-help