[AEM Assets]{class="badge positive" title="Applies to AEM Assets)."}

Smart Tags for AEM Assets using-smart-tags

Organizations possess numerous digital assets, and this number continues to grow rapidly. Searching for a specific asset amidst such a vast amount of data poses a significant challenge. To address this, metadata and tags are employed to enhance the searchability of digital assets. Organizations use taxonomy-controlled vocabularies in asset metadata. These typically consist of keyword lists that employees, partners, and customers commonly use to refer to and locate digital assets.

Smart Tags are keywords that not only appear in the text but also best describe the asset. Tagging assets with taxonomy-controlled vocabulary ensures they can be easily identified and retrieved through search.

For instance, words arranged alphabetically in a dictionary are easier to find than randomly scattered ones. Tagging serves a similar purpose. It organizes assets according to business taxonomy, ensuring that the most relevant ones appear in search results. For example, a car manufacturer can tag car images with model names, so that only relevant images are displayed when designing a promotional campaign. Whether tagging “runners” or “running shoes,” users do not need to worry about typos, spelling variations, or alternate search terms—Smart Tags recognize them all.

In the background, the functionality uses the artificially intelligent framework of Adobe AI, which automatically applies Smart Tags to uploaded assets—by default—along with text aligned to the business taxonomy.

Prerequisites and configuration smart-tags-prereqs-config

Smart Tags is automatically provisioned for Adobe Experience Manager as a Cloud Service and hence no configuration is required.

Smart Tags workflow smart-tags-workflow

Adobe AI powered smart tagging uses artificial intelligence models to analyze content and add tags to the assets, thereby reducing time for DAM users to deliver rich experiences to their customers. The Smart Tags are displayed in descending order of their confidence score in asset properties.

  • Image-based assets
    For images, the Smart Tags are based on some visual aspect. Images in many formats are tagged using smart content services. Smart Tags are applied to the supported file types that generate renditions in JPG and PNG format.

  • Video-based assets
    For video-based assets, tagging is enabled by default in Adobe Experience Manager as a Cloud Service. As is the case with Image and text based tags, videos are also auto-tagged when you upload new videos or reprocess existing ones. Adobe AI generates two sets of tags for a video: One set corresponds to objects, scenes, and attributes in that video, and the other set relates to actions such as drinking, running, and jogging. Also check opt out video smart tagging.

  • Text-based assets
    For supported assets, Experience Manager already extracts the text, which is then indexed and is used to search for the assets. However, Smart Tags based on keywords in the text provide a dedicated, structured, and higher priority search facet. The latter helps improve asset discovery as compared to a search index.
    For text-based assets, the efficacy of Smart Tags does not depend on the amount of text in the asset but on the relevant keywords or entities present in the text of the asset.

    Smart-tag-types

Smart Tags are implemented in AEM Assets using the following workflow:

  1. Create or upload an asset in AEM. Out of the box tags are generated for image, video, and text based Assets.

  2. If you find that specific tags are not generated, then you can train your image-type tags accordingly. Refer to Smart Tags training.

Supported file formats for Smart Tags supported-file-formats

Images (MIME types)
Text-based assets (file formats)
Video assets (file formats and codecs)
image/jpeg
CSV
MP4 (H264/AVC)
image/tiff
DOC
MKV (H264/AVC)
image/png
DOCX
MOV (H264/AVC, Motion JPEG)
image/bmp
HTML
AVI (indeo4)
image/gif
PDF
FLV (H264/AVC, vp6f)
image/pjpeg
PPT
WMV (WMV2)
image/x-portable-anymap
PPTX
image/x-portable-bitmap
RTF
image/x-portable-graymap
SRT
image/x-portable-pixmap
TXT
image/x-rgb
VTT
image/x-xbitmap
image/x-xpixmap
image/x-icon
image/photoshop
image/x-photoshop
image/psd
image/vnd.adobe.photoshop

Preparing an asset for out of the box smart tagging

When you upload assets to Adobe Experience Manager as a Cloud Service, the uploaded assets are processed. Once the processing is complete, see the Basic tab of asset Properties page. Smart Tags are automatically added to the assets under Smart Tags. Asset microservices uses Adobe AI to create these Smart Tags.

Smart Tags are added to videos and seen in Basic tab of asset Properties

IMPORTANT
You are advised to review these automatically generated tags to ensure that they conform to your brand and its values.

Untagged Assets in DAM smart-tag-existing-assets

The existing or older assets in DAM are not smart tagged automatically. You need to Reprocess Assets manually to generate Smart Tags for them. Once the process completes, navigate to the Properties page of any asset within the folder. The automatically added tags are seen in the Smart Tags section of the Basic tab. These applied Smart Tags are sorted in descending order of confidence score.

Confidence Score confidence-score

Your asset search results are ranked based on the confidence scores, which generally improve search results beyond what an inspection of the assigned tags of any asset suggests. Inaccurate tags often have low confidence scores, so they seldom appear at the top of the Smart Tags list for assets.

NOTE
Manual tags are assigned a confidence of 100% (maximum confidence). Therefore, if there are assets with manual tags that match the search query, they are displayed before Smart Tags matching the search query.

Moderate Smart tags moderate-smart-tags

Adobe Experience Manager as a Cloud Service allows you to curate the Smart Tags to:

  • remove inaccurate tags assigned to your brand assets.

  • refine tag-based searches for assets by ensuring that your asset appears in search results for the most relevant tags. It, therefore, eliminates the chances of unrelated assets from showing up in search results.

  • assign a higher rank to a tag to increase its relevance with respect to an asset. Promoting a tag for an asset increases the chances of the particular asset appearing in search results when a search is performed based on that tag.

To know more about how to moderate the Smart Tags for assets, see Manage Smart Tags.

Moderate video Smart Tags

NOTE
Any tags that are moderated using the steps in Manage Smart Tags are not remembered on reprocessing of the asset. The original sets of tags are displayed again.

Manage Smart Tags and asset searches manage-smart-tags-and-searches

You can curate Smart Tags to remove any inaccurate tags that may have been assigned to your brand assets, so that only the most relevant tags are displayed.

Moderating Smart Tags also helps refine tag-based searches for assets by ensuring that your assets appear in search results for the most relevant tags. Essentially, it helps eliminate the chances of unrelated assets from showing up in search results.

You can also assign a higher rank to a tag to increase the tag’s relevance for the asset. Promoting a tag for an asset increases the chances of the asset appearing in search results when a search is performed based on the particular tag.

To moderate the Smart Tags of your digital assets:

  1. In the search field, search for digital assets based on a tag.

  2. To identify the digital assets that you do not find relevant to your search, inspect the search results.

  3. Select an asset, and then select Manage tags icon from the toolbar.

  4. From the Manage Tags page, inspect the tags. If you do not want the asset to be searched based on a specific tag, then select the tag and select Delete icon from the toolbar. Alternatively, select close icon next to the label.

  5. To assign a higher rank to a tag, select the tag and select Promote icon from the toolbar. The tag you promote is moved to the Tags section.

  6. Select Save and then select OK to close the Success dialog.

  7. Navigate to the Properties page for the asset. Observe that the tag that you promoted is assigned a high relevance and, therefore, appears higher in the search results.

By default, Experience Manager combines the search terms with an AND or OR clause to find any of the search terms in the applied Smart Tags. Using Smart Tags does not change this default behavior. For example, consider searching for woman running. Assets with just woman or just running keyword in the metadata do not appear in the search results by default. However, an asset tagged with either woman or running using Smart Tags appears in such a search query. So the search results are a combination of,

  • Assets with woman and running keywords in the metadata.

  • Assets smart tagged with either of the keywords.

The search results that match all search terms in metadata fields are displayed first, followed by the search results that match any of the search terms in the Smart Tags. In the above example, the approximate order of display of search results is:

  1. matches of woman running in the various metadata fields.
  2. matches of woman running in Smart Tags.
  3. matches of woman or of running in Smart Tags.

Opt out of smart tagging opt-out-smart-tagging

As the automated tagging of assets runs in parallel to other asset processing tasks like thumbnail creation and metadata extraction, it can be time consuming. To expedite the asset processing, you can opt out of smart tagging on upload at folder level. To opt out of automated Smart Tags generation for assets uploaded to specific folder:

  1. Open Asset Processing tab in folder Properties.

  2. In Smart Tags for Videos menu, for example, Inherited option is selected by default and video smart tag is enabled.

    When the Inherited option is selected, the inherited folder path is also visible along with the information whether it is set to Enable or Disable.

    Disable smart tagging

  3. Select Disable to opt out of smart tagging uploaded to the folder.

  4. Similarly, you can opt out smart tagging for Smart Tags for Text, Smart Tags for Image, and Color Tags for Images.

IMPORTANT
If you have opted out of tagging on a folder at the time of upload and want to smart tag the after upload, then Enable Smart Tags from Asset Processing tab of the folder Properties and use Reprocess Asset option to add Smart Tags to the assets.

These models are not always perfect at identifying tags. The current version of the Smart Tags has the following limitations:

  • Inability to recognize subtle differences in images. For example, slim-fit versus regular-fit shirts.

  • Inability to identify tags based on tiny patterns or parts of an image. For example, logos on shirts.

  • The tags that are not handled relate to:

    • Non-visual, abstract aspects. For example, the year or season of release of a product, mood or emotion evoked by an image, and a subjective connotation of a video.
    • Fine visual differences in products such as shirts with and without collars or small product logos embedded on products.
  • Only the videos smaller than 300 MB in file size are auto-tagged. The Adobe AI service skips video files that are larger in size.

  • To search for files with Smart Tags (regular or enhanced), use the Assets search (full-text search). There is no separate search predicate for Smart Tags.

  • In comparison of general tags, the assets that are tagged using business taxonomy are easier to identify and retrieve by tag-based searches.

Frequently asked questions faq-smart-tags

How do Smart Tags improve search experience of an asset?
Adobe AI tags the assets automatically once you upload them. The automated process runs so fast at the backend that you will see tags added in your assets after a few seconds once the upload is completed.
What happens if the Smart Tags list is inaccurate or showing unwanted tag?
An inaccurate or unwanted tag can be removed from the list. For example, as an automobile dealer, you might want to remove “damaged” tag from the list.
How can you prioritize assets containing same tags?
Yes, you can prioritize assets containing the same tags. You can promote a tag into the Smart Tags list of an asset to perform prioritization. Promoting a tag allows you to prioritize the images appearing in the search results for that particular tag.
Is the application of Smart Tags limited to a particular folder?
Smart Tags are configurable and can be applied on any folder inside DAM.
How may I know that tagging needs training?
Refer to Determining the requirement of Smart Tags training.
What are the supported file formats for tagging an asset?
Refer to Supported file formats.
In which language smart tags are generated?
Smart tags are generated in English language only. They can be translated to other languages by translating the whole asset including metadata.
I do not want to use Smart Tagging anymore.
You can opt out Smart Tagging anytime you want to discontinue.

Troubleshooting Scenarios troubleshooting-scenarios

Feature naming and navigation feature-naming-and-navigation

AI-Generated metadata or Smart Tags automatically tags assets using artificial intelligence. There are two related but distinct configuration/navigation surfaces.

Task
Correct location
Frequently Encountered Issues
Enable or disable the AI-Generated metadata (Smart Tag enhancements) at the platform level
Tools > Assets > Assets Configurations > Smart Tag Enhancement Configuration
This is a configuration toggle, self-service.
Train a custom Smart Tag model
Assets > Smart Tag Training, reached from /aem/start.html
This feature is not available under Tools or under Tools > Assets > Smart Tag Training.

If Smart Tag Training is not visible under Assets in /aem/start.html, check (in order) the user’s group or role assignment, whether Smart Tags auto-tagging is enabled for the instance, the availability of Adobe Experience Manager (AEM) as a Cloud Service version or feature, and whether the asset types in question are supported for custom model training. For more information on AI-Generated metadata or Smart Tags, see Enhancing content discovery with AI-Generated metadata.

Enablement history and self-service status enablement-history-and-self-service-status

AI-Generated metadata enablement or disablement previously required Adobe Engineering intervention in some cases. This has now been moved to a fully self-service model. You can enable or disable it at any time through the Smart Tag Enhancement Configuration screen, without needing to open a support ticket for enablement.

Requirements and Limitations requirements-and-limitations

  • Smart Tags requires the Smart Content Services add-on or entitlement.
  • Smart Tags and Dynamic Media both process and deliver content through Adobe’s global cloud infrastructure. For customers with data residency requirements (for example, data must stay within a specific country), be explicit that using these features may involve processing or transferring asset data outside that region. This is a characteristic of the global infrastructure, not a configurable regional restriction.
  • Image generation through Smart Tags or AI features in AEM Assets uses Adobe-owned (first-party) Firefly models such as Image Model 4 only by default. There is currently no AEM-side configuration or model picker to select or restrict which model is used, and model attribution is not exposed in asset metadata. Plan compliance reporting accordingly if model provenance needs to be tracked externally.
  • The separate Generate Variations generative feature is licensed specifically under AEM Sites with Edge Delivery Services (EDS) and is not available to Assets-only environments regardless of Smart Tags or AI-Generated metadata status. Do not conflate the two generative AI capabilities.

Resolution Workflow resolution-workflow

  1. Smart Tag Training menu option is not visible: Confirm you are looking under Assets > Smart Tag Training in /aem/start.html, and not under Tools.
  2. Feature seems disabled and no one can find the toggle: Check Tools > Assets > Assets Configurations > Smart Tag Enhancement Configuration. This is self-service and does not require an Adobe support ticket.
  3. Data residency or compliance question: Clarify that Smart Tags and Dynamic Media uses Adobe’s global infrastructure, which can include processing outside your home region.
  4. Compliance question about which AI model was used: Clarify that only Adobe-owned Firefly models are invoked, there is no selectable alternative, and model attribution is not currently surfaced in metadata.
  5. Generate Variations are not available: Confirm the organisation has the AEM Sites and Edge Delivery Services (EDS) SKU. This is a separate entitlement from Smart Tags or AI-Generated metadata.

See also

recommendation-more-help
experience-manager-cloud-service-help-main-toc