Configure transcription in Experience Manager Assets configure-transcription-service

Transcription is the process of translating the audio from an audio or video file into text (speech to text) using the speech recognition technology.
Adobe Experience Manager Assets is configured with Azure Media Services which automatically generates text transcript of the spoken language in a supported audio or video file in the WebVTT (.vtt) format. When an audio or video asset is processed in Experience Manager Assets, the transcription service automatically generates the text transcript rendition of the audio or video asset and stores it at the same location within your Assets repository where the original asset resides. The Experience Manager Assets transcription service allows marketers to effectively manage their audio and video content with added discoverability of the text content and increase the ROI of these assets by supporting accessibility and localization.

Transcripts are text versions of spoken content; an example is a movie that you are watching on any OTT platform which often includes captions to help with accessibility or consuming the content in other languages. Or any audio or video file used for marketing, learning, or entertainment purpose. These experiences start with a transcription that is then formatted or translated as appropriate. Transcribing audio or video is a time intensive and error-prone process when performed manually. It is also a challenge to scale the manual process, given the ever-increasing need for audio-video content. Experience Manager Assets uses the Azure’s AI-based transcription which allows high-scale processing of the audio and video assets and generates the text transcripts (.vtt files) along with the timestamp details. Along with Assets, the transcription feature is also supported with Dynamic Media.

The transcription feature is available without any cost in Experience Manager Assets. However, the administrators require the user’s Azure credentials to configure the transcription service in Experience Manager Assets. You can also get the trial credentials directly from Microsoft® to experience the audio or video transcription feature in Assets.

Transcription prerequisites prerequisites

Configure transcription in Experience Manager Assets configure-transcription

Following are the configurations required to enable the transcription feature in Experience Manager Assets:

Configure Azure Media Services configure-azure-media-services

Experience Manager Assets uses the Azure Media Services which automatically generates text transcripts of the spoken language in a supported audio or video file in the WebVTT (.vtt) format. The administrators can configure Azure Media Services in Experience Manager Assets using the Azure credentials. The transcription prerequisites list the Azure credentials that are required for the configuration. If you do not have Azure account and credentials, see Azure Media Services documentation to get trial credentials.

Go to Tools > Cloud Services > Azure Media Services Configuration. Select a folder (location) from the left rail and click the Create button to configure the connection with your Azure account. This folder is the location where your Azure cloud configuration is stored in Experience Manager Assets. Enter the Azure credentials and click Save & Close.

Configure Processing Profile for transcription configure-processing-profile

Once the Azure Media Services is configured in Experience Manager Assets, the next step is to create an asset Processing Profile for generating AI-based transcription of the audio and video assets. The AI-based processing profile generates transcripts of the supported audio or video asset as a rendition in Experience Manager Assets and stores the transcript (.vtt file) in the same folder where the original asset resides. Thus, it is easier for the users to search and locate the asset and its transcript rendition.

Go to Tools > Assets > Processing Profiles and click the Create button to create an AI-based processing profile for generating transcription of your audio and video files. By default, the processing profile page reflects only three tabs (Image, Video, and Custom). However, a Content AI tab is visible if you have configured Azure Media Services in your Experience Manager Assets instance. Verify your Azure credentials if you do not see the Content AI tab while creating a Processing Profile.

In the Content AI tab, click the Add New button to configure transcription. Here, you can include and exclude the file formats (MIME types) for generating transcripts by selecting file types from the drop-down list. In the following Illustration, all the supported audio and video files are included and the text files are excluded.

Enable the Create VTT transcript in same directory toggle to create and store the transcript rendition (.vtt file) in the same folder where the original asset resides. The other renditions are also generated by the default DAM asset processing workflow regardless of this setting.

The following Illustration details a custom video profile that is created in Experience Manager Assets.

The video profile also contains the following custom configurations. See processing profile documentation for details on how to create custom processing profile.

Let us now configure transcription in this video profile. Navigate to the Content AI tab and click the Add New button. Include all the audio and video files and exclude the image and application files. Enable the Create VTT transcript in same directory toggle and save the configuration.

Once the processing profile is configured for transcription of audio and video files, you can apply this processing profile to folders using one of the following methods:

Generate transcription of your audio or video assets generate-transcription

When processing a video asset, the AI-based processing profile automatically generates the transcript (.vtt file) as a rendition along with the original asset in the same folder.

You can also see the transcript rendition by accessing the Renditions of the original video asset. To access the Renditions panel, select the original video asset and open the left rail. You can see that the transcript rendition (.vtt file) is visible under the TRANSCRIPTVTT head.

You can download the transcript (.vtt text file) directly from the folder as a separate asset rendition, or from within the Renditions panel of the original asset by downloading all the renditions of the asset.

Currently, Experience Manager does not support full text preview or editing of VTT files natively. However, you can download the transcript rendition and use any text editor to edit or verify the transcript. The transcript reflects the spoken language as a text at the given timestamp in the video with the confidence score (accuracy) of the transcription.

Using transcription in Dynamic Media using-transcription-in-dynamic-media

If you have configured Dynamic Media in your Experience Manager Assets instance, you can publish the asset (audio or video file) and its transcript (.vtt file) to Dynamic Media. By doing so, the original asset (audio or video file) and its transcribed rendition (.vtt file) are published to Dynamic Media in the same folder. The Dynamic Media administrator can enable CC Closed Caption experience for the audio or video file using the transcript rendition (.vtt file).

See also:

In the following Illustration, the URL reflects the caption part which refers to the transcript (.vtt file). The video reflects the spoken language (transcribed text) as a Closed Caption at the given timestamp in the video. The user can enable or disable the caption by using the CC button.

Supported file formats for transcription supported-file-format

The following audio and video file formats are supported for transcription:

Known limitations known-limitations

Troubleshooting tips troubleshooting

Log in to your Azure Media Services account with the same credentials (that you have used for configuration) to verify the request status. Contact Azure support if your request is not processed successfully.

See also