Documentation Experience Platform Source Connectors Guide

Beta

Ingest encrypted data in the sources UI

Last update: April 2, 2025

Topics:
Sources

CREATED FOR:

Developer

AVAILABILITY

Support for encrypted data ingestion in the sources UI is in beta. The feature and documentation are subject to change.

You can ingest encrypted data files and folders to Adobe Experience Platform using cloud storage batch sources. With encrypted data ingestion, you can leverage asymmetric encryption mechanisms to securely transfer batch data into Experience Platform. The supported asymmetric encryption mechanisms are PGP and GPG.

Read this guide to learn how you can ingest encrypted data with cloud storage batch sources using the UI.

Get started

Before continuing with this tutorial, please read the following documents to better understand the following Experience Platform features and concepts.

Sources: Use sources in Experience Platform to ingest data from an Adobe Application or a third-party data source.
Dataflows: Dataflows are representations of data jobs that move data across Experience Platform. You can use the sources workspace to create dataflows that ingest data from a given source to Experience Platform.
Sandboxes: Use sandboxes in Experience Platform to create virtual partitions between your Experience Platform instances and create environments dedicated to development or production.

High-level outline

Create an encryption key pair using the sources workspace in the Experience Platform UI.
- Optionally, you may also create your own sign verification key pair to provide an additional layer of security to your encrypted data.
Use the public key from your encryption key pair to encrypt your data.
Place your encrypted data in your cloud storage. During this step, you must also ensure that you have a sample file of your data in your cloud storage that can be used as a reference to map your source data to an Experience Data Model (XDM) schema.
Use your cloud storage batch source and begin the data ingestion process in the sources workspace in the Experience Platform UI.
During the source connection creation process, provide the key ID that corresponds with the public key that you used to encrypt your data.
- If you also used the sign verification key pair mechanism, then you must also provide the sign verification key ID that corresponds to your encrypted data.
Proceed to the dataflow creation steps.

Create an encryption key pair

What is an encryption key pair?

An encryption key pair is an asymmetric cryptography mechanism that consists of a public key and a private key. The public key is used to encrypt data and the private key is then used to decrypt said data.

You can create your encryption key pair through the Experience Platform UI. When generated, you will receive a public key and a corresponding key ID. Use the public key to encrypt your data and then use the key ID to confirm your identity, when you are in the process of ingesting your encrypted data. The private key automatically goes to Experience Platform, where it is stored in a secure vault, and will only be used once your data is ready for decryption.

In the Experience Platform UI, navigate to the sources workspace and then select Key Pairs from the top header.

The sources catalog with the "Key Pairs" header selected.

You are taken to a page that displays a list of existing encryption key pairs in your organization. This page provides information on a given key’s title, ID, type, encryption algorithm, expiry, and status. To create a new key pair, select Create Key.

The Key Pairs page, with "encryption key" selected as the key type and the "create key" button selected.

Next, choose the key type that you want to make. To create an encryption key, select Encryption Key and then select Continue.

The key creation window, with encryption key selected.

Provide a title and a passphrase for your encryption key. The passphrase is an additional layer of protection for your encryption keys. Upon creation, Experience Platform stores the passphrase in a different secure vault from the public key. You must provide a non-empty string as a passphrase. When finished, select Create.

The encryption key creation window, where a title and a passphrase is provided.

If successful, a new window appears, displaying your new encryption key, including its title, public key, and key ID. Use the public key value to encrypt your data. You will use the key ID in a later step to prove your identity when ingesting your encrypted data during the dataflow creation process.

The window that displays information on your newly created encryption key pair.

To view information on an existing encryption key, select the ellipses (...) beside the key title. Select Key details to view the public key and key ID. Alternatively, if you want to delete your encryption key, select Delete.

The key pairs page, where a list of encryption keys are displayed. The ellipses beside "acme-encryption-key" is selected and the dropdown displays options to view key details or delete the keys.

Create a sign verification key

What is a sign verification key?

A sign verification key is another encryption mechanism that involves a private key and a public key. In this case, you can create your sign verification key pair and use the private key to sign and provide an additional layer of encryption to your data. You will then share the corresponding public key to Experience Platform. During ingestion, Experience Platform will use the public key to verify the signature associated with your private key.

To create a sign verification key, select Sign Verification Key from the key type selection window and then select Continue.

The key type selection window where sign verification key is selected.

Next, provide a title and a Base64-encoded PGP key as your public key and then select Create.

The create sign verification key window.

If successful, a new window appears, displaying your new sign verification key, including its title and key ID.

The details of the newly created sign verification key.

Ingest encrypted data

You can ingest encrypted data using the following cloud storage batch sources:

Authenticate with the cloud storage source of your choice. During the data selection step of the workflow, select the encrypted file or folder that you want to ingest and then enable the Is the file encrypted toggle.

The "select data" step of the sources workflow, where an encrypted data file is selected for ingested.

Next, select a sample file from your source data. Since your data is encrypted, Experience Platform will require a sample file in order to create an XDM schema that can be mapped to your source data.

The "Is this file encrypted?" toggle enabled and the "Select sample file" button selected.

Once you have selected your sample file, configure settings of your data, such as its corresponding data format, delimiter, and compression type. Allow some time for the preview interface to fully render, and then select Save.

A sample is selected for ingestion and the file preview is fully loaded.

From here, use the dropdown menu to select the public key title of public key ID that corresponds with the public key that you used to encrypt your data.

The public key title of the public key ID that corresponds with the public key used to encrypt your data.

If you also used the sign verification key pair to provide and additional layer of encryption, then enable the sign verification key toggle and then similarly, use the dropdown to select the sign verification key ID that corresponds with the key that you used to encrypt your data.

The sign verification key title of the the key ID that corresponds with your sign verification encryption.

When finished select Next.

Complete the remaining steps in the sources workflow to finish creating your dataflow.

You can continue to make updates to your dataflow once it has been successfully created.

Next steps

By reading this document, you can now ingest encrypted data from your cloud storage batch source to Experience Platform. For information on how to ingest encrypted data using the APIs, read the guide on ingesting encrypted data using the Flow Service API. For general information about sources on Experience Platform, read the sources overview.