Activate audiences to file-based destinations by using the Flow Service API
April 2, 2025
- Topics:
- Destinations
CREATED FOR:
- Admin
- User
Use the enhanced file export capabilities to access enhanced customization functionality when exporting files out of Experience Platform:
- Additional file naming options.
- Ability to set custom file headers in your exported files via the improved mapping step.
- Ability to select the file type of the exported file.
- Ability to customize the formatting of exported CSV data files.
This functionality is supported by the six cloud storage cards listed below:
This article explains the workflow required to use the Flow Service API to export qualified profiles from Adobe Experience Platform to one of the cloud storage locations linked above.
TIPYou can also use the Experience Platform user interface to export profiles to cloud storage destinations. Read the activate file-based destinations tutorial for more information.
Getting started
This guide requires a working understanding of the following components of Adobe Experience Platform:
- Experience Data Model (XDM) System: The standardized framework by which Experience Platform organizes customer experience data.
- Segmentation Service: Adobe Experience Platform Segmentation Service allows you to build audiences and generate audiences in Adobe Experience Platform from your Real-Time Customer Profile data.
- Sandboxes: Experience Platform provides virtual sandboxes which partition a single Experience Platform instance into separate virtual environments to help develop and evolve digital experience applications.
The following sections provide additional information that you need to know in order to activate data to file-based destinations in Experience Platform.
Required permissions
To export profiles, you need the View Destinations, Activate Destinations, View Profiles, and View Segments access control permissions. Read the access control overview or contact your product administrator to obtain the required permissions.
To export identities, you need the View Identity Graph access control permission.
Reading sample API calls
This tutorial provides example API calls to demonstrate how to format your requests. These include paths, required headers, and properly formatted request payloads. Sample JSON returned in API responses is also provided. For information on the conventions used in documentation for sample API calls, see the section on how to read example API calls in the Experience Platform troubleshooting guide.
Gather values for required and optional headers
In order to make calls to Experience Platform APIs, you must first complete the Experience Platform authentication tutorial. Completing the authentication tutorial provides the values for each of the required headers in all Experience Platform API calls, as shown below:
- Authorization: Bearer
{ACCESS_TOKEN} - x-api-key:
{API_KEY} - x-gw-ims-org-id:
{ORG_ID}
Resources in Experience Platform can be isolated to specific virtual sandboxes. In requests to Experience Platform APIs, you can specify the name and ID of the sandbox that the operation will take place in. These are optional parameters.
- x-sandbox-name:
{SANDBOX_NAME}
NOTEFor more information on sandboxes in Experience Platform, see the sandbox overview documentation.
All requests that contain a payload (POST, PUT, PATCH) require an additional media type header:
- Content-Type:
application/json
API reference documentation
You can find accompanying reference documentation for all the API operations in this tutorial. Refer to the Flow Service - Destinations API documentation on the Adobe Developer website. We recommend that you use this tutorial and the API reference documentation in parallel.
Glossary
For descriptions of the terms that you will be encountering in this API tutorial, read the glossary section of the API reference documentation.
Select destination where to export audiences
Before starting the workflow to export profiles, identify the connection spec and flow spec IDs of the destination to which you are intending to export audiences to. Use the table below for reference.
Destination
Connection spec
Flow spec
Amazon S3
4fce964d-3f37-408f-9778-e597338a21ee1a0514a6-33d4-4c7f-aff8-594799c47549Azure Blob Storage
6d6b59bf-fb58-4107-9064-4d246c0e5bb2752d422f-b16f-4f0d-b1c6-26e448e3b388Azure Data Lake Gen 2(ADLS Gen2)
be2c3209-53bc-47e7-ab25-145db8b873e117be2013-2549-41ce-96e7-a70363bec293Data Landing Zone(DLZ)
10440537-2a7b-4583-ac39-ed38d4b848e8cd2fc47e-e838-4f38-a581-8fff2f99b63aGoogle Cloud Storage
c5d93acb-ea8b-4b14-8f53-02138444ae99585c15c4-6cbf-4126-8f87-e26bff78b657SFTP
36965a81-b1c6-401b-99f8-22508f1e6a26fd36aaa4-bf2b-43fb-9387-43785eeeb799You need these IDs to construct various the flow service entities in the next steps of this tutorial. You also need to refer to parts of the connection spec itself to set up certain entities so you can retrieve the Connection Spec from Flow Service APIs. See the examples below of retrieving connection specs for all the destinations in the table:
Amazon S3
Request
Retrieve connection spec for Amazon S3
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/4fce964d-3f37-408f-9778-e597338a21ee' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
Response
Amazon S3 - Connection spec
{
"items": [
{
"id": "4fce964d-3f37-408f-9778-e597338a21ee",
"name": "Amazon S3",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
Azure Blob Storage
Request
Retrieve connection spec for Azure Blob Storage
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/6d6b59bf-fb58-4107-9064-4d246c0e5bb2' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
Response
Azure Blob Storage - Connection spec
{
"items": [
{
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2",
"name": "Azure Blob Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
Azure Data Lake Gen 2(ADLS Gen2)
Request
Retrieve connection spec for Azure Data Lake Gen 2(ADLS Gen2)
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/be2c3209-53bc-47e7-ab25-145db8b873e1' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
Response
Azure Data Lake Gen 2(ADLS Gen2) - Connection spec
{
"items": [
{
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1",
"name": "Azure Data Lake Gen2",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
Data Landing Zone(DLZ)
Request
Retrieve connection spec for Data Landing Zone(DLZ)
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/10440537-2a7b-4583-ac39-ed38d4b848e8' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
Response
Data Landing Zone(DLZ) - Connection spec
{
"items": [
{
"id": "10440537-2a7b-4583-ac39-ed38d4b848e8",
"name": "Data Landing Zone",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
Google Cloud Storage
Request
Retrieve connection spec for Google Cloud Storage
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/c5d93acb-ea8b-4b14-8f53-02138444ae99' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
Response
Google Cloud Storage - Connection spec
{
"items": [
{
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99",
"name": "Google Cloud Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
SFTP
Request
Retrieve connection spec for SFTP
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/36965a81-b1c6-401b-99f8-22508f1e6a26' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
Response
SFTP - Connection spec
{
"items": [
{
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26",
"name": "SFTP",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
Follow the steps below to set up an audience export dataflow to a cloud storage destination. For some steps, the requests and responses differ between the various cloud storage destinations. In those cases, use the tabs on the page to retrieve the requests and responses specific to the destination that you want to connect and export audiences to. Be sure to use the correct connection spec and flow spec for the destination you are configuring.
Create a Source Connection
After deciding which destination you are exporting audiences to, you need to create a source connection. The source connection represents the connection to the internal Experience Platform Profile store.
recommendation-more-help
Request
Create source connection - Request
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/sourceConnections' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name":"Connect to Profile store",
"description":"Optional",
"connectionSpec":{
"id":"8a9c3494-9708-43d7-ae3f-cda01e5030e1", // this connection spec ID is always the same for Source Connections
"version":"1.0"
}
}'
Response
Create source connection - Response
{
"id": "900df191-b983-45cd-90d5-4c7a0326d650",
"etag": "\"0500ebe1-0000-0200-0000-63e28d060000\""
}
A successful response returns the ID (id) of the newly created source connection and an etag. Note down the source connection ID as you will need it later when creating the dataflow.
Create a base connection
A base connection securely stores the credentials to your destination. Depending on the destination type, the credentials needed to authenticate against that destination can vary. To find these authentication parameters, first retrieve the connection spec for your desired destination as described in the section Select destination where to export audiences and then look at the authSpec of the response. Reference the tabs below for the authSpec properties of all supported destinations.
Amazon S3
Amazon S3 - Connection spec showing auth spec
Note the highlighted line with inline comments in the connection spec example below, which provide additional information about where to find the authentication parameters in the connection spec.
{
"items": [
{
"id": "4fce964d-3f37-408f-9778-e597338a21ee",
"name": "amazon-s3",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [
{
"name": "Access Key",
"type": "KeyBased",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Defines auth params required for connecting to amazon-s3",
"type": "object",
"properties": {
"s3AccessKey": {
"description": "Access key id",
"type": "string",
"pattern": "^[A-Z2-7]{20}$"
},
"s3SecretKey": {
"description": "Secret access key for the user account",
"type": "string",
"format": "password",
"pattern": "^[A-Za-z0-9\\/+]{40}$"
}
},
"required": [
"s3SecretKey",
"s3AccessKey"
]
}
},
{
"name": "Assumed Role",
"type": "S3RoleBased",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Defines role based auth params required for connecting to amazon-s3",
"type": "object",
"properties": {
"s3Role": {
"title": "Role",
"description": "S3 role",
"type": "string",
"format": "password"
}
},
"required": [
"s3Role"
]
}
}
],
//...
Azure Blob Storage
Azure Blob Storage - Connection spec showing auth spec
Note the highlighted line with inline comments in the connection spec example below, which provide additional information about where to find the authentication parameters in the connection spec.
{
"items": [
{
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2",
"name": "Azure Blob Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "ConnectionString",
"type": "ConnectionString",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Connection String for Azure Blob based destinations",
"type": "object",
"properties": {
"connectionString": {
"description": "connection string for login",
"type": "string",
"format": "password"
}
},
"required": [
"connectionString"
]
}
}
],
//...
Azure Data Lake Gen 2(ADLS Gen2)
Azure Data Lake Gen 2(ADLS Gen2) - Connection spec showing auth spec
Note the highlighted line with inline comments in the connection spec example below, which provide additional information about where to find the authentication parameters in the connection spec.
{
"items": [
{
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1",
"name": "Azure Data Lake Gen2",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "Azure Service Principal Auth",
"type": "AzureServicePrincipal",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "defines auth params required for connecting to adlsgen2 using service principal",
"type": "object",
"properties": {
"url": {
"description": "Endpoint for Azure Data Lake Storage Gen2.",
"type": "string"
},
"servicePrincipalId": {
"description": "Service Principal Id to connect to ADLSGen2.",
"type": "string"
},
"servicePrincipalKey": {
"description": "Service Principal Key to connect to ADLSGen2.",
"type": "string",
"format": "password"
},
"tenant": {
"description": "Tenant information(domain name or tenant ID).",
"type": "string"
}
},
"required": [
"servicePrincipalKey",
"url",
"tenant",
"servicePrincipalId"
]
}
}
],
//...
Data Landing Zone(DLZ)
Data Landing Zone(DLZ) - Connection spec showing auth spec
NOTEThe Data Landing Zone destination does not require an auth spec.
{
"items": [
{
"id": "10440537-2a7b-4583-ac39-ed38d4b848e8",
"name": "Data Landing Zone",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [],
//...
Google Cloud Storage
Google Cloud Storage - Connection spec showing auth spec
Note the highlighted line with inline comments in the connection spec example below, which provide additional information about where to find the authentication parameters in the connection spec.
{
"items": [
{
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99",
"name": "Google Cloud Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "Google Cloud Storage authentication credentials",
"type": "GoogleCloudStorageAuth",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "defines auth params required for connecting to google cloud storage connector.",
"type": "object",
"properties": {
"accessKeyId": {
"description": "Access Key Id for the user account",
"type": "string"
},
"secretAccessKey": {
"description": "Secret Access Key for the user account",
"type": "string",
"format": "password"
}
},
"required": [
"accessKeyId",
"secretAccessKey"
]
}
}
],
//...
SFTP
SFTP - Connection spec showing auth spec
NOTEThe SFTP destination contains two separate items in the auth spec, as it supports both password and SSH key authentication.
Note the highlighted line with inline comments in the connection spec example below, which provide additional information about where to find the authentication parameters in the connection spec.
{
"items": [
{
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26",
"name": "SFTP",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "SFTP with Password",
"type": "SFTP",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "defines auth params required for connecting to sftp locations with a password",
"type": "object",
"properties": {
"domain": {
"description": "Domain of server",
"type": "string"
},
"username": {
"description": "Username",
"type": "string"
},
"password": {
"description": "Password",
"type": "string",
"format": "password"
}
},
"required": [
"password",
"domain",
"username"
]
}
},
{
"name": "SFTP with SSH Key",
"type": "SFTP",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "defines auth params required for connecting to sftp locations using SSH Key",
"type": "object",
"properties": {
"domain": {
"description": "Domain of server",
"type": "string"
},
"username": {
"description": "Username",
"type": "string"
},
"sshKey": {
"description": "Base64 string of the private SSH key",
"type": "string",
"format": "password",
"contentEncoding": "base64",
"uiAttributes": {
"tooltip": {
"id": "platform_destinations_connect_sftp_ssh",
"fallbackUrl": "http://www.adobe.com/go/destinations-sftp-connection-parameters-en "
}
}
}
},
"required": [
"sshKey",
"domain",
"username"
]
}
}
],
//...
Using the properties specified in the authentication spec (i.e. authSpec from the response) you can create a base connection with the required credentials, specific to each destination type, as shown in the examples below:
Amazon S3
Request
Amazon S3 - Base connection request with access key and secret key authentication
TIPFor information on how to obtain the required authentication credentials, refer to the authenticate to destination section of the Amazon S3 destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Amazon S3 Base Connection",
"auth": {
"specName": "Access Key",
"params": {
"s3SecretKey": "<Add secret key>",
"s3AccessKey": "<Add access key>"
}
},
"connectionSpec": {
"id": "4fce964d-3f37-408f-9778-e597338a21ee", // Amazon S3 connection spec
"version": "1.0"
}
}'
Amazon S3 - Base connection request with assumed role authentication
TIPFor information on how to obtain the required authentication credentials, refer to the authenticate to destination section of the Amazon S3 destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Amazon S3 Base Connection",
"auth": {
"specName": "Assumed Role",
"params": {
"s3Role": "<Add s3 role>"
}
},
"connectionSpec": {
"id": "4fce964d-3f37-408f-9778-e597338a21ee", // Amazon S3 connection spec
"version": "1.0"
}
}'
Response
Amazon S3 Base connection response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Azure Blob Storage
Request
Azure Blob Storage - Base connection request
TIPFor information on how to obtain the required authentication credentials, refer to the authenticate to destination section of the Azure Blob Storage destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Azure Blob Storage Base Connection",
"auth": {
"specName": "ConnectionString",
"params": {
"connectionString": "<Add Azure Blob connection string>"
}
},
"connectionSpec": {
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2", // Azure Blob Storage connection spec
"version": "1.0"
}
}'
Response
Azure Blob Storage - Base connection response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Azure Data Lake Gen 2(ADLS Gen2)
Request
Azure Data Lake Gen 2(ADLS Gen2) - Base connection request
TIPFor information on how to obtain the required authentication credentials, refer to the authenticate to destination section of the Azure Data Lake Gen 2(ADLS Gen2) destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Azure Data Lake Gen 2(ADLS Gen2) Base Connection",
"auth": {
"specName": "Azure Service Principal Auth",
"params": {
"servicePrincipalKey": "<Add servicePrincipalKey>",
"url": "<Add url>",
"tenant": "<Add tenant>",
"servicePrincipalId": "<Add servicePrincipalId>"
}
},
"connectionSpec": {
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1", // Azure Data Lake Gen 2(ADLS Gen2) connection spec
"version": "1.0"
}
}'
Response
Azure Data Lake Gen 2(ADLS Gen2) - Base connection response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Data Landing Zone(DLZ)
Request
Data Landing Zone(DLZ) - Base connection request
TIPNo authentication credentials are required for the Data Landing Zone destination. For more information, refer to the authenticate to destination section of the Data Landing Zone destination documentation page.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Data Landing Zone(DLZ) Base Connection"
}'
Response
Data Landing Zone - Base connection response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Google Cloud Storage
Request
Google Cloud Storage - Base connection request
TIPFor information on how to obtain the required authentication credentials, refer to the authenticate to destination section of the Google Cloud Storage destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Google Cloud Storage Base Connection",
"auth": {
"specName": "Google Cloud Storage authentication credentials",
"params": {
"accessKeyId": "<Add accessKeyId>",
"secretAccessKey": "<Add secret Access Key>"
}
},
"connectionSpec": {
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99", // Google Cloud Storage connection spec
"version": "1.0"
}
}'
Response
Google Cloud Storage - Base connection response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
SFTP
Request
SFTP with password - Base connection request
TIPFor information on how to obtain the required authentication credentials, refer to the authenticate to destination section of the SFTP destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "SFTP with password Base Connection",
"auth": {
"specName": "SFTP with Password",
"params": {
"domain": "<Add domain>",
"username": "<Add username>",
"password": "<Add password>",
"port": "<Add port>"
}
},
"connectionSpec": {
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26", // SFTP connection spec
"version": "1.0"
}
}'
Property
Description
specNameUse
SFTP with Password.domainThe IP address or the domain name of your SFTP storage location.
usernameThe username to log in to your SFTP storage location.
passwordThe password to log in to your SFTP storage location.
portThe port used by your SFTP storage location.
SFTP with SSH key - Base connection request
TIPFor information on how to obtain the required authentication credentials, refer to the authenticate to destination section of the SFTP destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "SFTP with SSH key Base Connection",
"auth": {
"specName": "SFTP with SSH Key",
"params": {
"domain": "<Add domain>",
"username": "<Add username>",
"sshKey": "<Add SSH key>",
"port": "<Add port>"
}
},
"connectionSpec": {
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26", // SFTP connection spec
"version": "1.0"
}
}'
Property
Description
specNameUse
SFTP with Password.domainThe IP address or the domain name of your SFTP storage location.
usernameThe username to log in to your SFTP storage location.
sshKeyThe private SSH key used to log in to your SFTP storage location. The private key must be formatted as a Base64-encoded string and must not be password-protected.
portThe port used by your SFTP storage location.
Response
SFTP - Base connection response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Add encryption to exported files
Optionally, you can add encryption to your exported files. To do this, you need to add items from the encryptionSpecs. See the request example below with the mandatory parameters highlighted:
View encryption specs for cloud storage destinations
"encryptionSpecs": [
{
"name": "File PGP/GPG Encryption",
"type": "FileAsymmetric",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Defines parameters required for capturing user's inputs for encryption",
"type": "object",
"properties": {
"publicKey": {
"description": "Base64 encoded RSA public key",
"type": "string",
"contentEncoding": "base64"
},
"encryptionAlgo": {
"description": "Algorithm for encryption.",
"type": "string",
"default": "PGPGPG",
"enum": [
"PGPGPG",
"NONE"
]
}
},
"required": [
"encryptionAlgo",
"publicKey"
]
}
}
]
Request
Add encryption to base connection - Request
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "SFTP with SSH key Base Connection",
"auth": {
"specName": "SFTP with SSH Key",
"params": {
"domain": "<Add domain>",
"username": "<Add username>",
"sshKey": "<Add SSH key>"
}
},
"encryptionSpecs":{
"specName": "Encryption spec",
"params": {
"encryptionAlgo":"PGPGPG",
"publicKey":"<Add public key>"
}
},
"connectionSpec": {
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26", // SFTP connection spec
"version": "1.0"
}
}'
Response
Add encryption to base connection - Response
{
"id": "900df191-b983-45cd-90d5-4c7a0326d650",
"etag": "\"0500ebe1-0000-0200-0000-63e28d060000\""
}
Note the connection ID from the response. This ID will be required in the next step when creating the target connection.
Create a target connection
Next, you need to create a target connection. Target connections store the export parameters for the exported audiences. Export parameters include export location, file format, compression, and other details. For example, for CSV files, you can select several export options. Get extensive information about all supported CSV export options in the file formatting configurations page.
Refer to the targetSpec properties provided in the destination’s connection spec to understand the supported properties for each destination type. Reference the tabs below for the targetSpec properties of all supported destinations.
Amazon S3
Amazon S3 - Connection spec showing target connection parameters
Note the highlighted lines with inline comments in the connection spec example below, which provide additional information about where to find the target spec parameters in the connection spec. You can see also in the example below which target parameters are not applicable to audience export destinations.
{
"items": [
{
"id": "4fce964d-3f37-408f-9778-e597338a21ee",
"name": "Amazon S3",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { //describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"bucketName": {
"title": "Bucket name",
"description": "Bucket name",
"type": "string",
"pattern": "(?=^.{3,63}$)(?!^(\\d+\\.)+\\d+$)(^(([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])\\.)*([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])$)",
"uiAttributes": {
"tooltip": {
"id": "platform_destinations_connect_s3_bucket",
"fallbackUrl": "http://www.adobe.com/go/destinations-amazon-s3-connection-parameters-en"
}
}
},
"path": {
"title": "Folder path",
"description": "Output path for copying files",
"type": "string",
"pattern": "^[0-9a-zA-Z\/\\!\\-_\\.\\*\\''\\(\\)]*((\\%SEGMENT_(NAME|ID)\\%)?\/?)+$",
"uiAttributes": {
"tooltip": {
"id": "platform_destinations_connect_s3_folderpath",
"fallbackUrl": "http://www.adobe.com/go/destinations-amazon-s3-connection-parameters-en"
}
}
},
"fileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "NOT_CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"CSV",
"JSON",
"PARQUET"
]
},
"datasetFileType": { // does not apply to audience export destinations
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {
"conditional": {
"field": "fileType",
"operator": "EQUALS",
"value": "CSV"
},
"title": "CSV Options",
"description": "Select your CSV options",
"type": "object",
"properties": {
"quote": {
"title": "Quote Character",
"description": "Select your Quote character",
"type": "string",
"oneOf": [
{
"title": "Double Quotes (\")",
"const": "\""
},
{
"title":"Null Character (\\)",
"const": ""
}
],
"default": ""
},
"escape": {
"title": "Escape Character",
"description": "Select your Escape character",
"type": "string",
"oneOf": [
{
"title": "Back Slash (\\)",
"const": "\\"
},
{
"title": "Single Quote (')",
"const": "'"
}
],
"default": "\\"
},
"delimiter": {
"title": "Delimiter",
"description": "Select your Delimiter",
"type": "string",
"oneOf": [
{
"title": "Comma (,)",
"const": ","
},
{
"title": "Tab (\\t)",
"const": "\t"
},
{
"title": "Pipe (|)",
"const": "|"
},
{
"title": "Semicolon (;)",
"const": ";"
},
{
"title": "Colon (:)",
"const": ":"
}
],
"default": ","
},
"nullValue": {
"title": "Null Value",
"description": "Select the output value of 'null' fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": "null"
},
"emptyValue": {
"title": "Empty Value",
"description": "Select the output value of blank fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": ""
}
}
},
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"bucketName",
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
Azure Blob Storage
Azure Blob Storage - Connection spec showing target connection parameters
Note the highlighted lines with inline comments in the connection spec example below, which provide additional information about where to find the target spec parameters in the connection spec. You can see also in the example below which target parameters are not applicable to audience export destinations.
{
"items": [
{
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2",
"name": "Azure Blob Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"path": {
"title": "Folder path",
"description": "Output path (relative) indicating where to upload the data",
"type": "string",
"pattern": "^[0-9a-zA-Z\/\\!\\-_\\.\\*\\'\\(\\)]+$"
},
"container": {
"title": "Container",
"description": "Container within the storage where to upload the data",
"type": "string",
"pattern": "^[a-z0-9](?!.*--)[a-z0-9-]{1,61}[a-z0-9]$"
},
"fileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "NOT_CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"CSV",
"JSON",
"PARQUET"
]
},
"datasetFileType": { // does not apply to audience export destinations
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {
"conditional": {
"field": "fileType",
"operator": "EQUALS",
"value": "CSV"
},
"title": "CSV Options",
"description": "Select your CSV options",
"type": "object",
"properties": {
"quote": {
"title": "Quote Character",
"description": "Select your Quote character",
"type": "string",
"oneOf": [
{
"title": "Double Quotes (\")",
"const": "\""
},
{
"title":"Null Character (\\)",
"const": ""
}
],
"default": ""
},
"escape": {
"title": "Escape Character",
"description": "Select your Escape character",
"type": "string",
"oneOf": [
{
"title": "Back Slash (\\)",
"const": "\\"
},
{
"title": "Single Quote (')",
"const": "'"
}
],
"default": "\\"
},
"delimiter": {
"title": "Delimiter",
"description": "Select your Delimiter",
"type": "string",
"oneOf": [
{
"title": "Comma (,)",
"const": ","
},
{
"title": "Tab (\\t)",
"const": "\t"
},
{
"title": "Pipe (|)",
"const": "|"
},
{
"title": "Semicolon (;)",
"const": ";"
},
{
"title": "Colon (:)",
"const": ":"
}
],
"default": ","
},
"nullValue": {
"title": "Null Value",
"description": "Select the output value of 'null' fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": "null"
},
"emptyValue": {
"title": "Empty Value",
"description": "Select the output value of blank fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": ""
}
}
},
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"container",
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
Azure Data Lake Gen 2(ADLS Gen2)
Azure Data Lake Gen 2(ADLS Gen2) - Connection spec showing target connection parameters
Note the highlighted lines with inline comments in the connection spec example below, which provide additional information about where to find the target spec parameters in the connection spec. You can see also in the example below which target parameters are not applicable to audience export destinations.
{
"items": [
{
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1",
"name": "Azure Data Lake Gen2",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"path": {
"title": "Folder path",
"description": "Enter the path to your Azure Data Lake Storage folder",
"type": "string"
},
"fileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "NOT_CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"CSV",
"JSON",
"PARQUET"
]
},
"datasetFileType": { // does not apply to audience export destinations
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {
"conditional": {
"field": "fileType",
"operator": "EQUALS",
"value": "CSV"
},
"title": "CSV Options",
"description": "Select your CSV options",
"type": "object",
"properties": {
"quote": {
"title": "Quote Character",
"description": "Select your Quote character",
"type": "string",
"oneOf": [
{
"title": "Double Quotes (\")",
"const": "\""
},
{
"title":"Null Character (\\)",
"const": ""
}
],
"default": ""
},
"escape": {
"title": "Escape Character",
"description": "Select your Escape character",
"type": "string",
"oneOf": [
{
"title": "Back Slash (\\)",
"const": "\\"
},
{
"title": "Single Quote (')",
"const": "'"
}
],
"default": "\\"
},
"delimiter": {
"title": "Delimiter",
"description": "Select your Delimiter",
"type": "string",
"oneOf": [
{
"title": "Comma (,)",
"const": ","
},
{
"title": "Tab (\\t)",
"const": "\t"
},
{
"title": "Pipe (|)",
"const": "|"
},
{
"title": "Semicolon (;)",
"const": ";"
},
{
"title": "Colon (:)",
"const": ":"
}
],
"default": ","
},
"nullValue": {
"title": "Null Value",
"description": "Select the output value of 'null' fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": "null"
},
"emptyValue": {
"title": "Empty Value",
"description": "Select the output value of blank fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": ""
}
}
},
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
Data Landing Zone(DLZ)
Data Landing Zone(DLZ) - Connection spec showing target connection parameters
Note the highlighted lines with inline comments in the connection spec example below, which provide additional information about where to find the target spec parameters in the connection spec. You can see also in the example below which target parameters are not applicable to audience export destinations.
"items": [
{
"id": "10440537-2a7b-4583-ac39-ed38d4b848e8",
"name": "Data Landing Zone",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [],
"encryptionSpecs": [],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"path": {
"title": "Folder path",
"description": "Enter the path to your Azure Data Lake Storage folder",
"type": "string"
},
"fileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "NOT_CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"CSV",
"JSON",
"PARQUET"
]
},
"datasetFileType": { // does not apply to audience export destinations
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {
"conditional": {
"field": "fileType",
"operator": "EQUALS",
"value": "CSV"
},
"title": "CSV Options",
"description": "Select your CSV options",
"type": "object",
"properties": {
"quote": {
"title": "Quote Character",
"description": "Select your Quote character",
"type": "string",
"oneOf": [
{
"title": "Double Quotes (\")",
"const": "\""
},
{
"title":"Null Character (\\)",
"const": ""
}
],
"default": ""
},
"escape": {
"title": "Escape Character",
"description": "Select your Escape character",
"type": "string",
"oneOf": [
{
"title": "Back Slash (\\)",
"const": "\\"
},
{
"title": "Single Quote (')",
"const": "'"
}
],
"default": "\\"
},
"delimiter": {
"title": "Delimiter",
"description": "Select your Delimiter",
"type": "string",
"oneOf": [
{
"title": "Comma (,)",
"const": ","
},
{
"title": "Tab (\\t)",
"const": "\t"
},
{
"title": "Pipe (|)",
"const": "|"
},
{
"title": "Semicolon (;)",
"const": ";"
},
{
"title": "Colon (:)",
"const": ":"
}
],
"default": ","
},
"nullValue": {
"title": "Null Value",
"description": "Select the output value of 'null' fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": "null"
},
"emptyValue": {
"title": "Empty Value",
"description": "Select the output value of blank fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": ""
}
}
},
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
Google Cloud Storage
Google Cloud Storage - Connection spec showing target connection parameters
Note the highlighted lines with inline comments in the connection spec example below, which provide additional information about where to find the target spec parameters in the connection spec. You can see also in the example below which target parameters are not applicable to audience export destinations.
{
"items": [
{
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99",
"name": "Google Cloud Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"bucketName": {
"title": "Bucket name",
"description": "Bucket name",
"type": "string",
"pattern": "(?!^goog.*$)(?!^.*g(o|0)(o|0)gle.*$)(((?=^.{3,63}$)(^([a-z0-9]|[a-z0-9][a-z0-9\\-_]*)[a-z0-9]$))|((?=^.{3,222}$)(?!^(\\d+\\.)+\\d+$)(^(([a-z0-9]{1,63}|[a-z0-9][a-z0-9\\-_]{1,61}[a-z0-9])\\.)*([a-z0-9]{1,63}|[a-z0-9][a-z0-9\\-_]{1,61}[a-z0-9])$)))"
},
"path": {
"title": "Folder path",
"description": "Output path for copying files",
"type": "string",
"pattern": "^[0-9a-zA-Z\/\\!\\-_\\.\\*\\''\\(\\)]*((\\%SEGMENT_(NAME|ID)\\%)?\/?)+$"
},
"fileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "NOT_CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"CSV",
"JSON",
"PARQUET"
]
},
"datasetFileType": { // does not apply to audience export destinations
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {
"conditional": {
"field": "fileType",
"operator": "EQUALS",
"value": "CSV"
},
"title": "CSV Options",
"description": "Select your CSV options",
"type": "object",
"properties": {
"quote": {
"title": "Quote Character",
"description": "Select your Quote character",
"type": "string",
"oneOf": [
{
"title": "Double Quotes (\")",
"const": "\""
},
{
"title":"Null Character (\\)",
"const": ""
}
],
"default": ""
},
"escape": {
"title": "Escape Character",
"description": "Select your Escape character",
"type": "string",
"oneOf": [
{
"title": "Back Slash (\\)",
"const": "\\"
},
{
"title": "Single Quote (')",
"const": "'"
}
],
"default": "\\"
},
"delimiter": {
"title": "Delimiter",
"description": "Select your Delimiter",
"type": "string",
"oneOf": [
{
"title": "Comma (,)",
"const": ","
},
{
"title": "Tab (\\t)",
"const": "\t"
},
{
"title": "Pipe (|)",
"const": "|"
},
{
"title": "Semicolon (;)",
"const": ";"
},
{
"title": "Colon (:)",
"const": ":"
}
],
"default": ","
},
"nullValue": {
"title": "Null Value",
"description": "Select the output value of 'null' fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": "null"
},
"emptyValue": {
"title": "Empty Value",
"description": "Select the output value of blank fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": ""
}
}
},
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"bucketName",
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
SFTP
SFTP - Connection spec showing target connection parameters
Note the highlighted lines with inline comments in the connection spec example below, which provide additional information about where to find the target spec parameters in the connection spec. You can see also in the example below which target parameters are not applicable to audience export destinations.
{
"items": [
{
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26",
"name": "SFTP",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"remotePath": {
"title": "Folder path",
"description": "Enter your folder path",
"type": "string"
},
"fileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "NOT_CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"CSV",
"JSON",
"PARQUET"
]
},
"datasetFileType": { // does not apply to audience export destinations
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {
"conditional": {
"field": "fileType",
"operator": "EQUALS",
"value": "CSV"
},
"title": "CSV Options",
"description": "Select your CSV options",
"type": "object",
"properties": {
"quote": {
"title": "Quote Character",
"description": "Select your Quote character",
"type": "string",
"oneOf": [
{
"title": "Double Quotes (\")",
"const": "\""
},
{
"title":"Null Character (\\)",
"const": ""
}
],
"default": ""
},
"escape": {
"title": "Escape Character",
"description": "Select your Escape character",
"type": "string",
"oneOf": [
{
"title": "Back Slash (\\)",
"const": "\\"
},
{
"title": "Single Quote (')",
"const": "'"
}
],
"default": "\\"
},
"delimiter": {
"title": "Delimiter",
"description": "Select your Delimiter",
"type": "string",
"oneOf": [
{
"title": "Comma (,)",
"const": ","
},
{
"title": "Tab (\\t)",
"const": "\t"
},
{
"title": "Pipe (|)",
"const": "|"
},
{
"title": "Semicolon (;)",
"const": ";"
},
{
"title": "Colon (:)",
"const": ":"
}
],
"default": ","
},
"nullValue": {
"title": "Null Value",
"description": "Select the output value of 'null' fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": "null"
},
"emptyValue": {
"title": "Empty Value",
"description": "Select the output value of blank fields",
"type": "string",
"oneOf": [
{
"title": "null",
"const": "null"
},
{
"title": "\"\"",
"const": "\"\""
},
{
"title": "Empty String",
"const": ""
}
],
"default": ""
}
}
},
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"GZIP",
"NONE"
]
}
},
"required": [
"remotePath",
"datasetFileType",
"compression",
"fileType"
]
},
//...
By using the above spec, you can construct a target connection request specific to your desired cloud storage destination, as shown in the tabs below.
Amazon S3
Request
Amazon S3 - Target connection request
TIPFor information on how to obtain the required target parameters, refer to the fill in destination details section of the Amazon S3 destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Amazon S3 Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"bucketName": "your-bucket-name",
"path": "folder/subfolder",
"compression": "NONE",
"fileType": "JSON"
},
"connectionSpec": {
"id": "4fce964d-3f37-408f-9778-e597338a21ee", // Amazon S3 connection spec id
"version": "1.0"
}
}'
Amazon S3 - Target connection request with CSV options
TIPFor extensive information about the available CSV options for file export, refer to the file formatting configurations page .
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name":"Amazon S3 Target Connection",
"baseConnectionId":"<FROM_STEP_CREATE_BASE_CONNECTION>",
"params":{
"mode":"Server-to-server",
"bucketName":"your-bucket-name",
"path":"folder/subfolder",
"compression":"GZIP",
"fileType":"CSV",
"csvOptions":{
"nullValue":"null",
"emptyValue":"",
"escape":"\\",
"quote":"",
"delimiter":","
}
},
"connectionSpec":{
"id":"4fce964d-3f37-408f-9778-e597338a21ee",
"version":"1.0"
}
}'
Response
Target connection - Response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Azure Blob Storage
Request
Azure Blob Storage - Target connection request
TIPFor information on how to obtain the required target parameters, refer to the fill in destination details section of the Azure Blob Storage destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Azure Blob Storage Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"container": "your-container-name",
"path": "folder/subfolder",
"compression": "NONE",
"fileType": "JSON"
},
"connectionSpec": {
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2", // Azure Blob Storage connection spec id
"version": "1.0"
}
}'
Azure Blob Storage - Target connection request with CSV options
TIPFor extensive information about the available CSV options for file export, refer to the file formatting configurations page .
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name":"Azure Blob Storage Target Connection",
"baseConnectionId":"<FROM_STEP_CREATE_BASE_CONNECTION>",
"params":{
"mode":"Server-to-server",
"bucketName":"your-bucket-name",
"path":"folder/subfolder",
"compression":"GZIP",
"fileType":"CSV",
"csvOptions":{
"nullValue":"null",
"emptyValue":"",
"escape":"\\",
"quote":"",
"delimiter":","
}
},
"connectionSpec":{
"id":"6d6b59bf-fb58-4107-9064-4d246c0e5bb2",
"version":"1.0"
}
}'
Response
Target connection - Response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Azure Data Lake Gen 2(ADLS Gen2)
Request
Azure Data Lake Gen 2(ADLS Gen2) - Target connection request
TIPFor information on how to obtain the required target parameters, refer to the fill in destination details section of the Azure Data Lake Gen 2(ADLS Gen2) destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Azure Data Lake Gen 2(ADLS Gen2) Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"path": "folder/subfolder",
"compression": "NONE",
"fileType": "JSON"
},
"connectionSpec": {
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1", // Azure Data Lake Gen 2(ADLS Gen2) connection spec id
"version": "1.0"
}
}'
Azure Data Lake Gen 2(ADLS Gen2) - Target connection request with CSV options
TIPFor extensive information about the available CSV options for file export, refer to the file formatting configurations page .
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name":"Azure Data Lake Gen 2(ADLS Gen2)",
"baseConnectionId":"<FROM_STEP_CREATE_BASE_CONNECTION>",
"params":{
"mode":"Server-to-server",
"bucketName":"your-bucket-name",
"path":"folder/subfolder",
"compression":"GZIP",
"fileType":"CSV",
"csvOptions":{
"nullValue":"null",
"emptyValue":"",
"escape":"\\",
"quote":"",
"delimiter":","
}
},
"connectionSpec":{
"id":"be2c3209-53bc-47e7-ab25-145db8b873e1",
"version":"1.0"
}
}'
Response
Target connection - Response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Data Landing Zone(DLZ)
Request
Data Landing Zone - Target connection request
TIPFor information on how to obtain the required target parameters, refer to the fill in destination details section of the Data Landing Zone destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Data Landing Zone Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"path": "folder/subfolder",
"compression": "NONE",
"fileType": "JSON"
},
"connectionSpec": {
"id": "10440537-2a7b-4583-ac39-ed38d4b848e8", // Data Landing Zone connection spec id
"version": "1.0"
}
}'
Data Landing Zone - Target connection request with CSV options
TIPFor extensive information about the available CSV options for file export, refer to the file formatting configurations page .
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name":"Data Landing Zone Target Connection",
"baseConnectionId":"<FROM_STEP_CREATE_BASE_CONNECTION>",
"params":{
"mode":"Server-to-server",
"bucketName":"your-bucket-name",
"path":"folder/subfolder",
"compression":"GZIP",
"fileType":"CSV",
"csvOptions":{
"nullValue":"null",
"emptyValue":"",
"escape":"\\",
"quote":"",
"delimiter":","
}
},
"connectionSpec":{
"id":"10440537-2a7b-4583-ac39-ed38d4b848e8",
"version":"1.0"
}
}'
Response
Target connection - Response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Google Cloud Storage
Request
Google Cloud Storage - Target connection request
TIPFor information on how to obtain the required target parameters, refer to the fill in destination details section of the Google Cloud Storage destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Google Cloud Storage Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"bucketName": "your-bucket-name",
"path": "folder/subfolder",
"compression": "NONE",
"fileType": "JSON"
},
"connectionSpec": {
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99", // Google Cloud Storage connection spec id
"version": "1.0"
}
}'
Google Cloud Storage - Target connection request with CSV options
TIPFor extensive information about the available CSV options for file export, refer to the file formatting configurations page .
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name":"Google Cloud Storage Target Connection",
"baseConnectionId":"<FROM_STEP_CREATE_BASE_CONNECTION>",
"params":{
"mode":"Server-to-server",
"bucketName":"your-bucket-name",
"path":"folder/subfolder",
"compression":"GZIP",
"fileType":"CSV",
"csvOptions":{
"nullValue":"null",
"emptyValue":"",
"escape":"\\",
"quote":"",
"delimiter":","
}
},
"connectionSpec":{
"id":"c5d93acb-ea8b-4b14-8f53-02138444ae99",
"version":"1.0"
}
}'
Response
Target connection - Response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
SFTP
Request
SFTP - Target connection request
TIPFor information on how to obtain the required target parameters, refer to the fill in destination details section of the SFTP destination documentation page.
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "SFTP Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"remotePath": "folder/subfolder",
"compression": "NONE",
"fileType": "JSON"
},
"connectionSpec": {
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26", // SFTP connection spec id
"version": "1.0"
}
}'
SFTP - Target connection request with CSV options
TIPFor extensive information about the available CSV options for file export, refer to the file formatting configurations page .
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name":"SFTP Target Connection",
"baseConnectionId":"<FROM_STEP_CREATE_BASE_CONNECTION>",
"params":{
"mode":"Server-to-server",
"bucketName":"your-bucket-name",
"path":"folder/subfolder",
"compression":"GZIP",
"fileType":"CSV",
"csvOptions":{
"nullValue":"null",
"emptyValue":"",
"escape":"\\",
"quote":"",
"delimiter":","
}
},
"connectionSpec":{
"id":"36965a81-b1c6-401b-99f8-22508f1e6a26",
"version":"1.0"
}
}'
Response
Target connection - Response
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Note the target connection ID from the response. This ID will be required in the next step when creating the dataflow to export audiences.
A successful response returns the ID (id) of the newly target source connection and an etag. Note down the target connection ID as you will need it later when creating the dataflow.
Create a dataflow
The next step in the destination configuration is to create a dataflow. A dataflow ties together previously created entities and also provides options for configuring the audience export schedule. To create the dataflow, use the payloads below, depending on your desired cloud storage destination, and replace the flow entity IDs from previous steps. Note that in this step, you are not adding any information related to attribute or identity mapping to the dataflow. That will follow in the next step.
Amazon S3
Request
Create audience export dataflow to Amazon S3 destination - Request
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate audiences to an Amazon S3 cloud storage destination",
"description": "This operation creates a dataflow to export audiences to an Amazon S3 cloud storage destination",
"flowSpec": {
"id": "1a0514a6-33d4-4c7f-aff8-594799c47549", // Amazon S3 flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": []
}'
Response
Create dataflow - Response
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Azure Blob Storage
Request
Create audience export dataflow to Azure Blob Storage destination - Request
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate audiences to an Azure Blob Storage cloud storage destination",
"description": "This operation creates a dataflow to export audiences to an Azure Blob Storage cloud storage destination",
"flowSpec": {
"id": "752d422f-b16f-4f0d-b1c6-26e448e3b388", // Azure Blob Storage flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": [
{
"name": "GeneralTransform",
"params": {
"mandatoryFields": [],
"primaryFields": [],
"profileMapping": {},
"segmentSelectors": {
"selectors": []
}
}
}
]
}'
Response
Create dataflow - Response
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Azure Data Lake Gen 2(ADLS Gen2)
Request
Create audience export dataflow to Azure Data Lake Gen 2(ADLS Gen2) destination - Request
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate audiences to an Azure Data Lake Gen 2(ADLS Gen2) cloud storage destination",
"description": "This operation creates a dataflow to export audiences to an Azure Data Lake Gen 2(ADLS Gen2) cloud storage destination",
"flowSpec": {
"id": "17be2013-2549-41ce-96e7-a70363bec293", // Azure Data Lake Gen 2(ADLS Gen2) flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": []
}'
Response
Create dataflow - Response
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Data Landing Zone(DLZ)
Request
Create audience export dataflow to Data Landing Zone destination - Request
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate audiences to a Data Landing Zone cloud storage destination",
"description": "This operation creates a dataflow to export audiences to a Data Landing Zone cloud storage destination",
"flowSpec": {
"id": "cd2fc47e-e838-4f38-a581-8fff2f99b63a", // Data Landing Zone flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": []
}'
Response
Create dataflow - Response
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Google Cloud Storage
Request
Create audience export dataflow to Google Cloud Storage destination - Request
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate audiences to a Google Cloud Storage cloud storage destination",
"description": "This operation creates a dataflow to export audiences to a Google Cloud Storage destination",
"flowSpec": {
"id": "585c15c4-6cbf-4126-8f87-e26bff78b657", // Google Cloud Storage flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": []
}'
Response
Create dataflow - Response
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
SFTP
Request
Create audience export dataflow to SFTP destination - Request
Note the highlighted lines with inline comments in the request example, which provide additional information. Remove the inline comments in the request when copy-pasting the request into your terminal of choice.
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate audiences to an SFTP cloud storage destination",
"description": "This operation creates a dataflow to export audiences to an SFTP cloud storage destination",
"flowSpec": {
"id": "fd36aaa4-bf2b-43fb-9387-43785eeeb799", // SFTP flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": []
}'
Response
Create dataflow - Response
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Note the Dataflow ID from the response. This ID will be required in later steps.
Add audiences to the export
In this step, you can also select which audiences you want to export to the destination. For extensive information about this step and the request format to add an audience to the dataflow, view the examples in the Update a destination dataflow section of the API reference documentation.
Set up attribute and identity mapping
After creating your dataflow, you need to set up mapping for the attributes and identities that you would like to export. This consists of three steps, listed below:
- Create an input schema
- Create an output schema
- Set up a mapping set to connect the created schemas
For example, to obtain the following mapping shown in the UI, you would need to go through the three steps listed above and detailed in the next headings.
Create an input schema
To create an input schema, you first need to retrieve your union schema and the identities that can be exported to the destination. This is the schema of attributes and identities which you can select as your source mapping.
View below examples of requests and responses to retrieve attributes and identities.
Request to get attributes
Get available attributes from your union schema - Request
curl --location --request GET 'https://platform.adobe.io/data/core/ups/config/entityTypes/_xdm.context.profile?property=fullSchema==true&property=includeRelationshipDescriptors==true' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
Response
Get available attributes from your union schema - Response
The response below has been shortened for brevity.
"person": {
"title": "Person",
"description": "An individual actor, contact, or owner.",
"meta:referencedFrom": [
"https://ns.adobe.com/xdm/context/person"
],
"type": "object",
"meta:xdmType": "object",
"properties": {
"birthDate": {
"meta:xdmType": "date",
"type": "string",
"format": "date",
"title": "Birth date(YYYY-MM-DD)",
"description": "The full date a person was born."
},
"birthDayAndMonth": {
"type": "string",
"meta:xdmType": "string",
"title": "Birth date (MM-DD)",
"description": "The day and month a person was born, in the format MM-DD. This field should be used when the day and month of a person's birth is known, but not the year."
},
"birthYear": {
"meta:xdmType": "short",
"type": "integer",
"minimum": 1,
"maximum": 32767,
"title": "Birth year",
"description": "The year a person was born including the century, for example, 1983. This field should be used when only the person's age is known, not the full birth date."
},
"gender": {
"type": "string",
"meta:xdmType": "string",
"title": "Gender",
"description": "Gender identity of the person.\n",
"enum": [
"male",
"female",
"not_specified",
"non_specific"
],
"meta:enum": {
"male": "Male",
"female": "Female",
"not_specified": "Not Specified",
"non_specific": "Non-specific"
},
"default": "not_specified"
},
"maritalStatus": {
"type": "string",
"meta:xdmType": "string",
"title": "Marital Status",
"description": "Describes a person's relationship with a significant other.",
"enum": [
"married",
"single",
"divorced",
"widowed",
"not_specified"
],
"meta:enum": {
"divorced": "Divorced",
"not_specified": "Not Specified",
"married": "Married",
"single": "Single",
"widowed": "Widowed"
},
"default": "not_specified"
},
"name": {
"title": "Full name",
"description": "The person's full name.",
"meta:referencedFrom": [
"https://ns.adobe.com/xdm/context/person-name"
],
"type": "object",
"meta:xdmType": "object",
"properties": {
"courtesyTitle": {
"type": "string",
"meta:xdmType": "string",
"title": "Courtesy title",
"description": "Normally an abbreviation of a persons title, honorific, or salutation. The `courtesyTitle` is used in front of full or last name in opening texts. For example, Mr. Miss. or Dr."
},
"firstName": {
"type": "string",
"meta:xdmType": "string",
"title": "First name",
"description": "The first audience of the name in the writing order most commonly accepted in the language of the name. In many cultures this is the preferred personal or given name. The `firstName` and `lastName` properties have been introduced to maintain compatibility with existing systems that model names in a simplified, non-semantic, and non-internationalizable way. Using `xdm:fullName` is always preferable."
},
"fullName": {
"type": "string",
"meta:xdmType": "string",
"title": "Full name",
"description": "The full name of the person, in writing order most commonly accepted in the language of the name."
},
"lastName": {
"type": "string",
"meta:xdmType": "string",
"title": "Last name",
"description": "The last audience of the name in the writing order most commonly accepted in the language of the name. In many cultures this is the inherited family name, surname, patronymic, or matronymic name. The `firstName` and `lastName` properties have been introduced to maintain compatibility with existing systems that model names in a simplified, non-semantic, and non-internationalizable way. Using `xdm:fullName` is always preferable."
},
"middleName": {
"type": "string",
"meta:xdmType": "string",
"title": "Middle name",
"description": "Middle, alternative, or additional names supplied between the first name and last name."
},
"suffix": {
"type": "string",
"meta:xdmType": "string",
"title": "Suffix",
"description": "A group of letters provided after a person's name to provide additional information. The `suffix` is used at the end of someones name. For example Jr., Sr., M.D., PhD, I, II, III, etc."
}
}
},
"nationality": {
"type": "string",
"meta:xdmType": "string",
"title": "Nationality",
"description": "The legal relationship between a person and their state represented using the ISO 3166-1 Alpha-2 code."
},
"taxId": {
"type": "string",
"meta:xdmType": "string",
"title": "Tax ID",
"description": "The Tax / Fiscal ID of the person, e.g. the TIN in the US or the CIF/NIF in Spain.",
"meta:status": "deprecated"
},
"type": {
"type": "string",
"meta:xdmType": "string",
"title": "Type",
"description": "The type of individual in different business contexts like B2C."
}
}
}
Request to get identities
Get available identities that you can use in the mapping step
curl --location --request GET 'https://platform.adobe.io/data/core/idnamespace/identities' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
Response
View available identities to use in the input schema
The response returns the identities that you can use when creating the input schema. Note that this response returns both standard and custom identity namespaces that you set up in Experience Platform.
[
{
"updateTime": 1551688425455,
"code": "CORE",
"status": "ACTIVE",
"description": "Adobe Audience Manger UUID",
"id": 0,
"createTime": 1551688425455,
"idType": "COOKIE",
"namespaceType": "Standard",
"name": "CORE",
"custom": false
},
{
"updateTime": 1551688425455,
"code": "ECID",
"status": "ACTIVE",
"description": "Adobe Experience Cloud ID",
"id": 4,
"createTime": 1551688425455,
"idType": "COOKIE",
"namespaceType": "Standard",
"name": "ECID",
"custom": false
},
{
"updateTime": 1551688425455,
"code": "AdCloud",
"status": "ACTIVE",
"description": "Adobe AdCloud - ID Syncing Partner",
"id": 411,
"createTime": 1551688425455,
"idType": "COOKIE",
"namespaceType": "Standard",
"name": "AdCloud",
"custom": false
},
{
"updateTime": 1551688425455,
"code": "Email_LC_SHA256",
"status": "ACTIVE",
"description": "Email addresses need to be hashed using SHA256 and lowercased. Please also note that leading and trailing spaces need to be trimmed before an email address is normalized. You won't be able to change this setting later",
"id": 11,
"createTime": 1551688425455,
"idType": "Email",
"namespaceType": "Standard",
"name": "Emails (SHA256, lowercased)",
"custom": false,
"hashFunction": "SHA256",
"transform": "lowercase"
},
{
"updateTime": 1597996026101,
"code": "Phone_E.164",
"status": "ACTIVE",
"description": "Namespace for raw phone numbers in E.164 format. + sign is required",
"id": 17,
"createTime": 1597996026101,
"idType": "Phone",
"namespaceType": "Standard",
"name": "Phone (E.164)",
"custom": false
},
{
"updateTime": 1551688425455,
"code": "GAID",
"status": "ACTIVE",
"description": "This datasource is associated to a Google Ad ID",
"id": 20914,
"createTime": 1551688425455,
"idType": "DEVICE",
"namespaceType": "Standard",
"name": "Google Ad ID (GAID)",
"custom": false
},
{
"updateTime": 1476993749000,
"code": "IDFA",
"status": "ACTIVE",
"description": "Apple ID for Advertisers. See: https://support.apple.com/en-us/HT202074 for more info.",
"id": 20915,
"createTime": 1476993749000,
"idType": "DEVICE",
"namespaceType": "Standard",
"name": "Apple IDFA (ID for Advertisers)",
"custom": false
},
{
"updateTime": 1551688425455,
"code": "AAID",
"status": "ACTIVE",
"description": "Adobe Analytics (Legacy ID)",
"id": 10,
"createTime": 1551688425455,
"idType": "COOKIE",
"namespaceType": "Standard",
"name": "Adobe Analytics (Legacy ID)",
"custom": false
},
{
"updateTime": 1551688425455,
"code": "Email",
"status": "ACTIVE",
"description": "Email",
"id": 6,
"createTime": 1551688425455,
"idType": "Email",
"namespaceType": "Standard",
"name": "Email",
"custom": false
},
{
"updateTime": 1551688425455,
"code": "WAID",
"status": "ACTIVE",
"description": "Windows AID",
"id": 8,
"createTime": 1551688425455,
"idType": "DEVICE",
"namespaceType": "Standard",
"name": "Windows AID",
"custom": false
},
{
"updateTime": 1551688425455,
"code": "TNTID",
"status": "ACTIVE",
"description": "Adobe Target (TNTID)",
"id": 9,
"createTime": 1551688425455,
"idType": "COOKIE",
"namespaceType": "Standard",
"name": "TNTID",
"custom": false
},
{
"updateTime": 1556676464714,
"code": "Google",
"status": "ACTIVE",
"id": 771,
"createTime": 1556676464714,
"idType": "COOKIE",
"namespaceType": "Integration",
"name": "Google",
"custom": false
},
{
"updateTime": 1604597776019,
"code": "Phone_SHA256_E.164",
"status": "ACTIVE",
"description": "Phone numbers need to be hashed using SHA256 without any dashes. Hash should be completed by customers on raw phone numbers in E.164 format. Please note that some destinations may have different phone number formatting requirements. Refer to documentation or consult your Adobe representative",
"id": 18,
"createTime": 1604597776019,
"idType": "Phone",
"namespaceType": "Standard",
"name": "Phone (SHA256_E.164)",
"custom": false,
"hashFunction": "SHA256"
},
{
"updateTime": 1604597776019,
"code": "Phone_SHA256",
"status": "ACTIVE",
"description": "Remove symbols, letters, and any leading zeroes before hashing. Prefix the country code before hashing. Please note that some destinations may have different phone number formatting requirements. Refer to documentation or consult your Adobe representative",
"id": 15,
"createTime": 1597995542594,
"idType": "Phone",
"namespaceType": "Standard",
"name": "Phone (SHA256)",
"custom": false,
"hashFunction": "SHA256"
},
{
"updateTime": 1551688425455,
"code": "Phone",
"status": "ACTIVE",
"description": "Phone",
"id": 7,
"createTime": 1551688425455,
"idType": "PHONE_NUMBER",
"namespaceType": "Standard",
"name": "Phone",
"custom": false
}
]
Next, you need to copy the response from above and use it to create your input schema. You can copy the entire JSON response from the response above and place it into the jsonSchema object indicated below.
Request to create input schema
Create an input schema - request
curl --location --request POST 'https://platform.adobe.io/data/foundation/conversion/schemas/' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'Content-Type: application/json' \
--data-raw '{
"name":"Sample Schema for Flow Mapping",
"jsonSchema":{...insert response from union schema response here}
'
Response
Create an input schema - Response
{
"id":"8db56468c2ab475dbf17c2621f92c0f8",
"version":0,
"jsonSchema":{
"title":"XDM Individual Profile",
"description":"An XDM Individual Profile forms a singular representation of the attributes and interests of both identified and partially-identified individuals. Less-identified profiles may contain only anonymous behavioral signals, such as browser cookies, while highly-identified profiles may contain detailed personal information such as name, date of birth, location, and email address. As a profile grows, it becomes a robust repository of personal information, identification information, contact details, and communication preferences for an individual.",
"type":"object",
"properties":{ // this section returns the contents that you have added to the jsonSchema object in the request
}
}
}
The ID in the response represents the unique identifier of the input schema that you have created. Copy the ID from the response as you will reuse this in a later step.
Create an output schema
Next, you must set up the output schema for your export. First, you need to find and inspect your existing partner schema.
Request
Request to get partner schema for the output schema
Note that the example below uses the connection spec ID for Amazon S3. Please replace this value with the connection spec ID specific to your destination.
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/4fce964d-3f37-408f-9778-e597338a21ee' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
Response with an example schema
Inspect the response you obtain when performing the call above. You need to drill down into the response to find the object targetSpec.attributes.partnerSchema.jsonSchema
Response to get partner schema for the output schema
{
"title":"defaultschema",
"type":"object",
"properties":{
"attributes":{
"type":"object",
"meta:xdmType":"map",
"additionalProperties":{
"type":"object",
"properties":{
"value":{
"type":"string",
"title":"Value"
}
},
"meta:xdmType":"object"
}
},
"identityMap":{
"type":"object",
"meta:xdmField":"xdm:identityMap",
"meta:xdmType":"map",
"additionalProperties":{
"type":"array",
"items":{
"type":"object",
"properties":{
"id":{
"type":"string",
"title":"Identifier",
"description":"Identity of the consumer in the related namespace.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:id"
},
"primary":{
"type":"boolean",
"title":"Primary",
"default":false,
"description":"Indicates this identity is the preferred identity. Is used as a hint to help systems better organize how identities are queried.",
"meta:xdmType":"boolean",
"meta:xdmField":"xdm:primary"
},
"authenticatedState":{
"enum":[
"ambiguous",
"authenticated",
"loggedOut"
],
"type":"string",
"default":"ambiguous",
"meta:enum":{
"ambiguous":"Ambiguous",
"loggedOut":"User was identified by a login action at some point of time previously, but is not currently logged in.",
"authenticated":"User identified by a login or similar action that was valid at the time of the event observation."
},
"description":"The state this identity is authenticated as for this observed ExperienceEvent.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:authenticatedState"
}
},
"meta:xdmType":"object",
"meta:referencedFrom":"https://ns.adobe.com/xdm/context/identityitem"
},
"meta:xdmType":"array"
}
},
"segmentMembership":{
"title":"Segment membership map",
"type":"object",
"meta:xdmField":"xdm:segmentMembership",
"meta:xdmType":"map",
"additionalProperties":{
"type":"object",
"title":"Segment membership per namespace",
"meta:xdmType":"map",
"additionalProperties":{
"type":"object",
"properties":{
"status":{
"enum":[
"realized",
"exited"
],
"type":"string",
"title":"Status",
"default":"realized",
"meta:enum":{
"exited":"Entity is exiting the segment.",
"realized":"Entity is entering the segment."
},
"description":"Is the audience participation realized as part of the current request.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:status"
},
"payload":{
"type":"object",
"title":"Payload",
"required":[
"payloadType"
],
"properties":{
"payloadType":{
"enum":[
"boolean",
"number",
"propensity",
"string"
],
"type":"string",
"title":"Payload Type",
"meta:enum":{
"number":"Number",
"string":"String",
"boolean":"Boolean",
"propensity":"Propensity"
},
"description":"The type of payload.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:payloadType"
},
"payloadNumberValue":{
"type":"number",
"title":"Value",
"description":"The number.",
"meta:xdmType":"number",
"meta:xdmField":"xdm:payloadNumberValue"
},
"payloadStringValue":{
"type":"string",
"title":"Value",
"description":"The string value.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:payloadStringValue"
},
"payloadBooleanValue":{
"type":"boolean",
"title":"Value",
"description":"The boolean value.",
"meta:xdmType":"boolean",
"meta:xdmField":"xdm:payloadBooleanValue"
},
"payloadPropensityValue":{
"type":"number",
"title":"Value",
"maximum":1,
"description":"The propensity.",
"meta:xdmType":"number",
"meta:xdmField":"xdm:payloadPropensityValue",
"exclusiveMinimum":0
}
},
"description":"Values that are directly related with the audience realization. This payload exists with the same 'validUntil' as the audience realization. Note that the intention is that exactly one payload value be included, as indicated by the payload type. This was originally modeled using 'oneOf', but due to limitations in our tooling that was removed. This more semantically meaningful representation will be re-introduced in the future.",
"meta:xdmType":"object",
"meta:xdmField":"xdm:payload"
},
"version":{
"type":"string",
"title":"Version",
"description":"The version of the audience definition used in this audience assertion. Version can be omitted in audience lists when all memberships versions are the same.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:version"
},
"segmentID":{
"type":"object",
"title":"Segment ID",
"properties":{
"_id":{
"type":"string",
"title":"Identifier",
"format":"uri-reference",
"description":"Identity of the audience in the related namespace.",
"meta:xdmType":"string",
"meta:xdmField":"@id"
},
"xid":{
"type":"string",
"title":"Experience identifier",
"description":"When present, this value represents a cross-namespace identifier that is unique across all namespace-scoped identifiers in all namespaces.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:xid"
},
"namespace":{
"type":"object",
"title":"Namespace",
"required":[
"code"
],
"properties":{
"code":{
"type":"string",
"title":"Code",
"description":"The code is a human readable identifier for the namespace and can be used to request the technical namespace id which is used for identity graph processing.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:code"
}
},
"description":"The namespace associated with the `xid` attribute.",
"meta:xdmType":"object",
"meta:xdmField":"xdm:namespace",
"meta:referencedFrom":"https://ns.adobe.com/xdm/context/namespace"
}
},
"description":"The identity of the audience or snapshot definition in with the domain of the specific system that processes that type of segment. Deprecated.",
"meta:status":"deprecated",
"meta:xdmType":"object",
"meta:xdmField":"xdm:segmentID",
"meta:referencedFrom":"https://ns.adobe.com/xdm/context/segmentidentity"
},
"validUntil":{
"type":"string",
"title":"Valid until",
"format":"date-time",
"description":"The timestamp for when the audienceassertion should no longer be assumed to be valid and should either be ignored or revalidated.",
"meta:xdmType":"date-time",
"meta:xdmField":"xdm:validUntil"
},
"profileStitchID":{
"type":"object",
"properties":{
"_id":{
"type":"string",
"title":"Identifier",
"format":"uri-reference",
"description":"Identity of the profile stitch in the related namespace.",
"meta:xdmType":"string",
"meta:xdmField":"@id"
},
"xid":{
"type":"string",
"title":"Experience identifier",
"description":"When present, this value represents a cross-namespace identifier that is unique across all namespace-scoped identifiers in all namespaces.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:xid"
},
"namespace":{
"type":"object",
"title":"Namespace",
"required":[
"code"
],
"properties":{
"code":{
"type":"string",
"title":"Code",
"description":"The code is a human readable identifier for the namespace and can be used to request the technical namespace id which is used for identity graph processing.",
"meta:xdmType":"string",
"meta:xdmField":"xdm:code"
}
},
"description":"The namespace associated with the `xid` attribute.",
"meta:xdmType":"object",
"meta:xdmField":"xdm:namespace",
"meta:referencedFrom":"https://ns.adobe.com/xdm/context/namespace"
}
},
"meta:xdmType":"object",
"meta:xdmField":"xdm:profileStitchID",
"meta:referencedFrom":"https://ns.adobe.com/xdm/context/profileStitchIdentity"
},
"lastQualificationTime":{
"type":"string",
"title":"Last qualification time",
"format":"date-time",
"description":"The timestamp when the assertion of audience membership was made.",
"meta:xdmType":"date-time",
"meta:xdmField":"xdm:lastQualificationTime"
}
},
"meta:xdmType":"object",
"meta:referencedFrom":"https://ns.adobe.com/xdm/context/segmentmembership"
}
}
}
}
}
Next, you need to create an output schema. Copy the JSON response you got above and paste it into the jsonSchema object below.
Request
Create an output schema - Request
curl --location --request POST 'https://platform.adobe.io/data/foundation/conversion/schemas' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Sample Schema for Flow Mapping",
"jsonSchema":{...insert JSON from the response above here}
}
'
Response
Create output schema - Response
{
"id": "2f4fd51934c1409fb1d8207dd9f43dc9",
"version": 0,
"jsonSchema": {
"title": "defaultschema",
"type": "object",
"properties": {
"identityMap": {
"type": "object",
"meta:xdmField": "xdm:identityMap",
"meta:xdmType": "map",
"additionalProperties": {
"meta:xdmType": "array",
"items": {
"meta:referencedFrom": "https://ns.adobe.com/xdm/context/identityitem",
"properties": {
"primary": {
"meta:xdmField": "xdm:primary",
"meta:xdmType": "boolean",
"description": "Indicates this identity is the preferred identity. Is used as a hint to help systems better organize how identities are queried.",
"default": false,
"type": "boolean",
"title": "Primary"
},
"id": {
"meta:xdmField": "xdm:id",
"meta:xdmType": "string",
"description": "Identity of the consumer in the related namespace.",
"type": "string",
"title": "Identifier"
},
"authenticatedState": {
"meta:xdmField": "xdm:authenticatedState",
"meta:xdmType": "string",
"meta:enum": {
"loggedOut": "User was identified by a login action at some point of time previously, but is not currently logged in.",
"authenticated": "User identified by a login or similar action that was valid at the time of the event observation.",
"ambiguous": "Ambiguous"
},
"enum": [
"ambiguous",
"authenticated",
"loggedOut"
],
"default": "ambiguous",
"type": "string",
"description": "The state this identity is authenticated as for this observed ExperienceEvent."
}
},
"meta:xdmType": "object",
"type": "object"
},
"type": "array"
}
},
"segmentMembership": {
"title": "Segment membership map",
"type": "object",
"meta:xdmField": "xdm:segmentMembership",
"meta:xdmType": "map",
"additionalProperties": {
"additionalProperties": {
"meta:referencedFrom": "https://ns.adobe.com/xdm/context/segmentmembership",
"properties": {
"version": {
"meta:xdmField": "xdm:version",
"meta:xdmType": "string",
"description": "The version of the audience definition used in this audience assertion. Version can be omitted in audience lists when all memberships versions are the same.",
"type": "string",
"title": "Version"
},
"validUntil": {
"meta:xdmField": "xdm:validUntil",
"meta:xdmType": "date-time",
"description": "The timestamp for when the audienceassertion should no longer be assumed to be valid and should either be ignored or revalidated.",
"format": "date-time",
"type": "string",
"title": "Valid until"
},
"status": {
"meta:xdmField": "xdm:status",
"meta:xdmType": "string",
"meta:enum": {
"exited": "Entity is exiting the segment.",
"realized": "Entity is entering the segment."
},
"enum": [
"realized",
"exited"
],
"default": "realized",
"description": "Is the audience participation realized as part of the current request.",
"type": "string",
"title": "Status"
},
"segmentID": {
"meta:xdmField": "xdm:segmentID",
"meta:referencedFrom": "https://ns.adobe.com/xdm/context/segmentidentity",
"properties": {
"xid": {
"meta:xdmField": "xdm:xid",
"meta:xdmType": "string",
"description": "When present, this value represents a cross-namespace identifier that is unique across all namespace-scoped identifiers in all namespaces.",
"type": "string",
"title": "Experience identifier"
},
"namespace": {
"meta:xdmField": "xdm:namespace",
"required": [
"code"
],
"meta:referencedFrom": "https://ns.adobe.com/xdm/context/namespace",
"properties": {
"code": {
"meta:xdmField": "xdm:code",
"meta:xdmType": "string",
"description": "The code is a human readable identifier for the namespace and can be used to request the technical namespace id which is used for identity graph processing.",
"type": "string",
"title": "Code"
}
},
"meta:xdmType": "object",
"type": "object",
"description": "The namespace associated with the `xid` attribute.",
"title": "Namespace"
},
"_id": {
"meta:xdmField": "@id",
"meta:xdmType": "string",
"description": "Identity of the audience in the related namespace.",
"format": "uri-reference",
"type": "string",
"title": "Identifier"
}
},
"meta:xdmType": "object",
"type": "object",
"description": "The identity of the audience or snapshot definition in with the domain of the specific system that processes that type of segment. Deprecated.",
"meta:status": "deprecated",
"title": "Segment ID"
},
"profileStitchID": {
"meta:xdmField": "xdm:profileStitchID",
"meta:referencedFrom": "https://ns.adobe.com/xdm/context/profileStitchIdentity",
"properties": {
"xid": {
"meta:xdmField": "xdm:xid",
"meta:xdmType": "string",
"description": "When present, this value represents a cross-namespace identifier that is unique across all namespace-scoped identifiers in all namespaces.",
"type": "string",
"title": "Experience identifier"
},
"namespace": {
"meta:xdmField": "xdm:namespace",
"required": [
"code"
],
"meta:referencedFrom": "https://ns.adobe.com/xdm/context/namespace",
"properties": {
"code": {
"meta:xdmField": "xdm:code",
"meta:xdmType": "string",
"description": "The code is a human readable identifier for the namespace and can be used to request the technical namespace id which is used for identity graph processing.",
"type": "string",
"title": "Code"
}
},
"meta:xdmType": "object",
"type": "object",
"description": "The namespace associated with the `xid` attribute.",
"title": "Namespace"
},
"_id": {
"meta:xdmField": "@id",
"meta:xdmType": "string",
"description": "Identity of the profile stitch in the related namespace.",
"format": "uri-reference",
"type": "string",
"title": "Identifier"
}
},
"meta:xdmType": "object",
"type": "object"
},
"payload": {
"meta:xdmField": "xdm:payload",
"meta:xdmType": "object",
"required": [
"payloadType"
],
"properties": {
"payloadType": {
"meta:xdmField": "xdm:payloadType",
"meta:xdmType": "string",
"description": "The type of payload.",
"meta:enum": {
"string": "String",
"propensity": "Propensity",
"number": "Number",
"boolean": "Boolean"
},
"enum": [
"boolean",
"number",
"propensity",
"string"
],
"type": "string",
"title": "Payload Type"
},
"payloadStringValue": {
"meta:xdmField": "xdm:payloadStringValue",
"meta:xdmType": "string",
"description": "The string value.",
"type": "string",
"title": "Value"
},
"payloadPropensityValue": {
"meta:xdmField": "xdm:payloadPropensityValue",
"meta:xdmType": "number",
"maximum": 1,
"exclusiveMinimum": 0,
"description": "The propensity.",
"type": "number",
"title": "Value"
},
"payloadNumberValue": {
"meta:xdmField": "xdm:payloadNumberValue",
"meta:xdmType": "number",
"description": "The number.",
"type": "number",
"title": "Value"
},
"payloadBooleanValue": {
"meta:xdmField": "xdm:payloadBooleanValue",
"meta:xdmType": "boolean",
"description": "The boolean value.",
"type": "boolean",
"title": "Value"
}
},
"type": "object",
"description": "Values that are directly related with the audience realization. This payload exists with the same 'validUntil' as the audience realization. Note that the intention is that exactly one payload value be included, as indicated by the payload type. This was originally modeled using 'oneOf', but due to limitations in our tooling that was removed. This more semantically meaningful representation will be re-introduced in the future.",
"title": "Payload"
},
"lastQualificationTime": {
"meta:xdmField": "xdm:lastQualificationTime",
"meta:xdmType": "date-time",
"description": "The timestamp when the assertion of audience membership was made.",
"format": "date-time",
"type": "string",
"title": "Last qualification time"
}
},
"meta:xdmType": "object",
"type": "object"
},
"meta:xdmType": "map",
"type": "object",
"title": "Segment membership per namespace"
}
},
"attributes": {
"type": "object",
"meta:xdmType": "map",
"additionalProperties": {
"properties": {
"value": {
"type": "string",
"title": "Value"
}
},
"meta:xdmType": "object",
"type": "object"
}
},
"firstName": {
"title": "firstName",
"type": "string"
},
"Email": {
"title": "Email",
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"title": "id",
"type": "string",
"meta:xdmType": "string"
}
},
"meta:xdmType": "object"
},
"meta:xdmType": "array"
}
}
}
}
The ID in the response represents the unique identifier of the input schema that you have created. Copy the ID from the response as you will reuse this in a later step.
Create mapping set
Next, use the data prep API to create the mapping set by using the input schema ID, the output schema ID, and the desired field mappings.
Request
Create mapping set - Request
IMPORTANT- In the mappings object shown below, the
destinationparameter does not accept dots".". For example you would need to use personalEmail_address or segmentMembership_status as highlighted in the configuration example. - There is one particular case when the source attribute is an identity attribute and contains a dot. In this case, the attribute needs to be escaped with
//, as highlighted below. - Note also that even though the example configuration below includes
EmailandPhone_E.164, you are only able to export one identity attribute per dataflow.
curl --location --request POST 'https://platform.adobe.io/data/foundation/conversion/mappingSets' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
"inputSchema": {
"id": "81d7fc5376e54eb58d5186fd30d5a8c9"
},
"outputSchema": {
"id": "2f4fd51934c1409fb1d8207dd9f43dc9"
},
"mappings": [
{
"destination": "firstName",
"source": "person.name.firstName",
"sourceType": "ATTRIBUTE"
},
{
"destination": "Email",
"source": "identityMap.Email",
"sourceType": "ATTRIBUTE"
},
{
"destination": "Phone_E_164",
"source": "identityMap.Phone_E//.164",
"sourceType": "ATTRIBUTE"
},
{
"destination": "personalEmail_address",
"source": "personalEmail.address",
"sourceType": "ATTRIBUTE"
},
{
"destination": "segmentMembership_status",
"source": "segmentMembership.ups.seg_id.status",
"sourceType": "ATTRIBUTE"
}
],
"xdmVersion": "1.0"
}'
Response
Create mapping set - Response
{
"id": "5f0031f8ccd4444dac9c5c391389e9e8",
"version": 0,
"createdDate": 1678999005197,
"modifiedDate": 1678999005197,
"createdBy": "example@AdobeID",
"modifiedBy": "example@AdobeID"
}
Note the ID of the mapping set as you will need it in the next step to update the existing dataflow with the mapping set ID.
Next, get the ID of the dataflow that you want to update.
See retrieve a destination dataflow’s details for information about retrieving the ID of a dataflow.
Finally, you need to PATCH the dataflow with the mapping set information that you just created.
Request
Update dataflow with mapping set information - Request
curl --location --request PATCH 'https://platform.adobe.io/data/foundation/flowservice/flows/df8245a8-dd8f-42da-a04a-2d3a92654d9e' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'If-Match: ETAG_HERE' \
--data-raw '[
{
"op":"ADD",
"path":"/transformations/0/params/profileMapping",
"value":{
"mappingId":"304ca6a2fff244949c956cad1cd9eae6",
"mappingVersion":0
}
}
]'
Response
Update dataflow with mapping set information - Response
The response from the Flow Service API returns the ID of the updated dataflow.
{
"id": "1c246ae4-ba0d-43cd-a0ec-f16fe0a56b55",
"etag": "\"1500c67f-0000-0200-0000-64137ee00000\""
}
Make other dataflow updates
To make any updates to your dataflow, use the PATCH operation. For example, you can add a marketing action to your dataflows. Or, you can update your dataflows to select fields as mandatory keys or deduplication keys.
Add a marketing action
To add a marketing action, see the request and response examples below.
IMPORTANTThe
If-Match header is required when making a PATCH request. The value for this header is the unique version of the dataflow you want to update. The etag value updates with every successful update of a flow entity such as dataflow, target connection, and others.To get the latest version of the etag value, perform a GET request to the
https://platform.adobe.io/data/foundation/flowservice/flows/{ID} endpoint, where {ID} is the dataflow ID that you are looking to update.Make sure to wrap the value of the
If-Match header in double quotes like in the examples below when making PATCH requests.Request
TIPBefore adding a marketing action to a dataflow, you can look up your existing core and custom marketing actions. View how to retrieve a list of existing marketing actions.
Add a marketing action to a destination dataflow - Request
curl --location --request PATCH 'https://platform.adobe.io/data/foundation/flowservice/flows/{DATAFLOW_ID}' \
--header 'accept: application/json' \
--header 'Content-Type: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'If-Match: "{ETAG_HERE}"' \
--data-raw '[
{
"op":"add",
"path":"/policy",
"value":{
"enforcementRefs":[
]
}
},
{
"op":"add",
"path":"/policy/enforcementRefs/-",
"value":"/dulepolicy/marketingActions/custom/6b935bc8-bb9e-451b-a327-0ffddfb91e66/constraints"
}
]'
Response
Add a marketing action - Response
A successful response returns response code 200 along with the ID of the updated dataflow and the updated eTag.
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Add a mandatory key
To add a mandatory key, see the request and response examples below.
IMPORTANTThe
If-Match header is required when making a PATCH request. The value for this header is the unique version of the dataflow you want to update. The etag value updates with every successful update of a flow entity such as dataflow, target connection, and others.To get the latest version of the etag value, perform a GET request to the
https://platform.adobe.io/data/foundation/flowservice/flows/{ID} endpoint, where {ID} is the dataflow ID that you are looking to update.Make sure to wrap the value of the
If-Match header in double quotes like in the examples below when making PATCH requests.Request
Add an identity as mandatory field - Request
curl --location --request PATCH 'https://platform.adobe.io/data/foundation/flowservice/flows/{DATAFLOW_ID}' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'If-Match: "{ETAG_HERE}"' \
--data-raw '
[
{
"op": "add",
"path": "/transformations/0/params/mandatoryFields",
"value": [
"GAID"
]
}
]'
Add an XDM attribute as mandatory field - Request
curl --location --request PATCH 'https://platform.adobe.io/data/foundation/flowservice/flows/{DATAFLOW_ID}' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'If-Match: "{ETAG_HERE}"' \
--data-raw '
[
{
"op": "add",
"path": "/transformations/0/params/mandatoryFields",
"value": [
"GAID"
]
}
]'
Response
Add a mandatory field - Response
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Add a deduplication key
To add a deduplication key, see the request and response examples below
IMPORTANTThe
If-Match header is required when making a PATCH request. The value for this header is the unique version of the dataflow you want to update. The etag value updates with every successful update of a flow entity such as dataflow, target connection, and others.To get the latest version of the etag value, perform a GET request to the
https://platform.adobe.io/data/foundation/flowservice/flows/{ID} endpoint, where {ID} is the dataflow ID that you are looking to update.Make sure to wrap the value of the
If-Match header in double quotes like in the examples below when making PATCH requests.Request
Add an identity as deduplication key - Request
curl --location --request PATCH 'https://platform.adobe.io/data/foundation/flowservice/flows/{DATAFLOW_ID}' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'If-Match: "{ETAG_HERE}"' \
--data-raw '
[
{
"op": "add",
"path": "/transformations/0/params/primaryFields",
"value": [
{
"identityNamespace": "GAID",
"fieldType": "IDENTITY"
}
]
}
]'
Add an XDM attribute as deduplication key - Request
curl --location --request PATCH 'https://platform.adobe.io/data/foundation/flowservice/flows/{DATAFLOW_ID}' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'If-Match: "{ETAG_HERE}"' \
--data-raw '
[
{
"op": "add",
"path": "/transformations/0/params/primaryFields",
"value": [
{
"identityNamespace": "GAID",
"fieldType": "IDENTITY"
}
]
}
]'
Response
Add a mandatory field - Response
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
Validate dataflow (Get the dataflow runs)
To check the executions of a dataflow, use the Dataflow Runs API:
Request
Get dataflow runs - Request
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/runs?property=flowId==eb54b3b3-3949-4f12-89c8-64eafaba858f' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw ''
Response
Get dataflow runs - Response
{
"items": [
{
"id": "4b7728dd-83c9-4c38-95a4-24ddab545404",
"createdAt": 1675807718296,
"updatedAt": 1675807731834,
"createdBy": "aep_activation_batch@AdobeID",
"updatedBy": "acp_foundation_connectors@AdobeID",
"createdClient": "aep_activation_batch",
"updatedClient": "acp_foundation_connectors",
"sandboxId": "7dfdcd30-0a09-11ea-8ea6-7bf93ce86c28",
"sandboxName": "sand-1",
"imsOrgId": "5555467B5D8013E50A494220@AdobeOrg",
"flowId": "aae5ec63-b0ac-4808-9a44-abf2ea67bd5a",
"flowSpec": {
"id": "615d3489-36d2-4671-9467-4ae1129facd3",
"version": "1.0"
},
"providerRefId": "ba56f98e0c49b572adb249980c39b1c7",
"etag": "\"08005e9e-0000-0200-0000-63e2cbf30000\"",
"metrics": {
"durationSummary": {
"startedAtUTC": 1675807719411,
"completedAtUTC": 1675807719416
},
"sizeSummary": {
"inputBytes": 0
},
"recordSummary": {
"inputRecordCount": 0,
"skippedRecordCount": 0,
"sourceSummaries": [
{
"id": "ea2b1205-4692-49de-b448-ebf75b1d188a",
"inputRecordCount": 0,
"skippedRecordCount": 0,
"entitySummaries": [
{
//...
You can find information about the various parameters returned by the Dataflow runs API in the API reference documentation.
API error handling
The API endpoints in this tutorial follow the general Experience Platform API error message principles. Refer to API status codes and request header errors in the Experience Platform troubleshooting guide for more information on interpreting error responses.
Next steps
By following this tutorial, you have successfully connected Experience Platform to one of your preferred cloud storage destinations and set up a dataflow to the respective destination to export audiences. See the following pages for more details, such as how to edit existing dataflows using the Flow Service API:
7f4d1967-bf93-4dba-9789-bb6b505339d6