Configure a file-based destination to export prospect audiences to a storage location
Overview overview
This page describes how to use Destination SDK to configure a file-based destination with custom file formatting options and a custom file name configuration to export prospect audiences. The examples in this guide describe how to export audiences of prospect profiles to an Amazon S3 location.
You can also set up STFP or other storage locations to export prospect audiences. The important part to remember is to add the snippet below to the destination configuration in step 2 to enable the workflow to export prospect audiences to the destination.
"sources": [
"UNIFIED_PROFILE_PROSPECTS"
],
For detailed descriptions of the parameters used below, see configuration options in Destinations SDK.
Prerequisites prerequisites
Before advancing to the steps outlined below, please read the Destination SDK getting started page for information on obtaining the necessary authentication credentials and other prerequisites to work with Destination SDK APIs.
Step 1: Create a server and file configuration create-server-file-configuration
Start by using the /destination-server endpoint to create a server and file configuration.
API format
POST platform.adobe.io/data/core/activation/authoring/destination-servers
Request
The following request creates a new destination server configuration, configured by the parameters provided in the payload.
The payload below includes a generic Amazon S3 configuration, with custom CSV file formatting configuration parameters that users can define in the Experience Platform UI.
curl -X POST https://platform.adobe.io/data/core/activation/authoring/destination-server \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'Content-Type: application/json' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-sandbox-name: {SANDBOX_NAME}' \
-d ' {
"name":"Amazon S3 destination server with custom file formatting options",
"destinationServerType":"FILE_BASED_S3",
"fileBasedS3Destination":{
"bucket":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.bucketName}}"
},
"path":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.path}}"
}
},
"fileConfigurations":{
"compression":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.compression}}"
},
"fileType":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.fileType}}"
},
"csvOptions":{
"sep":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.sep}}"
},
"encoding":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.encoding}}"
},
"quote":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.quote}}"
},
"quoteAll":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.quoteAll}}"
},
"escape":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.escape}}"
},
"escapeQuotes":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.escapeQuotes}}"
},
"header":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.header}}"
},
"ignoreLeadingWhiteSpace":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.ignoreLeadingWhiteSpace}}"
},
"nullValue":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.nullValue}}"
},
"dateFormat":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.dateFormat}}"
},
"charToEscapeQuoteEscaping":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.charToEscapeQuoteEscaping}}"
},
"emptyValue":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.dateFormat}}"
}
}
}
}'
A successful response returns the new destination server configuration, including the unique identifier (instanceId) of the configuration. Store this value as it is required in the next step.
Step 2: Create destination configuration create-destination-configuration
After creating the destination server and file formatting configuration in the previous step, you can now use the /destinations API endpoint to create a destination configuration.
To connect the server configuration in step 1 to this destination configuration, replace the destinationServerId value in the API request below with the value obtained when creating your destination server in step 1.
API format
POST platform.adobe.io/data/core/activation/authoring/destinations
Request
curl -X POST https://platform.adobe.io/data/core/activation/authoring/destinations \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'Content-Type: application/json' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-sandbox-name: {SANDBOX_NAME}' \
-d '
{
"name":"Amazon S3 destination to export prospect audiences",
"description":"Amazon S3 destination to export prospect audiences",
"status":"TEST",
"sources": [
"UNIFIED_PROFILE_PROSPECTS"
],
"customerAuthenticationConfigurations":[
{
"authType":"S3"
}
],
"customerEncryptionConfigurations":[
],
"customerDataFields":[
{
"name":"bucketName",
"title":"Enter the name of your Amazon S3 bucket",
"description":"Amazon S3 bucket name",
"type":"string",
"isRequired":true,
"pattern": "(?=^.{3,63}$)(?!^(\\d+\\.)+\\d+$)(^(([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])\\.)*([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])$)",
"readOnly":false,
"hidden":false
},
{
"name":"path",
"title":"Enter the path to your S3 bucket folder",
"description":"Enter the path to your S3 bucket folder",
"type":"string",
"isRequired":true,
"pattern": "^[0-9a-zA-Z\\/\\!\\-_\\.\\*\\''\\(\\)]*((\\%SEGMENT_(NAME|ID)\\%)?\\/?)+$",
"readOnly":false,
"hidden":false
},
{
"name":"sep",
"title":"Enter your desired separator for each field and value",
"description":"Enter your desired separator for each field and value",
"type":"string",
"isRequired":false,
"readOnly":false,
"hidden":false
},
{
"name":"encoding",
"title":"Select the desired CSV file encoding",
"description":"Select the desired CSV file encoding",
"type":"string",
"enum":[
"UTF-8",
"UTF-16"
],
"isRequired":false,
"readOnly":false,
"hidden":false
},
{
"name":"quote",
"title":"Quoted values escape character",
"description":"Enter the desired character to be used for escaping quoted values.",
"type":"string",
"isRequired":false,
"readOnly":false,
"hidden":false
},
{
"name":"quoteAll",
"title":"Escape all quoted values",
"description":"Select whether to escape all quoted values.",
"type":"string",
"enum":[
"true",
"false"
],
"default":"true",
"isRequired":true,
"readOnly":false,
"hidden":false
},
{
"name":"escape",
"title":"Quote escaping character",
"description":"Enter the desired character to be used for escaping quotes inside an already quoted value.",
"type":"string",
"isRequired":false,
"readOnly":false,
"hidden":false
},
{
"name":"escapeQuotes",
"title":"Enclose quoted values within quotes",
"description":"Select whether values containing quotes should always be enclosed in quotes.",
"type":"string",
"enum":[
"true",
"false"
],
"isRequired":false,
"default":"true",
"readOnly":false,
"hidden":false
},
{
"name":"header",
"title":"Generate file header.",
"description":"Select whether to write the names of columns as the first line of the exported files.",
"type":"string",
"isRequired":false,
"enum":[
"true",
"false"
],
"readOnly":false,
"default":"true",
"hidden":false
},
{
"name":"ignoreLeadingWhiteSpace",
"title":"Ignore leading white space",
"description":"Select whether leading whitespaces should be trimmed from exported values.",
"type":"string",
"isRequired":false,
"enum":[
"true",
"false"
],
"readOnly":false,
"default":"true",
"hidden":false
},
{
"name":"nullValue",
"title":"NULL value string format",
"description":"Enter the string representation of a NULL value. ",
"type":"string",
"isRequired":false,
"readOnly":false,
"hidden":false
},
{
"name":"dateFormat",
"title":"Date format",
"description":"Enter the desired date format. ",
"type":"string",
"default":"yyyy-MM-dd",
"isRequired":false,
"readOnly":false,
"hidden":false
},
{
"name":"charToEscapeQuoteEscaping",
"title":"Quote escaping escape character",
"description":"Enter the desired character to be used for escaping the escaping of a quote character.",
"type":"string",
"isRequired":false,
"readOnly":false,
"hidden":false
},
{
"name":"emptyValue",
"title":"Empty value string format",
"description":"Enter the string representation of an empty value.",
"type":"string",
"isRequired":false,
"readOnly":false,
"default":"",
"hidden":false
},
{
"name":"compression",
"title":"Compression format",
"description":"Select the desired file compression format.",
"type":"string",
"isRequired":true,
"readOnly":false,
"enum":[
"SNAPPY",
"GZIP",
"DEFLATE",
"NONE"
]
},
{
"name":"fileType",
"title":"File type",
"description":"Select the exported file type.",
"type":"string",
"isRequired":true,
"readOnly":false,
"hidden":false,
"enum":[
"csv",
"json",
"parquet"
],
"default":"csv"
}
],
"uiAttributes":{
"documentationLink":"https://www.adobe.com/go/destinations-amazon-s3-en",
"category":"cloudStorage",
"connectionType":"S3",
"flowRunsSupported":true,
"monitoringSupported":true,
"frequency":"Batch"
},
"destinationDelivery":[
{
"deliveryMatchers":[
{
"type":"SOURCE",
"value":[
"batch"
]
}
],
"authenticationRule":"CUSTOMER_AUTHENTICATION",
"destinationServerId":"{{destinationServerId}}"
}
],
"schemaConfig":{
"profileRequired":true,
"segmentRequired":true,
"identityRequired":true
},
"batchConfig":{
"allowMandatoryFieldSelection":true,
"allowDedupeKeyFieldSelection":true,
"defaultExportMode":"DAILY_FULL_EXPORT",
"allowedExportMode":[
"DAILY_FULL_EXPORT"
],
"allowedScheduleFrequency":[
"DAILY",
"ONCE"
],
"defaultFrequency":"DAILY",
"defaultStartTime":"00:00",
"filenameConfig":{
"allowedFilenameAppendOptions":[
"SEGMENT_NAME",
"DESTINATION_INSTANCE_ID",
"DESTINATION_INSTANCE_NAME",
"ORGANIZATION_NAME",
"SANDBOX_NAME",
"DATETIME",
"CUSTOM_TEXT"
],
"defaultFilenameAppendOptions":[
"DATETIME"
],
"defaultFilename":"%DESTINATION%_%SEGMENT_ID%"
},
"backfillHistoricalProfileData":true
}
}'
A successful response returns the new destination configuration, including the unique identifier (instanceId) of the configuration. Store this value as it is required if you need to make further HTTP requests to update your destination configuration.
Step 3: Verify the Experience Platform UI verify-ui
Based on the configurations above, the Experience Platform catalog will now display a new private destination card for you to use.
In the images and recordings below, note how the options in the activation workflow for file-based destinations match the options that you selected in the destination configuration.
When filling in details about the destination, notice how the fields surfaced are the custom data fields that you set up in the configuration.
When scheduling export intervals, notice how the fields surfaced are the fields you set up in the batchConfig configuration.
When viewing the filename configuration options, notice how the fields surfaced represent the filenameConfig options that you set up in the configuration.
If you want to adjust any of the fields mentioned above, repeat steps one and two to modify the configurations according to your needs.
Step 4: (Optional) Publish your destination publish-destination
After configuring your destination, use the destination publishing API to submit your configuration to Adobe for review.
Step 5: (Optional) Document your destination document-destination
If you are an Independent Software Vendor (ISV) or System Integrator (SI) creating a productized integration, use the self-service documentation process to create a product documentation page for your destination in the Experience Platform destinations catalog.
Next steps next-steps
By reading this article, you now know how to use Destination SDK to author a custom Amazon S3 destination to export prospect audiences.