(Beta) Server and file configuration for file-based destination server specs
Overview
IMPORTANT
The functionality to configure and submit file-based destinations using Adobe Experience Platform Destination SDK is currently in Beta. The documentation and functionality are subject to change.
This page details all server configuration options for your file-based destination servers and instructs you how to set up various file configuration options for users exporting files from Experience Platform to your destination.
The server and file configuration specs for file-based destinations can be configured in Adobe Experience Platform Destination SDK via the /destination-servers
endpoint. Read Destination server API endpoint operations for a complete list of operations you can perform on the endpoint.
File-based Amazon S3 destination server spec
{
"name": "S3 destination",
"destinationServerType": "FILE_BASED_S3",
"fileBasedS3Destination": {
"bucket": {
"templatingStrategy": "PEBBLE_V1",
"value": "{{customerData.bucket}}"
},
"path": {
"templatingStrategy": "PEBBLE_V1",
"value": "{{customerData.path}}"
}
},
"fileConfigurations": {
// See the file formatting configuration section further below on this page
}
}
Parameter |
Type |
Description |
name |
String |
The name of your destination connection. |
destinationServerType |
String |
Set this value according to your destination platform. For Amazon S3, set this to FILE_BASED_S3 . |
fileBasedS3Destination.bucket.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedS3Destination.bucket.value |
String |
The name of the Amazon S3 bucket to be used by this destination. |
fileBasedS3Destination.path.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedS3Destination.path.value |
String |
The path to the destination folder that will host the exported files. |
fileConfigurations |
Object |
See file formatting configuration for detailed explanations about this section. |
File-based SFTP destination server spec
{
"name":"File-based SFTP destination server",
"destinationServerType":"FILE_BASED_SFTP",
"fileBasedSftpDestination":{
"rootDirectory":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.rootDirectory}}"
},
"hostName":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.hostName}}"
},
"port": 22,
"encryptionMode" : "PGP"
},
"fileConfigurations": {
// See the file formatting configuration section further below on this page
}
}
Parameter |
Type |
Description |
name |
String |
The name of your destination connection. |
destinationServerType |
String |
Set this value according to your destination platform. For SFTP destinations, set this to FILE_BASED_SFTP . |
fileBasedSftpDestination.rootDirectory.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedSftpDestination.rootDirectory.value |
String |
The root directory of the destination storage. |
fileBasedSftpDestination.hostName.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedSftpDestination.hostName.value |
String |
The host name of the destination storage. |
port |
Integer |
The SFTP file server port. |
encryptionMode |
String |
Indicates whether to use file encryption. Supported values: |
fileConfigurations |
Object |
See file formatting configuration for detailed explanations about this section. |
File-based Azure Data Lake Storage (ADLS) destination server spec
{
"name":"ADLS destination server",
"destinationServerType":"FILE_BASED_ADLS_GEN2",
"fileBasedAdlsGen2Destination":{
"path":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.path}}"
}
},
"fileConfigurations": {
// See the file formatting configuration section further below on this page
}
}
Parameter |
Type |
Description |
name |
String |
The name of your destination connection. |
destinationServerType |
String |
Set this value according to your destination platform. For Azure Data Lake Storage destinations, set this to FILE_BASED_ADLS_GEN2 . |
fileBasedAdlsGen2Destination.path.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedAdlsGen2Destination.path.value |
String |
The path to the destination folder that will host the exported files. |
fileConfigurations |
Object |
See file formatting configuration for detailed explanations about this section. |
File-based Azure Blob Storage destination server spec
{
"name":"Blob destination server",
"destinationServerType":"FILE_BASED_AZURE_BLOB",
"fileBasedAzureBlobDestination":{
"path":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.path}}"
},
"container":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.container}}"
}
},
"fileConfigurations": {
// See the file formatting configuration section further below on this page
}
}
Parameter |
Type |
Description |
name |
String |
The name of your destination connection. |
destinationServerType |
String |
Set this value according to your destination platform. For Azure Blob Storage destinations, set this to FILE_BASED_AZURE_BLOB . |
fileBasedAzureBlobDestination.path.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedAzureBlobDestination.path.value |
String |
The path to the destination folder that will host the exported files. |
fileBasedAzureBlobDestination.container.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedAzureBlobDestination.container.value |
String |
The name of the Azure Blob Storage container to be used by this destination. |
fileConfigurations |
Object |
See file formatting configuration for detailed explanations about this section. |
File-based Data Landing Zone (DLZ) destination server spec
{
"name":"DLZ destination server",
"destinationServerType":"FILE_BASED_DLZ",
"fileBasedDlzDestination":{
"path":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.path}}"
},
"useCase": "Your use case"
},
"fileConfigurations": {
// See the file formatting configuration section further below on this page
}
}
Parameter |
Type |
Description |
name |
String |
The name of your destination connection. |
destinationServerType |
String |
Set this value according to your destination platform. For Data Landing Zone destinations, set this to FILE_BASED_DLZ . |
fileBasedDlzDestination.path.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedDlzDestination.path.value |
String |
The path to the destination folder that will host the exported files. |
fileConfigurations |
Object |
See file formatting configuration for detailed explanations about this section. |
File-based Google Cloud Storage destination server spec
{
"name":"Google Cloud Storage Server",
"destinationServerType":"FILE_BASED_GOOGLE_CLOUD",
"fileBasedGoogleCloudStorageDestination":{
"bucket":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.bucket}}"
},
"path":{
"templatingStrategy":"PEBBLE_V1",
"value":"{{customerData.path}}"
}
},
"fileConfigurations":{
// See the file formatting configuration section further below on this page
}
}
Parameter |
Type |
Description |
name |
String |
The name of your destination connection. |
destinationServerType |
String |
Set this value according to your destination platform. For Google Cloud Storage destinations, set this to FILE_BASED_GOOGLE_CLOUD . |
fileBasedGoogleCloudStorageDestination.bucket.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedGoogleCloudStorageDestination.bucket.value |
String |
The name of the Google Cloud Storage bucket to be used by this destination. |
fileBasedGoogleCloudStorageDestination.path.templatingStrategy |
String |
Required. Use PEBBLE_V1 . |
fileBasedGoogleCloudStorageDestination.path.value |
String |
The path to the destination folder that will host the exported files. |
fileConfigurations |
Object |
See file formatting configuration for detailed explanations about this section. |
File formatting configuration
This section describes the file formatting settings for the exported CSV
files. You can modify several properties of the exported files to match the requirements of the file reception system on your side, in order to optimally read and interpret the files received from Experience Platform.
NOTE
CSV options are only supported when exporting CSV files. The fileConfigurations
section is not mandatory when setting up a new destination server. If you don’t pass any values in the API call for the CSV options, the default ones from the table below will be used.
"fileConfigurations": {
"compression": {
"templatingStrategy": "PEBBLE_V1",
"value": "{{customerData.compression}}"
},
"fileType": {
"templatingStrategy": "PEBBLE_V1",
"value": "{{customerData.fileType}}"
},
"csvOptions": {
"quote": {
"templatingStrategy": "NONE",
"value": "\""
},
"quoteAll": {
"templatingStrategy": "NONE",
"value": "false"
},
"escape": {
"templatingStrategy": "NONE",
"value": "\\"
},
"escapeQuotes": {
"templatingStrategy": "NONE",
"value": "true"
},
"header": {
"templatingStrategy": "NONE",
"value": "true"
},
"ignoreLeadingWhiteSpace": {
"templatingStrategy": "NONE",
"value": "true"
},
"ignoreTrailingWhiteSpace": {
"templatingStrategy": "NONE",
"value": "true"
},
"nullValue": {
"templatingStrategy": "NONE",
"value": ""
},
"dateFormat": {
"templatingStrategy": "NONE",
"value": "yyyy-MM-dd"
},
"timestampFormat": {
"templatingStrategy": "NONE",
"value": "yyyy-MM-dd'T':mm:ss[.SSS][XXX]"
},
"charToEscapeQuoteEscaping": {
"templatingStrategy": "NONE",
"value": "\\"
},
"emptyValue": {
"templatingStrategy": "NONE",
"value": ""
},
"lineSep": {
"templatingStrategy": "NONE",
"value": "\n"
}
},
"maxFileRowCount":5000000
}
Field |
Required/Optional |
Description |
Default value |
compression.value |
Optional |
Compression codec to use when saving data to file. Supported values: none , bzip2 , gzip , lz4 , and snappy . |
none |
fileType.value |
Optional |
Specifies the output file format. Supported values: csv , parquet , and json . |
csv |
csvOptions.quote.value |
Optional |
Only for "fileType.value": "csv" . Sets a single character used for escaping quoted values where the separator can be part of the value. |
null |
csvOptions.quoteAll.value |
Optional |
Only for "fileType.value": "csv" . Indicates whether all values should always be enclosed in quotes. Default is to only escape values containing a quote character. |
false |
csvOptions.escape.value |
Optional |
Only for "fileType.value": "csv" . Sets a single character used for escaping quotes inside an already quoted value. |
\ |
csvOptions.escapeQuotes.value |
Optional |
Only for "fileType.value": "csv" . Indicates whether values containing quotes should always be enclosed in quotes. Default is to escape all values containing a quote character. |
true |
csvOptions.header.value |
Optional |
Only for "fileType.value": "csv" . Indicates whether to write the names of columns as the first line. |
true |
csvOptions.ignoreLeadingWhiteSpace.value |
Optional |
Only for "fileType.value": "csv" . Indicates whether to trim leading white spaces from values. |
true |
csvOptions.ignoreTrailingWhiteSpace.value |
Optional |
Only for "fileType.value": "csv" . Indicates whether to trim trailing whitespaces from values. |
true |
csvOptions.nullValue.value |
Optional |
Only for "fileType.value": "csv" . Sets the string representation of a null value. |
"" |
csvOptions.dateFormat.value |
Optional |
Only for "fileType.value": "csv" . Indicates the date format. |
yyyy-MM-dd |
csvOptions.timestampFormat.value |
Optional |
Only for "fileType.value": "csv" . Sets the string that indicates a timestamp format. |
yyyy-MM-dd'T'HH:mm:ss[.SSS][XXX] |
csvOptions.charToEscapeQuoteEscaping.value |
Optional |
Only for "fileType.value": "csv" . Sets a single character used for escaping the escape for the quote character. |
\ when the escape and quote characters are different. \0 when the escape and quote character are the same. |
csvOptions.emptyValue.value |
Optional |
Only for "fileType.value": "csv" . Sets the string representation of an empty value. |
"" |
csvOptions.lineSep.value |
Optional |
Only for "fileType.value": "csv" . Defines the line separator that should be used for writing. Maximum length is 1 character. |
\n |
maxFileRowCount |
Optional |
Maximum number of rows that the exported file can contain. Configure this based on your destination platform file size requirements. |
N/A |