(Beta) Server and file configuration for file-based destination server specs

Overview

IMPORTANT

The functionality to configure and submit file-based destinations using Adobe Experience Platform Destination SDK is currently in Beta. The documentation and functionality are subject to change.

This page details all server configuration options for your file-based destination servers and instructs you how to set up various file configuration options for users exporting files from Experience Platform to your destination.

The server and file configuration specs for file-based destinations can be configured in Adobe Experience Platform Destination SDK via the /destination-servers endpoint. Read Destination server API endpoint operations for a complete list of operations you can perform on the endpoint.

File-based Amazon S3 destination server spec

{
    "name": "S3 destination",
    "destinationServerType": "FILE_BASED_S3",
    "fileBasedS3Destination": {
        "bucket": {
            "templatingStrategy": "PEBBLE_V1",
            "value": "{{customerData.bucket}}"
        },
        "path": {
            "templatingStrategy": "PEBBLE_V1",
            "value": "{{customerData.path}}"
        }
    },
    "fileConfigurations": {
       // See the file formatting configuration section further below on this page
    }
}
Parameter Type Description
name String The name of your destination connection.
destinationServerType String Set this value according to your destination platform. For Amazon S3, set this to FILE_BASED_S3.
fileBasedS3Destination.bucket.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedS3Destination.bucket.value String The name of the Amazon S3 bucket to be used by this destination.
fileBasedS3Destination.path.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedS3Destination.path.value String The path to the destination folder that will host the exported files.
fileConfigurations Object See file formatting configuration for detailed explanations about this section.

File-based SFTP destination server spec

{
   "name":"File-based SFTP destination server",
   "destinationServerType":"FILE_BASED_SFTP",
   "fileBasedSftpDestination":{
      "rootDirectory":{
         "templatingStrategy":"PEBBLE_V1",
         "value":"{{customerData.rootDirectory}}"
      },
      "hostName":{
         "templatingStrategy":"PEBBLE_V1",
         "value":"{{customerData.hostName}}"
      },
      "port": 22,
      "encryptionMode" : "PGP"
   },
    "fileConfigurations": {
       // See the file formatting configuration section further below on this page
    }
}
Parameter Type Description
name String The name of your destination connection.
destinationServerType String Set this value according to your destination platform. For SFTP destinations, set this to FILE_BASED_SFTP.
fileBasedSftpDestination.rootDirectory.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedSftpDestination.rootDirectory.value String The root directory of the destination storage.
fileBasedSftpDestination.hostName.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedSftpDestination.hostName.value String The host name of the destination storage.
port Integer The SFTP file server port.
encryptionMode String Indicates whether to use file encryption. Supported values:
  • PGP
  • None
fileConfigurations Object See file formatting configuration for detailed explanations about this section.

File-based Azure Data Lake Storage (ADLS) destination server spec

{
   "name":"ADLS destination server",
   "destinationServerType":"FILE_BASED_ADLS_GEN2",
   "fileBasedAdlsGen2Destination":{
      "path":{
         "templatingStrategy":"PEBBLE_V1",
         "value":"{{customerData.path}}"
      }
   },
  "fileConfigurations": {
       // See the file formatting configuration section further below on this page
    }
}
Parameter Type Description
name String The name of your destination connection.
destinationServerType String Set this value according to your destination platform. For Azure Data Lake Storage destinations, set this to FILE_BASED_ADLS_GEN2.
fileBasedAdlsGen2Destination.path.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedAdlsGen2Destination.path.value String The path to the destination folder that will host the exported files.
fileConfigurations Object See file formatting configuration for detailed explanations about this section.

File-based Azure Blob Storage destination server spec

{
   "name":"Blob destination server",
   "destinationServerType":"FILE_BASED_AZURE_BLOB",
   "fileBasedAzureBlobDestination":{
      "path":{
         "templatingStrategy":"PEBBLE_V1",
         "value":"{{customerData.path}}"
      },
      "container":{
         "templatingStrategy":"PEBBLE_V1",
         "value":"{{customerData.container}}"
      }
   },
  "fileConfigurations": {
       // See the file formatting configuration section further below on this page
    }
}
Parameter Type Description
name String The name of your destination connection.
destinationServerType String Set this value according to your destination platform. For Azure Blob Storage destinations, set this to FILE_BASED_AZURE_BLOB.
fileBasedAzureBlobDestination.path.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedAzureBlobDestination.path.value String The path to the destination folder that will host the exported files.
fileBasedAzureBlobDestination.container.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedAzureBlobDestination.container.value String The name of the Azure Blob Storage container to be used by this destination.
fileConfigurations Object See file formatting configuration for detailed explanations about this section.

File-based Data Landing Zone (DLZ) destination server spec

{
   "name":"DLZ destination server",
   "destinationServerType":"FILE_BASED_DLZ",
   "fileBasedDlzDestination":{
      "path":{
         "templatingStrategy":"PEBBLE_V1",
         "value":"{{customerData.path}}"
      },
      "useCase": "Your use case"
   },
   "fileConfigurations": {
       // See the file formatting configuration section further below on this page
    }
}
Parameter Type Description
name String The name of your destination connection.
destinationServerType String Set this value according to your destination platform. For Data Landing Zone destinations, set this to FILE_BASED_DLZ.
fileBasedDlzDestination.path.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedDlzDestination.path.value String The path to the destination folder that will host the exported files.
fileConfigurations Object See file formatting configuration for detailed explanations about this section.

File-based Google Cloud Storage destination server spec

{
   "name":"Google Cloud Storage Server",
   "destinationServerType":"FILE_BASED_GOOGLE_CLOUD",
   "fileBasedGoogleCloudStorageDestination":{
      "bucket":{
         "templatingStrategy":"PEBBLE_V1",
         "value":"{{customerData.bucket}}"
      },
      "path":{
         "templatingStrategy":"PEBBLE_V1",
         "value":"{{customerData.path}}"
      }
   },
   "fileConfigurations":{
      // See the file formatting configuration section further below on this page
   }
}
Parameter Type Description
name String The name of your destination connection.
destinationServerType String Set this value according to your destination platform. For Google Cloud Storage destinations, set this to FILE_BASED_GOOGLE_CLOUD.
fileBasedGoogleCloudStorageDestination.bucket.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedGoogleCloudStorageDestination.bucket.value String The name of the Google Cloud Storage bucket to be used by this destination.
fileBasedGoogleCloudStorageDestination.path.templatingStrategy String Required. Use PEBBLE_V1.
fileBasedGoogleCloudStorageDestination.path.value String The path to the destination folder that will host the exported files.
fileConfigurations Object See file formatting configuration for detailed explanations about this section.

File formatting configuration

This section describes the file formatting settings for the exported CSV files. You can modify several properties of the exported files to match the requirements of the file reception system on your side, in order to optimally read and interpret the files received from Experience Platform.

NOTE

CSV options are only supported when exporting CSV files. The fileConfigurations section is not mandatory when setting up a new destination server. If you don’t pass any values in the API call for the CSV options, the default ones from the table below will be used.

"fileConfigurations": {
        "compression": {
            "templatingStrategy": "PEBBLE_V1",
            "value": "{{customerData.compression}}"
        },
        "fileType": {
            "templatingStrategy": "PEBBLE_V1",
            "value": "{{customerData.fileType}}"
        },
        "csvOptions": {
            "quote": {
                "templatingStrategy": "NONE",
                "value": "\""
            },
            "quoteAll": {
                "templatingStrategy": "NONE",
                "value": "false"
            },
            "escape": {
                "templatingStrategy": "NONE",
                "value": "\\"
            },
            "escapeQuotes": {
                "templatingStrategy": "NONE",
                "value": "true"
            },
            "header": {
                "templatingStrategy": "NONE",
                "value": "true"
            },
            "ignoreLeadingWhiteSpace": {
                "templatingStrategy": "NONE",
                "value": "true"
            },
            "ignoreTrailingWhiteSpace": {
                "templatingStrategy": "NONE",
                "value": "true"
            },
            "nullValue": {
                "templatingStrategy": "NONE",
                "value": ""
            },
            "dateFormat": {
                "templatingStrategy": "NONE",
                "value": "yyyy-MM-dd"
            },
            "timestampFormat": {
                "templatingStrategy": "NONE",
                "value": "yyyy-MM-dd'T':mm:ss[.SSS][XXX]"
            },
            "charToEscapeQuoteEscaping": {
                "templatingStrategy": "NONE",
                "value": "\\"
            },
            "emptyValue": {
                "templatingStrategy": "NONE",
                "value": ""
            },
            "lineSep": {
                "templatingStrategy": "NONE",
                "value": "\n"
            }
        },
        "maxFileRowCount":5000000
    }
Field Required/Optional Description Default value
compression.value Optional Compression codec to use when saving data to file. Supported values: none, bzip2, gzip, lz4, and snappy. none
fileType.value Optional Specifies the output file format. Supported values: csv, parquet, and json. csv
csvOptions.quote.value Optional Only for "fileType.value": "csv". Sets a single character used for escaping quoted values where the separator can be part of the value. null
csvOptions.quoteAll.value Optional Only for "fileType.value": "csv". Indicates whether all values should always be enclosed in quotes. Default is to only escape values containing a quote character. false
csvOptions.escape.value Optional Only for "fileType.value": "csv". Sets a single character used for escaping quotes inside an already quoted value. \
csvOptions.escapeQuotes.value Optional Only for "fileType.value": "csv". Indicates whether values containing quotes should always be enclosed in quotes. Default is to escape all values containing a quote character. true
csvOptions.header.value Optional Only for "fileType.value": "csv". Indicates whether to write the names of columns as the first line. true
csvOptions.ignoreLeadingWhiteSpace.value Optional Only for "fileType.value": "csv". Indicates whether to trim leading white spaces from values. true
csvOptions.ignoreTrailingWhiteSpace.value Optional Only for "fileType.value": "csv". Indicates whether to trim trailing whitespaces from values. true
csvOptions.nullValue.value Optional Only for "fileType.value": "csv". Sets the string representation of a null value. ""
csvOptions.dateFormat.value Optional Only for "fileType.value": "csv". Indicates the date format. yyyy-MM-dd
csvOptions.timestampFormat.value Optional Only for "fileType.value": "csv". Sets the string that indicates a timestamp format. yyyy-MM-dd'T'HH:mm:ss[.SSS][XXX]
csvOptions.charToEscapeQuoteEscaping.value Optional Only for "fileType.value": "csv". Sets a single character used for escaping the escape for the quote character. \ when the escape and quote characters are different. \0 when the escape and quote character are the same.
csvOptions.emptyValue.value Optional Only for "fileType.value": "csv". Sets the string representation of an empty value. ""
csvOptions.lineSep.value Optional Only for "fileType.value": "csv". Defines the line separator that should be used for writing. Maximum length is 1 character. \n
maxFileRowCount Optional Maximum number of rows that the exported file can contain. Configure this based on your destination platform file size requirements. N/A

On this page