本文说明了使用 Flow Service API 以导出 数据集 从Adobe Experience Platform到您的首选云存储位置,例如 Amazon S3、 SFTP位置或 Google Cloud Storage.
您还可以使用Experience Platform用户界面导出数据集。 阅读 导出数据集用户界面教程 以了解更多信息。
您可以导出的数据集取决于Experience Platform应用程序(Real-Time CDP、Adobe Journey Optimizer)、层(Prime或Ultimate)以及您购买的任何加载项(例如:Data Distiller)。
请参阅 UI教程页面上的表 以了解可以导出哪些数据集。
目前,您可以将数据集导出到屏幕快照中突出显示的云存储目标,如下所列。
本指南要求您对 Adobe Experience Platform 的以下组件有一定了解:
以下部分提供了要将数据集导出到Platform中的云存储目标必须了解的其他信息。
要导出数据集,您需要 查看目标, 查看数据集、和 管理和激活数据集目标 访问控制权限. 阅读 访问控制概述 或与产品管理员联系以获取所需的权限。
要确保您具有导出数据集的必要权限并且目标支持导出数据集,请浏览目标目录。 如果目标具有 激活 或 导出数据集 则您具有相应的权限。
本教程提供了示例API调用来演示如何格式化请求。 这些包括路径、必需的标头和格式正确的请求负载。还提供了在 API 响应中返回的示例 JSON。有关示例 API 调用的文档中使用的惯例信息,请参阅 故障排除指南中的如何读取示例 API 调用Experience Platform。
为了调用 Platform API,您必须先完成 Experience Platform身份验证教程. 完成身份验证教程会提供所有 Experience Platform API 调用中每个所需标头的值,如下所示:
{ACCESS_TOKEN}
{API_KEY}
{ORG_ID}
中的资源 Experience Platform 可以隔离到特定的虚拟沙箱。 在请求中 Platform API中,您可以指定将在其中执行操作的沙盒的名称和ID。 这些是可选参数。
{SANDBOX_NAME}
有关中沙箱的详细信息 Experience Platform,请参见 沙盒概述文档.
所有包含有效负载(POST、PUT、PATCH)的请求都需要额外的媒体类型标头:
application/json
您可以在本教程中找到所有API操作的随附参考文档。 请参阅 Flow Service - Adobe Developer网站上的目标API文档. 我们建议您并行使用此教程和API参考文档。
有关在此API教程中遇到的术语的描述,请参阅 词汇表部分 API参考文档的URL名称。
在开始导出数据集的工作流之前,请确定要将数据集导出到的目标的连接规范和流量规范ID。 请参考下表。
目标 | 连接规范 | 流量规范 |
---|---|---|
Amazon S3 | 4fce964d-3f37-408f-9778-e597338a21ee |
269ba276-16fc-47db-92b0-c1049a3c131f |
Azure Blob Storage | 6d6b59bf-fb58-4107-9064-4d246c0e5bb2 |
95bd8965-fc8a-4119-b9c3-944c2c2df6d2 |
Azure Data Lake Gen 2(ADLS Gen2) | be2c3209-53bc-47e7-ab25-145db8b873e1 |
17be2013-2549-41ce-96e7-a70363bec293 |
Data Landing Zone(DLZ) | 10440537-2a7b-4583-ac39-ed38d4b848e8 |
cd2fc47e-e838-4f38-a581-8fff2f99b63a |
Google Cloud Storage | c5d93acb-ea8b-4b14-8f53-02138444ae99 |
585c15c4-6cbf-4126-8f87-e26bff78b657 |
SFTP | 36965a81-b1c6-401b-99f8-22508f1e6a26 |
354d6aad-4754-46e4-a576-1b384561c440 |
您需要这些ID Flow Service 实体。 您还需要参考 Connection Spec 设置特定图元,以便您能够检索 Connection Spec 从 Flow Service APIs. 请参阅下面的示例,以了解如何检索表中所有目标的连接规范:
请求
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/4fce964d-3f37-408f-9778-e597338a21ee' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
响应
{
"items": [
{
"id": "4fce964d-3f37-408f-9778-e597338a21ee",
"name": "Amazon S3",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
请求
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/6d6b59bf-fb58-4107-9064-4d246c0e5bb2' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
响应
{
"items": [
{
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2",
"name": "Azure Blob Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
请求
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/be2c3209-53bc-47e7-ab25-145db8b873e1' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
响应
{
"items": [
{
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1",
"name": "Azure Data Lake Gen2",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
请求
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/10440537-2a7b-4583-ac39-ed38d4b848e8' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
响应
{
"items": [
{
"id": "10440537-2a7b-4583-ac39-ed38d4b848e8",
"name": "Data Landing Zone",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
请求
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/c5d93acb-ea8b-4b14-8f53-02138444ae99' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
响应
{
"items": [
{
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99",
"name": "Google Cloud Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
请求
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/36965a81-b1c6-401b-99f8-22508f1e6a26' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
响应
{
"items": [
{
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26",
"name": "SFTP",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
//...
请按照以下步骤设置传送到云存储目标的数据集数据流。 对于某些步骤,不同云存储目标之间的请求和响应会有所不同。 在这些情况下,使用页面上的选项卡可检索特定于您要连接并导出数据集的目标的请求和响应。 请确保使用正确的 connection spec 和 flow spec (对于您正在配置的目标)。
要检索符合激活条件的数据集列表,首先要对以下端点进行API调用。
请求
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/connectionSpecs/23598e46-f560-407b-88d5-ea6207e49db0/configs?outputType=activationDatasets&outputField=datasets&start=0&limit=20&properties=name,state' \
--header 'accept: application/json' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-api-key: {API_KEY}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
请注意,要检索符合条件的数据集,请 connection spec 请求URL中使用的ID必须是数据湖源连接规范ID, 23598e46-f560-407b-88d5-ea6207e49db0
和两个查询参数 outputField=datasets
和 outputType=activationDatasets
必须指定。 所有其他查询参数都是 目录服务API.
响应
{
"items": [
{
"id": "5ef3e324052581191aa6a466",
"name": "AAM Authenticated Profiles Meta Data",
"description": "Activation profile export dataset",
"fileDescription": {
"persisted": true,
"containerFormat": "parquet",
"format": "parquet"
},
"aspect": "production",
"state": "DRAFT"
},
{
"id": "5ef3e3259ad2a1191ab7dd7d",
"name": "AAM Devices Data",
"description": "Activation profile export dataset",
"fileDescription": {
"persisted": true,
"containerFormat": "parquet",
"format": "parquet"
},
"aspect": "production",
"state": "DRAFT"
},
{
"id": "5ef3e325582424191b1beb42",
"name": "AAM Devices Profile Meta Data",
"description": "Activation profile export dataset",
"fileDescription": {
"persisted": true,
"containerFormat": "parquet",
"format": "parquet"
},
"aspect": "production",
"state": "DRAFT"
},
{
"id": "5ef3e328582424191b1beb44",
"name": "AAM Realtime",
"description": "Activation profile export dataset",
"fileDescription": {
"persisted": true,
"containerFormat": "parquet",
"format": "parquet"
},
"aspect": "production",
"state": "DRAFT"
},
{
"id": "5ef3e328fe742a191b2b3ea5",
"name": "AAM Realtime Profile Updates",
"description": "Activation profile export dataset",
"fileDescription": {
"persisted": true,
"containerFormat": "parquet",
"format": "parquet"
},
"aspect": "production",
"state": "DRAFT"
}
],
"pageInfo": {
"start": 0,
"end": 4,
"total": 149,
"hasNext": true
}
}
成功的响应包含符合激活条件的数据集列表。 在下一步中构建源连接时,可以使用这些数据集。
有关每个返回的数据集的各种响应参数的信息,请参阅 数据集API开发人员文档.
在检索要导出的数据集列表后,您可以使用这些数据集ID创建源连接。
请求
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/sourceConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Connecting to Data Lake",
"description": "Data Lake source connection to export datasets",
"connectionSpec": {
"id": "23598e46-f560-407b-88d5-ea6207e49db0", // this connection spec ID is always the same for Source Connections
"version": "1.0"
},
"params": {
"datasets": [ // datasets to activate
{
"dataSetId": "5ef3e3259ad2a1191ab7dd7d",
"name": "AAM Devices Data"
}
]
}
}'
响应
{
"id": "900df191-b983-45cd-90d5-4c7a0326d650",
"etag": "\"0500ebe1-0000-0200-0000-63e28d060000\""
}
成功的响应会返回ID (id
)和 etag
. 记下源连接ID,因为稍后创建数据流时会需要它。
另请记住:
基本连接会将凭据安全地存储到您的目标。 根据目标类型,对该目标进行身份验证所需的凭据可能有所不同。 要查找这些验证参数,请先检索 connection spec (对于所需的目标,请参阅部分) 收集连接规格和流量规格 然后查看 authSpec
响应的。 请参考下面的选项卡 authSpec
所有受支持目标的属性。
请注意中带有内联注释的高亮显示行 connection spec 下面示例,其中提供了有关在何处查找身份验证参数的更多信息, connection spec.
{
"items": [
{
"id": "4fce964d-3f37-408f-9778-e597338a21ee",
"name": "Amazon S3",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "Access Key",
"type": "KeyBased",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Defines auth params required for connecting to amazon-s3",
"type": "object",
"properties": {
"s3AccessKey": {
"description": "Access key id",
"type": "string",
"pattern": "^[A-Z2-7]{20}$"
},
"s3SecretKey": {
"description": "Secret access key for the user account",
"type": "string",
"format": "password",
"pattern": "^[A-Za-z0-9\/\\+]{40}$"
}
},
"required": [
"s3SecretKey",
"s3AccessKey"
]
}
}
],
//...
请注意中带有内联注释的高亮显示行 connection spec 下面示例,其中提供了有关在何处查找身份验证参数的更多信息, connection spec.
{
"items": [
{
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2",
"name": "Azure Blob Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "ConnectionString",
"type": "ConnectionString",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Connection String for Azure Blob based destinations",
"type": "object",
"properties": {
"connectionString": {
"description": "connection string for login",
"type": "string",
"format": "password"
}
},
"required": [
"connectionString"
]
}
}
],
//...
请注意中带有内联注释的高亮显示行 connection spec 下面示例,其中提供了有关在何处查找身份验证参数的更多信息, connection spec.
{
"items": [
{
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1",
"name": "Azure Data Lake Gen2",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "Azure Service Principal Auth",
"type": "AzureServicePrincipal",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "defines auth params required for connecting to adlsgen2 using service principal",
"type": "object",
"properties": {
"url": {
"description": "Endpoint for Azure Data Lake Storage Gen2.",
"type": "string"
},
"servicePrincipalId": {
"description": "Service Principal Id to connect to ADLSGen2.",
"type": "string"
},
"servicePrincipalKey": {
"description": "Service Principal Key to connect to ADLSGen2.",
"type": "string",
"format": "password"
},
"tenant": {
"description": "Tenant information(domain name or tenant ID).",
"type": "string"
}
},
"required": [
"servicePrincipalKey",
"url",
"tenant",
"servicePrincipalId"
]
}
}
],
//...
数据登陆区目标不需要 auth spec.
{
"items": [
{
"id": "10440537-2a7b-4583-ac39-ed38d4b848e8",
"name": "Data Landing Zone",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [],
//...
请注意中带有内联注释的高亮显示行 connection spec 下面示例,其中提供了有关在何处查找身份验证参数的更多信息, connection spec.
{
"items": [
{
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99",
"name": "Google Cloud Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "Google Cloud Storage authentication credentials",
"type": "GoogleCloudStorageAuth",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "defines auth params required for connecting to google cloud storage connector.",
"type": "object",
"properties": {
"accessKeyId": {
"description": "Access Key Id for the user account",
"type": "string"
},
"secretAccessKey": {
"description": "Secret Access Key for the user account",
"type": "string",
"format": "password"
}
},
"required": [
"accessKeyId",
"secretAccessKey"
]
}
}
],
//...
SFTP目标包含两个单独的项,分别位于 auth spec,因为它同时支持密码和SSH密钥身份验证。
请注意中带有内联注释的高亮显示行 connection spec 下面示例,其中提供了有关在何处查找身份验证参数的更多信息, connection spec.
{
"items": [
{
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26",
"name": "SFTP",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [ // describes the authentication parameters
{
"name": "SFTP with Password",
"type": "SFTP",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "defines auth params required for connecting to sftp locations with a password",
"type": "object",
"properties": {
"domain": {
"description": "Domain of server",
"type": "string"
},
"username": {
"description": "Username",
"type": "string"
},
"password": {
"description": "Password",
"type": "string",
"format": "password"
}
},
"required": [
"password",
"domain",
"username"
]
}
},
{
"name": "SFTP with SSH Key",
"type": "SFTP",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "defines auth params required for connecting to sftp locations using SSH Key",
"type": "object",
"properties": {
"domain": {
"description": "Domain of server",
"type": "string"
},
"username": {
"description": "Username",
"type": "string"
},
"sshKey": {
"description": "Base64 string of the private SSH key",
"type": "string",
"format": "password",
"contentEncoding": "base64",
"uiAttributes": {
"tooltip": {
"id": "platform_destinations_connect_sftp_ssh",
"fallbackUrl": "http://www.adobe.com/go/destinations-sftp-connection-parameters-en "
}
}
}
},
"required": [
"sshKey",
"domain",
"username"
]
}
}
],
//...
使用身份验证规范中指定的属性(即 authSpec
(从响应中)可以使用特定于每种目标类型的所需凭据创建基本连接,如下面的示例所示:
请求
有关如何获取所需身份验证凭据的信息,请参阅 向目标进行身份验证 Amazon S3目标文档页面的部分。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: <API-KEY>' \
--header 'x-gw-ims-org-id: <IMS-ORG-ID>' \
--header 'x-sandbox-name: <SANDBOX-NAME>' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Amazon S3 Base Connection",
"auth": {
"specName": "Access Key",
"params": {
"s3SecretKey": "<Add secret key>",
"s3AccessKey": "<Add access key>"
}
},
"connectionSpec": {
"id": "4fce964d-3f37-408f-9778-e597338a21ee", // Amazon S3 connection spec
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需身份验证凭据的信息,请参阅 向目标进行身份验证 Azure Blob存储目标文档页面的部分。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: <API-KEY>' \
--header 'x-gw-ims-org-id: <IMS-ORG-ID>' \
--header 'x-sandbox-name: <SANDBOX-NAME>' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Azure Blob Storage Base Connection",
"auth": {
"specName": "ConnectionString",
"params": {
"connectionString": "<Add Azure Blob connection string>"
}
},
"connectionSpec": {
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2", // Azure Blob Storage connection spec
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需身份验证凭据的信息,请参阅 向目标进行身份验证 Azure Data Lake Gen 2(ADLS Gen2)目标文档页面的部分。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: <API-KEY>' \
--header 'x-gw-ims-org-id: <IMS-ORG-ID>' \
--header 'x-sandbox-name: <SANDBOX-NAME>' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Azure Data Lake Gen 2(ADLS Gen2) Base Connection",
"auth": {
"specName": "Azure Service Principal Auth",
"params": {
"servicePrincipalKey": "<Add servicePrincipalKey>",
"url": "<Add url>",
"tenant": "<Add tenant>",
"servicePrincipalId": "<Add servicePrincipalId>"
}
},
"connectionSpec": {
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1", // Azure Data Lake Gen 2(ADLS Gen2) connection spec
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
数据登陆区目标不需要身份验证凭据。 欲了解更多信息,请参见 向目标进行身份验证 数据登陆区目标文档页面的部分。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: <API-KEY>' \
--header 'x-gw-ims-org-id: <IMS-ORG-ID>' \
--header 'x-sandbox-name: <SANDBOX-NAME>' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Data Landing Zone(DLZ) Base Connection"
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需身份验证凭据的信息,请参阅 向目标进行身份验证 Google Cloud Storage目标文档页面的部分。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: <API-KEY>' \
--header 'x-gw-ims-org-id: <IMS-ORG-ID>' \
--header 'x-sandbox-name: <SANDBOX-NAME>' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "Google Cloud Storage Base Connection",
"auth": {
"specName": "Google Cloud Storage authentication credentials",
"params": {
"accessKeyId": "<Add accessKeyId>",
"secretAccessKey": "<Add secret Access Key>"
}
},
"connectionSpec": {
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99", // Google Cloud Storage connection spec
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需身份验证凭据的信息,请参阅 向目标进行身份验证 SFTP目标文档页面的部分。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: <API-KEY>' \
--header 'x-gw-ims-org-id: <IMS-ORG-ID>' \
--header 'x-sandbox-name: <SANDBOX-NAME>' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "SFTP with password Base Connection",
"auth": {
"specName": "SFTP with Password",
"params": {
"domain": "<Add domain>",
"username": "<Add username>",
"password": "<Add password>"
}
},
"connectionSpec": {
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26", // SFTP connection spec
"version": "1.0"
}
}'
有关如何获取所需身份验证凭据的信息,请参阅 向目标进行身份验证 SFTP目标文档页面的部分。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/connections' \
--header 'accept: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'x-api-key: <API-KEY>' \
--header 'x-gw-ims-org-id: <IMS-ORG-ID>' \
--header 'x-sandbox-name: <SANDBOX-NAME>' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "SFTP with SSH key Base Connection",
"auth": {
"specName": "SFTP with SSH Key",
"params": {
"domain": "<Add domain>",
"username": "<Add username>",
"sshKey": "<Add SSH key>"
}
},
"connectionSpec": {
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26", // SFTP connection spec
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
记下响应中的连接ID。 创建目标连接时,此ID将在下一步中是必需的。
接下来,您需要创建一个目标连接,用于存储数据集的导出参数。 导出参数包括位置、文件格式、压缩和其他详细信息。 请参阅 targetSpec
目标的连接规范中提供的属性,用于了解每个目标类型支持的属性。 请参考下面的选项卡 targetSpec
所有受支持目标的属性。
仅在压缩模式下支持导出到JSON文件。 导出到 Parquet 文件在压缩和未压缩模式下受支持。
请注意中带有内联注释的高亮显示行 connection spec 下面的示例,其中提供了有关在何处查找 target spec 连接规范中的参数。 您还可以在以下示例中看到以下目标参数 非 适用于数据集导出目标。
{
"items": [
{
"id": "4fce964d-3f37-408f-9778-e597338a21ee",
"name": "Amazon S3",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"bucketName": {
"title": "Bucket name",
"description": "Bucket name",
"type": "string",
"pattern": "(?=^.{3,63}$)(?!^(\\d+\\.)+\\d+$)(^(([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])\\.)*([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])$)",
"uiAttributes": {
"tooltip": {
"id": "platform_destinations_connect_s3_bucket",
"fallbackUrl": "http://www.adobe.com/go/destinations-amazon-s3-connection-parameters-en"
}
}
},
"path": {
"title": "Folder path",
"description": "Output path for copying files",
"type": "string",
"pattern": "^[0-9a-zA-Z\/\\!\\-_\\.\\*\\''\\(\\)]*((\\%SEGMENT_(NAME|ID)\\%)?\/?)+$",
"uiAttributes": {
"tooltip": {
"id": "platform_destinations_connect_s3_folderpath",
"fallbackUrl": "http://www.adobe.com/go/destinations-amazon-s3-connection-parameters-en"
}
}
},
"fileType": {...}, // not applicable to dataset destinations
"datasetFileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {...}, // not applicable to dataset destinations
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"bucketName",
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
请注意中带有内联注释的高亮显示行 connection spec 下面的示例,其中提供了有关在何处查找 target spec 连接规范中的参数。 您还可以在以下示例中看到以下目标参数 非 适用于数据集导出目标。
{
"items": [
{
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2",
"name": "Azure Blob Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"path": {
"title": "Folder path",
"description": "Output path (relative) indicating where to upload the data",
"type": "string",
"pattern": "^[0-9a-zA-Z\/\\!\\-_\\.\\*\\'\\(\\)]+$"
},
"container": {
"title": "Container",
"description": "Container within the storage where to upload the data",
"type": "string",
"pattern": "^[a-z0-9](?!.*--)[a-z0-9-]{1,61}[a-z0-9]$"
},
"fileType": {...}, // not applicable to dataset destinations
"datasetFileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {...}, // not applicable to dataset destinations
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"container",
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
请注意中带有内联注释的高亮显示行 connection spec 下面的示例,其中提供了有关在何处查找 target spec 连接规范中的参数。 您还可以在以下示例中看到以下目标参数 非 适用于数据集导出目标。
{
"items": [
{
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1",
"name": "Azure Data Lake Gen2",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"path": {
"title": "Folder path",
"description": "Enter the path to your Azure Data Lake Storage folder",
"type": "string"
},
"fileType": {...}, // not applicable to dataset destinations
"datasetFileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions":{...}, // not applicable to dataset destinations
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
请注意中带有内联注释的高亮显示行 connection spec 下面的示例,其中提供了有关在何处查找 target spec 连接规范中的参数。 您还可以在以下示例中看到以下目标参数 非 适用于数据集导出目标。
"items": [
{
"id": "10440537-2a7b-4583-ac39-ed38d4b848e8",
"name": "Data Landing Zone",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [],
"encryptionSpecs": [],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"path": {
"title": "Folder path",
"description": "Enter the path to your Azure Data Lake Storage folder",
"type": "string"
},
"fileType": {...}, // not applicable to dataset destinations
"datasetFileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {...}, // not applicable to dataset destinations
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
请注意中带有内联注释的高亮显示行 connection spec 下面的示例,其中提供了有关在何处查找 target spec 连接规范中的参数。 您还可以在以下示例中看到以下目标参数 非 适用于数据集导出目标。
{
"items": [
{
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99",
"name": "Google Cloud Storage",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"bucketName": {
"title": "Bucket name",
"description": "Bucket name",
"type": "string",
"pattern": "(?!^goog.*$)(?!^.*g(o|0)(o|0)gle.*$)(((?=^.{3,63}$)(^([a-z0-9]|[a-z0-9][a-z0-9\\-_]*)[a-z0-9]$))|((?=^.{3,222}$)(?!^(\\d+\\.)+\\d+$)(^(([a-z0-9]{1,63}|[a-z0-9][a-z0-9\\-_]{1,61}[a-z0-9])\\.)*([a-z0-9]{1,63}|[a-z0-9][a-z0-9\\-_]{1,61}[a-z0-9])$)))"
},
"path": {
"title": "Folder path",
"description": "Output path for copying files",
"type": "string",
"pattern": "^[0-9a-zA-Z\/\\!\\-_\\.\\*\\''\\(\\)]*((\\%SEGMENT_(NAME|ID)\\%)?\/?)+$"
},
"fileType": {...}, // not applicable to dataset destinations
"datasetFileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {...}, // not applicable to dataset destinations
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"NONE",
"GZIP"
]
}
},
"required": [
"bucketName",
"path",
"datasetFileType",
"compression",
"fileType"
]
}
//...
请注意中带有内联注释的高亮显示行 connection spec 下面的示例,其中提供了有关在何处查找 target spec 连接规范中的参数。 您还可以在以下示例中看到以下目标参数 非 适用于数据集导出目标。
{
"items": [
{
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26",
"name": "SFTP",
"providerId": "14e34fac-d307-11e9-bb65-2a2ae2dbcce4",
"version": "1.0",
"authSpec": [...],
"encryptionSpecs": [...],
"targetSpec": { // describes the target connection parameters
"name": "User based target",
"type": "UserNamespace",
"spec": {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"remotePath": {
"title": "Folder path",
"description": "Enter your folder path",
"type": "string"
},
"fileType": {...}, // not applicable to dataset destinations
"datasetFileType": {
"conditional": {
"field": "flowSpec.attributes._workflow",
"operator": "CONTAINS",
"value": "DATASETS"
},
"title": "File Type",
"description": "Select file format",
"type": "string",
"enum": [
"JSON",
"PARQUET"
]
},
"csvOptions": {...}, // not applicable to dataset destinations
"compression": {
"title": "Compression format",
"description": "Select the desired file compression format.",
"type": "string",
"enum": [
"GZIP",
"NONE"
]
}
},
"required": [
"remotePath",
"datasetFileType",
"compression",
"fileType"
]
},
//...
通过使用上述规范,您可以构建特定于所需云存储目标的目标连接请求,如下面的选项卡所示。
请求
有关如何获取所需目标参数的信息,请参阅 填写目标详细信息 的部分 Amazon S3 目标文档页面。
对于其他支持的值 datasetFileType
,请参阅API参考文档。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Amazon S3 Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_TARGET_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"bucketName": "your-bucket-name",
"path": "folder/subfolder",
"compression": "NONE",
"datasetFileType": "JSON"
},
"connectionSpec": {
"id": "4fce964d-3f37-408f-9778-e597338a21ee", // Amazon S3 connection spec id
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需目标参数的信息,请参阅 填写目标详细信息 的部分 Azure Blob Storage 目标文档页面。
对于其他支持的值 datasetFileType
,请参阅API参考文档。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Azure Blob Storage Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_TARGET_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"container": "your-container-name",
"path": "folder/subfolder",
"compression": "NONE",
"datasetFileType": "JSON"
},
"connectionSpec": {
"id": "6d6b59bf-fb58-4107-9064-4d246c0e5bb2", // Azure Blob Storage connection spec id
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需目标参数的信息,请参阅 填写目标详细信息 Azure的部分 Data Lake Gen 2(ADLS Gen2) 目标文档页面。
对于其他支持的值 datasetFileType
,请参阅API参考文档。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Azure Data Lake Gen 2(ADLS Gen2) Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_TARGET_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"path": "folder/subfolder",
"compression": "NONE",
"datasetFileType": "JSON"
},
"connectionSpec": {
"id": "be2c3209-53bc-47e7-ab25-145db8b873e1", // Azure Data Lake Gen 2(ADLS Gen2) connection spec id
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需目标参数的信息,请参阅 填写目标详细信息 的部分 Data Landing Zone 目标文档页面。
对于其他支持的值 datasetFileType
,请参阅API参考文档。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Data Landing Zone Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_TARGET_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"path": "folder/subfolder",
"compression": "NONE",
"datasetFileType": "JSON"
},
"connectionSpec": {
"id": "10440537-2a7b-4583-ac39-ed38d4b848e8", // Data Landing Zone connection spec id
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需目标参数的信息,请参阅 填写目标详细信息 的部分 Google Cloud Storage 目标文档页面。
对于其他支持的值 datasetFileType
,请参阅API参考文档。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Google Cloud Storage Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_TARGET_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"bucketName": "your-bucket-name",
"path": "folder/subfolder",
"compression": "NONE",
"datasetFileType": "JSON"
},
"connectionSpec": {
"id": "c5d93acb-ea8b-4b14-8f53-02138444ae99", // Google Cloud Storage connection spec id
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
有关如何获取所需目标参数的信息,请参阅 填写目标详细信息 SFTP目标文档页面的部分。
对于其他支持的值 datasetFileType
,请参阅API参考文档。
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/targetConnections' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "SFTP Target Connection",
"baseConnectionId": "<FROM_STEP_CREATE_TARGET_BASE_CONNECTION>",
"params": {
"mode": "Server-to-server",
"remotePath": "folder/subfolder",
"compression": "NONE",
"datasetFileType": "JSON"
},
"connectionSpec": {
"id": "36965a81-b1c6-401b-99f8-22508f1e6a26", // SFTP connection spec id
"version": "1.0"
}
}'
响应
{
"id": "12401496-2573-4ca7-8137-fef1aeb9dd4c",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
记下响应中的Target连接ID。 创建数据流以导出数据集时,在下一步中需要此ID。
目标配置的最后一步是设置数据流。 数据流将以前创建的实体连接在一起,还提供了用于配置数据集导出计划的选项。 要创建数据流,请根据所需的云存储目标使用以下有效负载,并替换之前步骤中的实体ID。
请求
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate datasets to an Amazon S3 cloud storage destination",
"description": "This operation creates a dataflow to export datasets to an Amazon S3 cloud storage destination",
"flowSpec": {
"id": "269ba276-16fc-47db-92b0-c1049a3c131f", // Amazon S3 flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": [],
"scheduleParams": { // specify the scheduling info
"interval": 3, // also supports 6, 9, 12 hour increments
"timeUnit": "hour", // also supports "day" for daily increments. Use "interval": 1 when you select "timeUnit": "day"
"startTime": 1675901210 // UNIX timestamp start time (in seconds)
}
}'
响应
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate datasets to an Azure Blob Storage cloud storage destination",
"description": "This operation creates a dataflow to export datasets to an Azure Blob Storage cloud storage destination",
"flowSpec": {
"id": "95bd8965-fc8a-4119-b9c3-944c2c2df6d2", // Azure Blob Storage flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": [],
"scheduleParams": { // specify the scheduling info
"interval": 3, // also supports 6, 9, 12, 24 hour increments
"timeUnit": "hour",
"startTime": 1675901210 // UNIX timestamp start time(in seconds)
}
}'
响应
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate datasets to an Azure Data Lake Gen 2(ADLS Gen2) cloud storage destination",
"description": "This operation creates a dataflow to export datasets to an Azure Data Lake Gen 2(ADLS Gen2) cloud storage destination",
"flowSpec": {
"id": "17be2013-2549-41ce-96e7-a70363bec293", // Azure Data Lake Gen 2(ADLS Gen2) flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": [],
"scheduleParams": { // specify the scheduling info
"interval": 3, // also supports 6, 9, 12, 24 hour increments
"timeUnit": "hour",
"startTime": 1675901210 // UNIX timestamp start time(in seconds)
}
}'
响应
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate datasets to a Data Landing Zone cloud storage destination",
"description": "This operation creates a dataflow to export datasets to a Data Landing Zone cloud storage destination",
"flowSpec": {
"id": "cd2fc47e-e838-4f38-a581-8fff2f99b63a", // Data Landing Zone flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": [],
"scheduleParams": { // specify the scheduling info
"interval": 3, // also supports 6, 9, 12, 24 hour increments
"timeUnit": "hour",
"startTime": 1675901210 // UNIX timestamp start time(in seconds)
}
}'
响应
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate datasets to a Google Cloud Storage cloud storage destination",
"description": "This operation creates a dataflow to export datasets to a Google Cloud Storage destination",
"flowSpec": {
"id": "585c15c4-6cbf-4126-8f87-e26bff78b657", // Google Cloud Storage flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": [],
"scheduleParams": { // specify the scheduling info
"interval": 3, // also supports 6, 9, 12, 24 hour increments
"timeUnit": "hour",
"startTime": 1675901210 // UNIX timestamp start time(in seconds)
}
}'
响应
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
请求
请注意请求示例中带有内联注释的高亮显示行,这些行提供了更多信息。 将请求复制粘贴到您选择的终端时,删除请求中的内联注释。
curl --location --request POST 'https://platform.adobe.io/data/foundation/flowservice/flows' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--data-raw '{
"name": "Activate datasets to an SFTP cloud storage destination",
"description": "This operation creates a dataflow to export datasets to an SFTP cloud storage destination",
"flowSpec": {
"id": "354d6aad-4754-46e4-a576-1b384561c440", // SFTP flow spec ID
"version": "1.0"
},
"sourceConnectionIds": [
"<FROM_STEP_CREATE_SOURCE_CONNECTION>"
],
"targetConnectionIds": [
"<FROM_STEP_CREATE_TARGET_CONNECTION>"
],
"transformations": [],
"scheduleParams": { // specify the scheduling info
"interval": 3, // also supports 6, 9, 12, 24 hour increments
"timeUnit": "hour",
"startTime": 1675901210 // UNIX timestamp start time(in seconds)
}
}'
响应
{
"id": "eb54b3b3-3949-4f12-89c8-64eafaba858f",
"etag": "\"0000d781-0000-0200-0000-63e29f420000\""
}
记下响应中的数据流ID。 检索数据流运行时在下一步中需要此ID来验证成功的日期集导出。
要检查数据流的执行,请使用数据流运行API:
请求
在检索数据流运行的请求中,在创建数据流时,将您在上一步中获取的数据流ID添加为查询参数。
curl --location --request GET 'https://platform.adobe.io/data/foundation/flowservice/runs?property=flowId==eb54b3b3-3949-4f12-89c8-64eafaba858f' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}' \
--header 'x-gw-ims-org-id: {ORG_ID}' \
--header 'x-sandbox-name: {SANDBOX_NAME}' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
响应
{
"items": [
{
"id": "4b7728dd-83c9-4c38-95a4-24ddab545404",
"createdAt": 1675807718296,
"updatedAt": 1675807731834,
"createdBy": "aep_activation_batch@AdobeID",
"updatedBy": "acp_foundation_connectors@AdobeID",
"createdClient": "aep_activation_batch",
"updatedClient": "acp_foundation_connectors",
"sandboxId": "7dfdcd30-0a09-11ea-8ea6-7bf93ce86c28",
"sandboxName": "sand-1",
"imsOrgId": "5555467B5D8013E50A494220@AdobeOrg",
"flowId": "aae5ec63-b0ac-4808-9a44-abf2ea67bd5a",
"flowSpec": {
"id": "615d3489-36d2-4671-9467-4ae1129facd3",
"version": "1.0"
},
"providerRefId": "ba56f98e0c49b572adb249980c39b1c7",
"etag": "\"08005e9e-0000-0200-0000-63e2cbf30000\"",
"metrics": {
"durationSummary": {
"startedAtUTC": 1675807719411,
"completedAtUTC": 1675807719416
},
"sizeSummary": {
"inputBytes": 0
},
"recordSummary": {
"inputRecordCount": 0,
"skippedRecordCount": 0,
"sourceSummaries": [
{
"id": "ea2b1205-4692-49de-b448-ebf75b1d188a",
"inputRecordCount": 0,
"skippedRecordCount": 0,
"entitySummaries": [
{
//...
您可以找到有关 数据流运行API返回的各种参数 API参考文档中的。
导出数据集时,Experience Platform会创建 .json
或 .parquet
文件存储位置。 预计在下列情况下,将根据您提供的导出计划在您的存储位置存放新文件: 创建数据流.
Experience Platform会在您指定的存储位置创建一个文件夹结构,存放导出的数据集文件。 每次导出时都会创建一个新文件夹,其模式如下所示:
folder-name-you-provided/datasetID/exportTime=YYYYMMDDHHMM
默认文件名是随机生成的,并确保导出的文件名是唯一的。
这些文件在存储位置中的存在是成功导出的确认。 要了解导出文件的结构形式,您可以下载示例 .parquet文件 或 .json文件.
在到步骤中 创建目标连接中,您可以选择要压缩的导出数据集文件。
请注意两种文件类型在压缩后的文件格式差异:
json.gz
gz.parquet
本教程中的API端点遵循常规Experience PlatformAPI错误消息原则。 请参阅 API状态代码 和 请求标头错误 有关解释错误响应的更多信息,请参阅平台故障排除指南。
通过学习本教程,您已成功将Platform连接到其中一个首选批量云存储目标,并将数据流设置到相应的目标以导出数据集。 有关更多详细信息,请参阅以下页面,例如如何使用流服务API编辑现有数据流: