Documentation Experience Platform Source Connectors Guide

Update dataflows using the Flow Service API

Last update: April 2, 2025

Topics:
Sources

CREATED FOR:

Developer

This tutorial covers the steps for updating a dataflow, including its basic information, schedule, and mapping sets using the Flow Service API.

TIP

Your source connection and your target connection should be mapped to a single dataflow. You should not update your source and target connections separately, as the changes will not be reflected in their corresponding dataflow. If your use case requires an update of your source and target connections, then you must create a new pair of source and target connections, as well as a new dataflow.

Getting started

This tutorial requires you to have a valid flow ID. If you do not have a valid flow ID, select your connector of choice from the sources overview and follow the steps outlined before attempting this tutorial.

This tutorial also requires you to have a working understanding of the following components of Adobe Experience Platform:

Sources: Experience Platform allows data to be ingested from various sources while providing you with the ability to structure, label, and enhance incoming data using Experience Platform services.
Sandboxes: Experience Platform provides virtual sandboxes which partition a single Experience Platform instance into separate virtual environments to help develop and evolve digital experience applications.

Using Experience Platform APIs

For information on how to successfully make calls to Experience Platform APIs, see the guide on getting started with Experience Platform APIs.

Look up dataflow details

The first step in updating your dataflow is to retrieve dataflow details using your flow ID. You can view the current details of an existing dataflow by making a GET request to the /flows endpoint.

API format

GET /flows/{FLOW_ID}

Parameter	Description
`{FLOW_ID}`	The unique `id` value for the dataflow you want to retrieve.

Request

The following request retrieves updated information regarding your flow ID.

curl -X GET \
  'https://platform.adobe.io/data/foundation/flowservice/flows/2edc08ac-4df5-4fe6-936f-81a19ce92f5c' \
  -H 'Authorization: Bearer {ACCESS_TOKEN}' \
  -H 'x-api-key: {API_KEY}' \
  -H 'x-gw-ims-org-id: {ORG_ID}' \
  -H 'x-sandbox-name: {SANDBOX_NAME}'

Response

A successful response returns the current details of your dataflow including its version, schedule, and unique identifier (id).

{
    "items": [
        {
            "id": "2edc08ac-4df5-4fe6-936f-81a19ce92f5c",
            "createdAt": 1612310475905,
            "updatedAt": 1614122324830,
            "createdBy": "{CREATED_BY}",
            "updatedBy": "{UPDATED_BY}",
            "createdClient": "{CREATED_CLIENT}",
            "updatedClient": "{UPDATED_CLIENT}",
            "sandboxId": "{SANDBOX_ID}",
            "sandboxName": "{SANDBOX_NAME}",
            "imsOrgId": "{ORG_ID}",
            "name": "Database dataflow using BigQuery",
            "description": "collecting test1.Mytable from Google BigQuery",
            "flowSpec": {
                "id": "14518937-270c-4525-bdec-c2ba7cce3860",
                "version": "1.0"
            },
            "state": "enabled",
            "version": "\"5400d99c-0000-0200-0000-60358d540000\"",
            "etag": "\"5400d99c-0000-0200-0000-60358d540000\"",
            "sourceConnectionIds": [
                "b7581b59-c603-4df1-a689-d23d7ac440f3"
            ],
            "targetConnectionIds": [
                "320f119a-5ac1-4ab1-88ea-eb19e674ea2e"
            ],
            "inheritedAttributes": {
                "sourceConnections": [
                    {
                        "id": "b7581b59-c603-4df1-a689-d23d7ac440f3",
                        "connectionSpec": {
                            "id": "3c9b37f8-13a6-43d8-bad3-b863b941fedd",
                            "version": "1.0"
                        },
                        "baseConnection": {
                            "id": "6990abad-977d-41b9-a85d-17ea8cf1c0e4",
                            "connectionSpec": {
                                "id": "3c9b37f8-13a6-43d8-bad3-b863b941fedd",
                                "version": "1.0"
                            }
                        }
                    }
                ],
                "targetConnections": [
                    {
                        "id": "320f119a-5ac1-4ab1-88ea-eb19e674ea2e",
                        "connectionSpec": {
                            "id": "c604ff05-7f1a-43c0-8e18-33bf874cb11c",
                            "version": "1.0"
                        }
                    }
                ]
            },
            "scheduleParams": {
                "startTime": "1612310466",
                "frequency": "week",
                "interval": "15",
                "backfill": "true"
            },
            "transformations": [
                {
                    "name": "Copy",
                    "params": {
                        "deltaColumn": {
                            "name": "Datefield",
                            "dateFormat": "YYYY-MM-DD",
                            "timezone": "UTC"
                        }
                    }
                },
                {
                    "name": "Mapping",
                    "params": {
                        "mappingId": "0b090130b58b4819afc78b6dc98b484d",
                        "mappingVersion": "0"
                    }
                }
            ],
            "runs": "/flows/2edc08ac-4df5-4fe6-936f-81a19ce92f5c/runs",
            "lastOperation": {
                "started": 1614122316652,
                "updated": 1614122324830,
                "percentCompleted": 100.0,
                "status": {
                    "value": "completed",
                    "errors": []
                },
                "ops": [
                    {
                        "op": "replace",
                        "path": "/scheduleParams/frequency",
                        "value": "week"
                    }
                ],
                "operation": "update"
            },
            "lastRunDetails": {
                "id": "a10cc80b-fbea-4c6b-873e-d7fd32f4d12d",
                "state": "success",
                "startedAtUTC": 1613079975512,
                "completedAtUTC": 1613080027511
            }
        }
    ]
}

Update dataflow

To update your dataflow’s run schedule, name, and description, perform a PATCH request to the Flow Service API while providing your flow ID, version, and the new schedule you want to use.

IMPORTANT

The If-Match header is required when making a PATCH request. The value for this header is the unique version of the connection you want to update. The etag value updates with every successful update of a dataflow.

API format

PATCH /flows/{FLOW_ID}

Request

The following request updates your flow run schedule, as well as your dataflow’s name and description.

curl -X PATCH \
    'https://platform.adobe.io/data/foundation/flowservice/flows/2edc08ac-4df5-4fe6-936f-81a19ce92f5c' \
    -H 'Authorization: Bearer {ACCESS_TOKEN}' \
    -H 'x-api-key: {API_KEY}' \
    -H 'x-gw-ims-org-id: {ORG_ID}' \
    -H 'x-sandbox-name: {SANDBOX_NAME}'
    -H 'If-Match: "1a0037e4-0000-0200-0000-602e06f60000"' \
    -d '[
            {
                "op": "replace",
                "path": "/scheduleParams/frequency",
                "value": "day"
            },
            {
                "op": "replace",
                "path": "/name",
                "value": "Database Dataflow Feb2021"
            },
            {
                "op": "replace",
                "path": "/description",
                "value": "Database dataflow for testing update API"
            }
        ]'

Property	Description
`op`	The operation call used to define the action needed to update the dataflow. Operations include: `add`, `replace`, and `remove`.
`path`	Defines the part of the flow that is to be updated.
`value`	The new value you want to update your parameter with.

Response

A successful response returns your flow ID and an updated etag. You can verify the update by making a GET request to the Flow Service API, while providing your flow ID.

{
    "id": "2edc08ac-4df5-4fe6-936f-81a19ce92f5c",
    "etag": "\"50014cc8-0000-0200-0000-6036eb720000\""
}

Update mapping

You can update the mapping set of an existing dataflow by making a PATCH request to the Flow Service API and providing updated values for your mappingId and mappingVersion.

API format

PATCH /flows/{FLOW_ID}

Request

The following request updates your dataflow’s mapping set.

curl -X PATCH \
    'https://platform.adobe.io/data/foundation/flowservice/flows/2edc08ac-4df5-4fe6-936f-81a19ce92f5c' \
    -H 'Authorization: Bearer {ACCESS_TOKEN}' \
    -H 'x-api-key: {API_KEY}' \
    -H 'x-gw-ims-org-id: {ORG_ID}' \
    -H 'x-sandbox-name: {SANDBOX_NAME}'
    -H 'If-Match: "50014cc8-0000-0200-0000-6036eb720000"' \
    -d '[
        {
            "op": "replace",
            "path": "/transformations/0",
            "value": {
                "name": "Mapping",
                "params": {
                    "mappingId": "c5f22f04e09f44498e528901546a83b1",
                    "mappingVersion": 2
                }
            }
        }
    ]'

Property	Description
`op`	The operation call used to define the action needed to update the dataflow. Operations include: `add`, `replace`, and `remove`.
`path`	Defines the part of the flow that is to be updated. In this example, `transformations` is being updated.
`value.name`	The name of the property that is to be updated.
`value.params.mappingId`	The new mapping ID to be used to update the mapping set of the dataflow.
`value.params.mappingVersion`	The new mapping version associated with the updated mapping ID.

Response

A successful response returns your flow ID and an updated etag. You can verify the update by making a GET request to the Flow Service API, while providing your flow ID.

{
    "id": "2edc08ac-4df5-4fe6-936f-81a19ce92f5c",
    "etag": "\"2c000802-0000-0200-0000-613976440000\""
}

Next steps

By following this tutorial, you have updated the basic information, schedule, and mapping sets of your dataflow using the Flow Service API. For more information on using source connectors, see the sources overview.

recommendation-more-help