Explore your cloud storage folders using the Flow Service API

Last update: April 2, 2025

CREATED FOR:

  • Developer

This tutorial provides steps on how to explore and preview the structure and contents of your cloud storage using the Flow Service API.

NOTE
In order to explore your cloud storage, you must already have a valid base connection ID for a cloud storage source. If you do not have this ID, then see the sources overview for a list of cloud storage sources that you can create a base connection with.

Getting started

This guide requires a working understanding of the following components of Adobe Experience Platform:

  • Sources: Experience Platform allows data to be ingested from various sources while providing you with the ability to structure, label, and enhance incoming data using Experience Platform services.
  • Sandboxes: Experience Platform provides virtual sandboxes which partition a single Experience Platform instance into separate virtual environments to help develop and evolve digital experience applications.

Using Experience Platform APIs

For information on how to successfully make calls to Experience Platform APIs, see the guide on getting started with Experience Platform APIs.

Explore your cloud storage folders

You can retrieve information on the structure of your cloud storage folders by making a GET request to the Flow Service API while providing the base connection ID of your source.

When performing GET requests to explore your cloud storage, you must include the query parameters that are listed in the table below:

ParameterDescription
objectType

The type of object that you wish to explore. Set this value as either:

  • folder: Explore a specific directory
  • root: Explore the root directory.
objectThis parameter is required only when viewing a specific directory. Its value represents the path of the directory you wish to explore.

API format

GET /connections/{BASE_CONNECTION_ID}/explore?objectType=root
GET /connections/{BASE_CONNECTION_ID}/explore?objectType=folder&object={PATH}
ParameterDescription
{BASE_CONNECTION_ID}The base connection ID of your cloud storage source.
{PATH}The path of a directory.

Request

curl -X GET \
  'http://platform.adobe.io/data/foundation/flowservice/connections/dc3c0646-5e30-47be-a1ce-d162cb8f1f07/explore?objectType=folder&object=root' \
  -H 'Authorization: Bearer {ACCESS_TOKEN}' \
  -H 'x-api-key: {API_KEY}' \
  -H 'x-gw-ims-org-id: {ORG_ID}' \
  -H 'x-sandbox-name: {SANDBOX_NAME}'

Response

A successful response returns an array of files and folders found within the queried directory. Take note of the path property of the file you wish you upload, as you are required to provide it in the next step to inspect its structure.

[
    {
        "type": "file",
        "name": "account.csv",
        "path": "/test-connectors/testFolder-fileIngestion/account.csv",
        "canPreview": true,
        "canFetchSchema": true
    },
    {
        "type": "file",
        "name": "profileData.json",
        "path": "/test-connectors/testFolder-fileIngestion/profileData.json",
        "canPreview": true,
        "canFetchSchema": true
    },
    {
        "type": "file",
        "name": "sampleprofile--3.parquet",
        "path": "/test-connectors/testFolder-fileIngestion/sampleprofile--3.parquet",
        "canPreview": true,
        "canFetchSchema": true
    }
]

Inspect the structure of a file

To inspect the structure of data file from your cloud storage, perform a GET request while providing the file’s path and type as a query parameter.

You can inspect the structure of a data file from your cloud storage source by performing a GET request while providing the file’s path and type. You can also inspect different file types such as CSV, TSV, or compressed JSON and delimited files by specifying their file types as part of the query parameters.

API format

GET /connections/{BASE_CONNECTION_ID}/explore?objectType=file&object={FILE_PATH}&fileType={FILE_TYPE}&{QUERY_PARAMS}&preview=true
GET /connections/{BASE_CONNECTION_ID}/explore?objectType=file&object={FILE_PATH}&preview=true&fileType=delimited&columnDelimiter=\t
GET /connections/{BASE_CONNECTION_ID}/explore?objectType=file&object={FILE_PATH}&preview=true&fileType=delimited&compressionType=gzip;
GET /connections/{BASE_CONNECTION_ID}/explore?objectType=FILE&object={FILE_PATH}&preview=true&fileType=delimited&encoding=ISO-8859-1;
ParameterDescription
{BASE_CONNECTION_ID}The connection ID of your cloud storage source connector.
{FILE_PATH}The path to the file you want to inspect.
{FILE_TYPE}

The type of the file. Supported file types include:

  • DELIMITED: Delimiter-separated value. DSV files must be comma-separated.
  • JSON: JavaScript Object Notation. JSON files must be XDM compliant
  • PARQUET: Apache Parquet. Parquet files must be XDM compliant.
{QUERY_PARAMS}Optional query parameters that can be used to filter results. See the section on query parameters for more information.

Request

curl -X GET \
    'http://platform.adobe.io/data/foundation/flowservice/connections/{BASE_CONNECTION_ID}/explore?objectType=file&object=/aep-bootcamp/Adobe%20Pets%20Customer%2020190801%20EXP.json&fileType=json&preview=true' \
    -H 'Authorization: Bearer {ACCESS_TOKEN}' \
    -H 'x-api-key: {API_KEY}' \
    -H 'x-gw-ims-org-id: {ORG_ID}' \
    -H 'x-sandbox-name: {SANDBOX_NAME}'

Response

A successful response returns the structure of the queried file including table names and data types.

[
    {
        "name": "Id",
        "type": "String"
    },
    {
        "name": "FirstName",
        "type": "String"
    },
    {
        "name": "LastName",
        "type": "String"
    },
    {
        "name": "Email",
        "type": "String"
    },
    {
        "name": "Phone",
        "type": "String"
    }
]