DocumentationExperience PlatformSource Connectors Guide

[Ultimate]{class="badge positive"}

Create a Snowflake base connection using the Flow Service API

Last update: Mon Jan 29 2024 00:00:00 GMT+0000 (Coordinated Universal Time)

CREATED FOR:

  • Developer
IMPORTANT
The Snowflake source is available in the sources catalog to users who have purchased Real-Time Customer Data Platform Ultimate.

A base connection represents the authenticated connection between a source and Adobe Experience Platform.

Use the following tutorial to learn how to create a base connection for Snowflake using the Flow Service API.

Getting started

This guide requires a working understanding of the following components of Adobe Experience Platform:

  • Sources: Experience Platform allows data to be ingested from various sources while providing you with the ability to structure, label, and enhance incoming data using Platform services.
  • Sandboxes: Experience Platform provides virtual sandboxes which partition a single Platform instance into separate virtual environments to help develop and evolve digital experience applications.

Using Platform APIs

For information on how to successfully make calls to Platform APIs, see the guide on getting started with Platform APIs.

The following section provides additional information that you will need to know in order to successfully connect to Snowflake using the Flow Service API.

Gather required credentials

You must provide values for the following credential properties to authenticate your Snowflake source.

Account key authentication
table 0-row-2 1-row-2 2-row-2 3-row-2 4-row-2 5-row-2 6-row-2 7-row-2
Credential Description
account An account name uniquely identifies an account within your organization. In this case, you must uniquely identify an account across different Snowflake organizations. To do this, you must prepend your organization name to the account name. For example: orgname-account_name. For more information on account names, read the Snowflake documentation on account identifiers.
warehouse The Snowflake warehouse manages the query execution process for the application. Each Snowflake warehouse is independent from one another and must be accessed individually when bringing data over to Platform.
database The Snowflake database contains the data you want to bring the Platform.
username The username for the Snowflake account.
password The password for the Snowflake user account.
role The default access control role to use in the Snowflake session. The role should be an existing one that has already been assigned to the specified user. The default role is PUBLIC.
connectionString The connection string used to connect to your Snowflake instance. The connection string pattern for Snowflake is jdbc:snowflake://{ACCOUNT_NAME}.snowflakecomputing.com/?user={USERNAME}&password={PASSWORD}&db={DATABASE}&warehouse={WAREHOUSE}
Key-pair authentication

To use key-pair authentication, you must generate a 2048-bit RSA key pair and then provide the following values when creating an account for your Snowflake source.

table 0-row-2 1-row-2 2-row-2 3-row-2 4-row-2 5-row-2 6-row-2
Credential Description
account An account name uniquely identifies an account within your organization. In this case, you must uniquely identify an account across different Snowflake organizations. To do this, you must prepend your organization name to the account name. For example: orgname-account_name. For more information on account names, read the Snowflake documentation on account identifiers.
username The username of your Snowflake account.
privateKey The Base64-encoded private key of your Snowflake account. You can generate either encrypted or unencrypted private keys. If you are using an encrypted private key, then you must also provide a private key passphrase when authenticating against Experience Platform.
privateKeyPassphrase The private key passphrase is an additional layer of security that you must use when authenticating with an encrypted private key. You are not required to provide the passphrase if you are using an unencrypted private key.
database The Snowflake database that contains the data you want to ingest to Experience Platform.
warehouse The Snowflake warehouse manages the query execution process for the application. Each Snowflake warehouse is independent from one another and must be accessed individually when bringing data over to Experience Platform.

For more information about these values, refer the Snowflake key-pair authentication guide.

NOTE
You must set the PREVENT_UNLOAD_TO_INLINE_URL flag to FALSE to allow data unloading from your Snowflake database to Experience Platform.

Create a base connection

A base connection retains information between your source and Platform, including your source’s authentication credentials, the current state of the connection, and your unique base connection ID. The base connection ID allows you to explore and navigate files from within your source and identify the specific items that you want to ingest, including information regarding their data types and formats.

To create a base connection ID, make a POST request to the /connections endpoint while providing your Snowflake authentication credentials as part of the request body.

API format

POST /connections
ConnectionString
accordion
Request

The following request creates a base connection for Snowflake:

code language-shell 
curl -X POST \
  'https://platform.adobe.io/data/foundation/flowservice/connections' \
  -H 'Authorization: Bearer {ACCESS_TOKEN}' \
  -H 'x-api-key: {API_KEY}' \
  -H 'x-gw-ims-org-id: {ORG_ID}' \
  -H 'x-sandbox-name: {SANDBOX_NAME}' \
  -H 'Content-Type: application/json' \
  -d '{
      "name": "Snowflake base connection",
      "description": "Snowflake base connection",
      "auth": {
          "specName": "ConnectionString",
          "params": {
              "connectionString": "jdbc:snowflake://{ACCOUNT_NAME}.snowflakecomputing.com/?user={USERNAME}&password={PASSWORD}&db={DATABASE}&warehouse={WAREHOUSE}"
          }
      },
      "connectionSpec": {
          "id": "b2e08744-4f1a-40ce-af30-7abac3e23cf3",
          "version": "1.0"
      }
  }'
table 0-row-2 1-row-2 2-row-2
Property Description
auth.params.connectionString The connection string used to connect to your Snowflake instance. The connection string pattern for Snowflake is jdbc:snowflake://{ACCOUNT_NAME}.snowflakecomputing.com/?user={USERNAME}&password={PASSWORD}&db={DATABASE}&warehouse={WAREHOUSE}.
connectionSpec.id The Snowflake connection specification ID: b2e08744-4f1a-40ce-af30-7abac3e23cf3.
accordion
Response

A successful response returns the newly created connection, including its unique connection identifier (id). This ID is required to explore your data in the next tutorial.

code language-json 
{
    "id": "2fce94c1-9a93-4971-8e94-c19a93097129",
    "etag": "\"d403848a-0000-0200-0000-5e978f7b0000\""
}
Key-pair authentication with encrypted private key
accordion
Request
code language-shell 
curl -X POST \
  'https://platform.adobe.io/data/foundation/flowservice/connections' \
  -H 'Authorization: Bearer {ACCESS_TOKEN}' \
  -H 'x-api-key: {API_KEY}' \
  -H 'x-gw-ims-org-id: {ORG_ID}' \
  -H 'x-sandbox-name: {SANDBOX_NAME}' \
  -H 'Content-Type: application/json' \
  -d '{
      "name": "Snowflake base connection with encrypted private key",
      "description": "Snowflake base connection with encrypted private key",
      "auth": {
        "specName": "KeyPair Authentication",
        "params": {
            "account": "acme-snowflake123",
            "username": "acme-cj123",
            "database": "ACME_DB",
            "privateKey": "{BASE_64_ENCODED_PRIVATE_KEY}",
            "privateKeyPassphrase": "abcd1234",
            "warehouse": "COMPUTE_WH"
        }
    },
    "connectionSpec": {
        "id": "b2e08744-4f1a-40ce-af30-7abac3e23cf3",
        "version": "1.0"
    }
  }'
table 0-row-2 1-row-2 2-row-2 3-row-2 4-row-2 5-row-2 6-row-2 7-row-2
Property Description
auth.params.account The name of your Snowflake account.
auth.params.username The username associated with your Snowflake account.
auth.params.database The Snowflake database from where the data will be pulled from.
auth.params.privateKey The Base64-encoded encrypted private key of your Snowflake account.
auth.params.privateKeyPassphrase The passphrase that corresponds with your private key.
auth.params.warehouse The Snowflake warehouse that you are using.
connectionSpec.id The Snowflake connection specification ID: b2e08744-4f1a-40ce-af30-7abac3e23cf3.
accordion
Response

A successful response returns the newly created connection, including its unique connection identifier (id). This ID is required to explore your data in the next tutorial.

code language-json 
{
    "id": "2fce94c1-9a93-4971-8e94-c19a93097129",
    "etag": "\"d403848a-0000-0200-0000-5e978f7b0000\""
}
Key-pair authentication with unencrypted private key
accordion
Request
code language-shell 
curl -X POST \
  'https://platform.adobe.io/data/foundation/flowservice/connections' \
  -H 'Authorization: Bearer {ACCESS_TOKEN}' \
  -H 'x-api-key: {API_KEY}' \
  -H 'x-gw-ims-org-id: {ORG_ID}' \
  -H 'x-sandbox-name: {SANDBOX_NAME}' \
  -H 'Content-Type: application/json' \
  -d '{
      "name": "Snowflake base connection with encrypted private key",
      "description": "Snowflake base connection with encrypted private key",
      "auth": {
        "specName": "KeyPair Authentication",
        "params": {
            "account": "acme-snowflake123",
            "username": "acme-cj123",
            "database": "ACME_DB",
            "privateKey": "{BASE_64_ENCODED_PRIVATE_KEY}",
            "warehouse": "COMPUTE_WH"
        }
    },
    "connectionSpec": {
        "id": "b2e08744-4f1a-40ce-af30-7abac3e23cf3",
        "version": "1.0"
    }
  }'
table 0-row-2 1-row-2 2-row-2 3-row-2 4-row-2 5-row-2 6-row-2
Property Description
auth.params.account The name of your Snowflake account.
auth.params.username The username associated with your Snowflake account.
auth.params.database The Snowflake database from where the data will be pulled from.
auth.params.privateKey The Base64-encoded unencrypted private key of your Snowflake account.
auth.params.warehouse The Snowflake warehouse that you are using.
connectionSpec.id The Snowflake connection specification ID: b2e08744-4f1a-40ce-af30-7abac3e23cf3.
accordion
Response

A successful response returns the newly created connection, including its unique connection identifier (id). This ID is required to explore your data in the next tutorial.

code language-json 
{
    "id": "2fce94c1-9a93-4971-8e94-c19a93097129",
    "etag": "\"d403848a-0000-0200-0000-5e978f7b0000\""
}

By following this tutorial, you have created a Snowflake base connection using the Flow Service API. You can use this base connection ID in the following tutorials:

recommendation-more-help
337b99bb-92fb-42ae-b6b7-c7042161d089