Google Cloud Storage connection

Overview overview

Create a live outbound connection to Google Cloud Storage to periodically export data files from Adobe Experience Platform into your own buckets.

Connect to your Google Cloud Storage storage through API or UI connect-api-or-ui

Supported audiences supported-audiences

This section describes which types of audiences you can export to this destination.

Audience origin
Supported
Description
Segmentation Service
Audiences generated through the Experience Platform Segmentation Service.
Custom uploads
Audiences imported into Experience Platform from CSV files.

Export type and frequency export-type-frequency

Refer to the table below for information about the destination export type and frequency.

Item
Type
Notes
Export type
Profile-based
You are exporting all members of a segment, together with the applicable schema fields, as chosen in the select profile attributes screen of the destination activation workflow.
Export frequency
Batch
Batch destinations export files to downstream platforms in increments of three, six, eight, twelve, or twenty-four hours. Read more about batch file-based destinations.

Export datasets export-datasets

This destination supports dataset exports. For complete information on how to set up dataset exports, read the tutorials:

File format of the exported data file-format

When exporting audience data, Platform creates a .csv, parquet, or .json file in the storage location that you provided. For more information about the files, see the supported file formats for export section in the audience activation tutorial.

When exporting datasets, Platform creates a .parquet or .json file in the storage location that you provided. For more information about the files, see the verify successful dataset export section in the export datasets tutorial.

Prerequisite setup for connecting your Google Cloud Storage account prerequisites

In order to connect Platform to Google Cloud Storage, you must first enable interoperability for your Google Cloud Storage account. To access the interoperability setting, open Google Cloud Platform and select Settings from the Cloud Storage option in the navigation panel.

Google Cloud Platform dashboard with Cloud Storage and Settings highlighted.

The Settings page appears. From here, you can see information regarding your Google project ID and details about your Google Cloud Storage account. To access interoperability settings, select Interoperability from the top header.

The Interoperability tab highlighted in the Google Cloud Platform dashboard.

The Interoperability page contains information on authentication, access keys, and the default project associated with your service account. To generate a new access key ID and a secret access key for your service account, select Create a Key for a Service Account.

The Create a key for a service account control highlighted in the Google Cloud Platform dashboard.

You can use your newly generated access key ID and secret access key to connect your Google Cloud Storage account to Platform.

Connect to the destination connect

IMPORTANT
To connect to the destination, you need the View Destinations and Manage Destinations access control permissions. Read the access control overview or contact your product administrator to obtain the required permissions.

To connect to this destination, follow the steps described in the destination configuration tutorial. In the destination configuration workflow, fill in the fields listed in the two sections below.

Authenticate to destination authenticate

To authenticate to the destination, fill in the required fields and select Connect to destination.

  • Access key ID: A 61-character, alphanumeric string used to authenticate your Google Cloud Storage account to Platform. For information on how to obtain this value, read the prerequisites section above.

  • Secret access key: A 40-character, base64-encoded string used to authenticate your Google Cloud Storage account to Platform. For information on how to obtain this value, read the prerequisites section above.

  • Encryption key: Optionally, you can attach your RSA-formatted public key to add encryption to your exported files. View an example of a correctly formatted encryption key in the image below.

    Image showing an example of a correctly formatted PGP key in the UI

For more information about these values, read the Google Cloud Storage HMAC keys guide. For steps on how to generate your own access key ID and secret access key, refer to the Google Cloud Storage source overview.

Fill in destination details destination-details

To configure details for the destination, fill in the required and optional fields below. An asterisk next to a field in the UI indicates that the field is required.

  • Name: Fill in the preferred name for this destination.

  • Description: Optional. For example, you can mention which campaign you are using this destination for.

  • Bucket name: Enter the name of the Google Cloud Storage bucket to be used by this destination.

  • Folder path: Enter the path to the destination folder that will host the exported files.

  • File type: Select the format Experience Platform should use for the exported files. When selecting the CSV option, you can also configure the file formatting options.

  • Compression format: Select the compression type that Experience Platform should use for the exported files.

  • Include manifest file: Toggle this option on if you’d like the exports to include a manifest JSON file that contains information about the export location, export size, and more. The manifest is named using the format manifest-<<destinationId>>-<<dataflowRunId>>.json. View a sample manifest file. The manifest file includes the following fields:

    • flowRunId: The dataflow run which generated the exported file.
    • scheduledTime: The time in UTC when the file was exported.
    • exportResults.sinkPath: The path in your storage location where the exported file is deposited.
    • exportResults.name: The name of the exported file.
    • size: The size of the exported file, in bytes.

Enable alerts enable-alerts

You can enable alerts to receive notifications on the status of the dataflow to your destination. Select an alert from the list to subscribe to receive notifications on the status of your dataflow. For more information on alerts, see the guide on subscribing to destinations alerts using the UI.

When you are finished providing details for your destination connection, select Next.

Required Google Cloud Storage permissions required-google-cloud-storage-permission

To successfully connect and export data to your Google Cloud Storage storage location, you need the following Google Cloud Storage permissions for your buckets:

  • orgpolicy.policy.get
  • resourcemanager.projects.get
  • resourcemanager.projects.list
  • storage.managedFolders.create
  • storage.multipartUploads.abort
  • storage.multipartUploads.create
  • storage.multipartUploads.listParts
  • storage.objects.create
  • storage.objects.list

Read more about access control and permissions in Google Cloud Storage.

Activate audiences to this destination activate

IMPORTANT

See Activate audience data to batch profile export destinations for instructions on activating audiences to this destination.

Scheduling

In the Scheduling step, you can set up the export schedule for your Google Cloud Storage destination and you can also configure the name of your exported files.

Map attributes and identities map

In the Mapping step, you can select which attribute and identity fields to export for your profiles. You can also select to change the headers in the exported file to any friendly name that you wish. For more information, view the mapping step in the activate batch destinations UI tutorial.

Validate successful data export exported-data

To verify if data has been exported successfully, check your Google Cloud Storage bucket and make sure that the exported files contain the expected profile populations.

IP address allowlist ip-address-allow-list

Refer to the IP address allowlist article if you need to add Adobe IPs to an allowlist.

recommendation-more-help
7f4d1967-bf93-4dba-9789-bb6b505339d6