[Ultimate]{class="badge positive"}

Google PubSub source

Adobe Experience Platform provides native connectivity for cloud providers like AWS, Google Cloud Platform, and Azure, allowing you to bring data from these systems into Platform for use in downstream services and destinations.

Cloud storage sources can bring your data into Platform without the need to download, format, or upload. Ingested data can be formatted as XDM JSON, XDM Parquet, or delimited. Every step of the process is integrated into the sources workflow. Platform allows you to bring in data from Google PubSub in real time.

Prerequisites prerequisites

This section outlines prerequisite set up that you must complete before connecting your Google PubSub account to Experience Platform.

Create service account create-service-account

A service account is a type of account often used by an application or compute workload, rather than a person. A service account is identified by its email address, which is unique to the account.

For more information on determining the right authentication type for your use case, read the Google guide on authentication methods.

Follow the steps outlined below to create a service account:

First, navigate to the IAM page of the Google Developer Console and then select Create Service Account.

Next, enter a a display name and an ID for your service account, and then select Create and Continue.

Generate service account keys generate-service-account-keys

To generate keys for your service account, select the keys header in the service accounts page. From there, select Add key and then select Create new key from the dropdown menu. You can also use this panel to upload an existing key.

When successful, you will receive a message indicating that the private key has been saved your computer and that a file will be downloaded. You can then use the content of this file as credentials, when creating your Google PubSub account on Experience Platform.

Grant permissions at topic and subscription level grant-permissions

To grant permissions at the topic and subscription level, navigate to the topic console page and then select Show info panel. Next, under the Permissions tab, select Add Principal and then add the service account principal along with the permissions.

Configurations for optimal Google PubSub usage optimal-configurations

This section outlines configurations you are recommended to make to optimize your usage of the Google PubSub source on Experience Platform.

Subscription properties subscription-properties

Use the Google Developer Console to increase your acknowledgement deadline. This allows the Google Publisher to wait according to the time that you configure before sending the message again. This delay helps in the reduction of unnecessary load at the subscriber level.

Enable exactly one delivery. This configuration informs the Google Publisher to guarantee that messages sent to the subscription do not get resent before the acknowledgement deadline expires. You can use this setting to ensure that acknowledgement messages do not get resent to the subscription.

You can enable Retry after exponential backoff delay to reduce the risk of further overwhelming the server. You can enable this configuration in the Google Developer Console to better mitigate transient failures (temporary errors that typically resolve themselves), by providing the system with more time to recover before attempting another connection.

You must set your subscription message retention duration to be 24 hours or more to ensure that unacknowledged data does not get lost during peak loads. Additionally, enable a dead letter topic to ensure that data loss does not happen even during rare edge cases.

Connect Google PubSub to Experience Platform

The documentation below provides information on how to connect Google PubSub to Platform using APIs or the user interface:

Using APIs

Using the UI