The Google PubSub source is available in the sources catalog to users who have purchased Real-Time Customer Data Platform Ultimate.
This tutorial provides steps for creating a Google PubSub (hereinafter referred to as “PubSub”) using the Platform user interface.
This tutorial requires a working understanding of the following components of Adobe Experience Platform:
If you already have a valid PubSub connection, you may skip the remainder of this document and proceed to the tutorial on configuring a dataflow.
In order to connect PubSub to Platform, you must provide a valid value for the following credentials:
|Project ID||The project ID required to authenticate PubSub.|
|Credentials||The credential required to authenticate PubSub. You must ensure that you put the complete JSON file after removing the white spaces from your credentials.|
|Topic name||The name of your PubSub subscription. In PubSub, subscriptions allow you to receive messages, by subscribing to the topic in which messages have been published to. Note: A single PubSub subscription can only be used for one dataflow. In order to make multiple dataflows, you must have multiple subscriptions.|
|Subscription name||The name of your PubSub subscription. In PubSub, subscriptions allow you to receive messages, by subscribing to the topic in which messages have been published to.|
For more information about these values, see the following PubSub authentication document. If you are using service account-based authentication, see the following PubSub guide for steps on how to generate your credentials.
If you are using service account-based authentication, ensure that you have granted sufficient user access to your service account and that there are no extra white spaces in the JSON, when copying and pasting your credentials.
Once you have gathered your required credentials, you can follow the steps below to link your PubSub account to Platform.
In the Platform UI, select Sources from the left navigation to access the Sources workspace. The Catalog screen displays a variety of sources you can create an account with.
You can select the appropriate category from the catalog on the left-hand side of your screen. Alternatively, you can find the specific source you wish to work with using the search option.
Under the Cloud storage category, select Google PubSub, and then select Add data.
The Connect to Google PubSub page appears. On this page, you can either use new credentials or existing credentials.
To use an existing account, select the PubSub account you want to create a new dataflow with, then select Next to proceed.
When creating an account with restricted access, you must provide at least one of your topic name or subscription name. Authentication will fail if both values are missing.
If you are creating a new account, select New account, and then provide a name and an optional description for your new PubSub account.
The PubSub source allows you to specify the type of access that you want to allow during authentication. You can set up your account to have either project-based authentication or topic and subscription-based authentication. Project-based authentication allows you to grant access to the root-level project in your account, while topic and subscription-based authentication allows you to restrict access to a particular PubSub topic and subscription.
To create an account with access to your root PubSub project folder. Select Google PubSub authentication credentials as your authentication type and provide your project ID and credentials. When finished, select Connect to source and then allow some time for the new connection to establish.
To create an account with restricted access only to a particular PubSub topic and subscription, select Google PubSub Scoped authentication credentials and then provide your credentials, topic name, and/or subscription name. When finished, select Connect to source and then allow some time for the new connection to establish.
Principal (roles) assigned to a PubSub project are inherited in all of the topics and subscriptions created inside a PubSub project. If you want a principal (role) to have access to a specific topic, then that principal (role) must also be added to the topic’s corresponding subscription as well. For more information, read the PubSub documentation on access control.
A successful authentication brings you to the Select data step, where you can navigate through your PubSub data hierarchy and select the data that you want to bring to Experience Platform.
If you have authenticated with project-based access, the Select data interface will display all subscriptions within your project that has a topic attached to them.
If you have authenticated with a topic and subscription-based access, the Select data interface display can vary depending on the information that you provided.
By following this tutorial, you have created a connection between your PubSub account and Platform. You can now continue on to the next tutorial and configure a dataflow to bring streaming data from your cloud storage into Platform.