Configure source connectors

In this video, we’ll give you a quick overview of source connectors in Adobe Journey Optimizer. Ingesting your customer data into Journey Optimizer is a critical step in building the real-time customer profile that informs and empowers the personalized experiences that you’ll orchestrate for your customers in Journey Optimizer. And source connectors, give you a set of pre-built tools for ingesting data from a wide variety of different systems of origin. From the Journey Optimizer home screen, you’ll see sources in the left navigation. Clicking on sources will take you to the source catalog screen, where you can see all of the source connectors that are currently available. Know that there are source connectors for Adobe applications, CRM solutions, cloud storage providers, and more. Source connectors are categorized based on their source type. The list of available source connectors is constantly growing and evolving as new sources are added to the catalog. You can find an available source either by browsing through the categories below or using the search bar. By default, you’ll see the full catalog of available source connectors, but you have the option to switch the view to show only the sources you’ve set up in your environment. Each source has its specific configuration details, but the general process will be similar. So let’s set up a source connected to Amazon S3 cloud storage as a representative example.

When you find the source connector you want to set up, you’ll see a button to either set up or add data. That’s because there are two parts to setting up a source. First, you configure the connection to your account in that system, and then set up a specific flow of data into a dataset in Adobe. For example, once I’ve set up the connection to my S3 account, I would be able to add additional dataflows without having to configure the connection again. Since we’re setting up the S3 source for the first time, we’ll click Setup and create a new account and here we provide the account authentication information and then click on Connect to source to test the connection. If the connection is successful, click Next to proceed to data selection.

In this step, we choose the source file for data ingestion and verify the file data format.

Once you’ve selected the desired file, you can then preview data from that file.

Next we’ll specify a target dataset for the incoming data. You can choose an existing dataset or create a new dataset. We’ll choose the new dataset option and provide a dataset name and description and select the schema that the dataset should be based on. If you want this data to be part of the real-time customer profile and available for use in Journey Optimizer which we do, make sure to select Profile enabled. After selecting a schema for this dataset, you’ll see a proposed mapping between the source file field and the target schema field based on the names and data types of the fields in the source file. You’ll need to review the automatic mapping and make any adjustments needed so that the fields from the source file are properly mapped to the target schema. You can add a calculated field if you need to manipulate some of the values in the source file to fit the target schema. For example, we could combine the first name field, and the last name field into a calculated field using the concatenation function before ingesting the data into a dataset field.

Once the mapping looks good, click Next.

Scheduling lets you choose the frequency at which data should flow from the source into a dataset. For the sake of this demo, we’ll set it to ingest the data every 15 minutes and set a start time for the dataflow. Select backfill to ingest existing files that are already in the specified path. Otherwise only new files loaded after the start time of the data flow will be ingested.

Next, we’ll give our dataflow a name and you can choose if you want to enable partial ingestion. This allows ingestion to proceed even if there are some errors in the data. The error of threshold specifies the percentage of acceptable errors before the entire batch fails and is set to 5% by default.

Finally, we can review the source configuration details and save our new data flow.

Once our data flow has had time to run, we see a successful run status. We can click into the target dataset for more details about the batch of data that was ingested.

Finally, going back to Sources, in addition to the catalog of available source connectors, you can also click on Dataflows to view and manage the dataflows you’ve already set up or on Accounts to view and manage the connections to your accounts in various sources. Finally, you can click on a system view to get an overall picture of how each of the sources you’ve set up contributes to the set of profiles you have in the system. In summary, Adobe Journey Optimizer allows you to integrate data from external sources into a real-time customer profile. And source connectors provide you a set of pre-built connectors for configuring the flow of data from a variety of sources, such as Adobe applications, cloud-based storage, databases and many others. We showed you a specific example for Amazon S3 and we encourage you to check out the documentation and tutorials for the Adobe Experience Platform. For more details on setting up this and other source connectors. -