Learn how to easily ingest data from Marketo Engage into Real-Time Customer Data Platform, B2B Edition, and Adobe Experience Platform using the source connector using two workflows.
Learn how to configure the source connector for Marketo using the template workflow. This workflow auto-generates assets needed for ingesting Marketo data based on templates. It saves you upfront time, and the assets can be customized according to your needs.
Learn how to configure the source connector for Marketo using the standard workflow. The standard workflow requires upfront creation of schemas and identity namespaces.
Hi, I’m Margaret Trinh, senior product manager, here today to share with you more about the source data connector for Marketo Engage.
In this video, we’ll discuss how to set up the Marketo Connector, how to work with custom attributes and custom objects, and to understand the data flow between Marketo and Experience Platform. Before watching this video, we will assume that you’ve already created the B2B namespaces and schemas in your instance. And if you’re using any custom attributes, the schemas have been modified to include those. First, setting up the Marketo Connector. We will go through these steps in detail once we reach the demo. RT CDP customers will have the Marketo Connector visible in their Sources catalog. Before the setup, we’ll need to have a user setup within Marketo, dedicated to the data ingestion. There will also be a required mapping between your IMS organization and your Marketo subscriptions. Do keep in mind that you are allowed to map multiple Marketo subscriptions to a single IMS organization.
Once these are configured, you can begin creating your flows for data ingestion.
The Marketo Connector utilizes Adobe’s business XDMs to map data from Marketo into Adobe Experience Platform. In addition, the connector will present users with a pre-mapped template so that standard attributes from Marketo are automatically mapped to the target schema. Please make sure to review these mappings and ensure that custom attributes are also mapped.
To add custom attributes from Marketo, you can leverage custom field groups and add them to your schema.
After you’ve configured your data flow from Marketo, we will initiate a bootstrap job to begin ingesting all historical data from Marketo into Experience Platform. The time will vary depending on the size of your subscription. Once the historical backfill is complete, we will switch over to ingest data via streaming. That will put incremental records in near real time. Because of this, users will not see a scheduling step when setting up the data flow.
If you have a CRM synced to your Marketo subscriptions, such as Salesforce or Microsoft Dynamics, the Postman utility will also create namespaces and schemas for these platforms. These namespaces will help distinguish the identities coming from each platform so that the Unified Identity Service can stitch and merge similar records together.
Now that you have a solid understanding, let’s see all of this in action.
Before we dive in to setting up the Marketo Engage source connector, it’s assumed that you’ve already utilized automated utility to create the B2B namespaces and schemas. In addition, if you’re ingesting any custom attributes that you have not yet defined in the classes or field groups, you’ve also built your schema to include those custom attributes. Now, to get started on setting up the connector, there are a couple of steps that will be handled within Marketo Engage. But before we head over there, let’s first take note of our organization ID, which we’ll use to map to our Marketo subscription. You can find your IMS organization ID by going to the Experience Cloud homepage and scrolling to the very bottom. At the bottom right, you’ll find your organization ID listed here. You can take note of this. Now, over to Marketo Engage. As an admin, you’ll first navigate over to the admin page and click on the Users and Roles page.
Let’s first start off by creating a new role for an API user.
This user will access the Marketo API to pull data from Marketo into Adobe Experience Platform. So let’s give it a role name with the description of, and give it the permission to access API. And click Create. Now, to create the user, we’ll navigate to the Users tab and invite a new user. Note that there will not be an actual email sent to this user so it does not need to be a real email address. For this dedicated user, we’ll set the email to…
After I click Next, I’ll select the AEP API User Role that I just created from the previous step.
And be sure that this is an API user only by checking the check box.
Once I click Next, I can go ahead and hit Send to save the new user.
Lastly, I’m going to create a custom service that will provide the authentication token to speak between the two services. From the Launch Point tab, create a new service.
First, we’ll select a name for the service.
And this is going to be custom.
Then we’ll add a brief but useful description.
Then select the user that we just created from the earlier step.
Go ahead and click Create. And to show you how to retrieve the client ID and client secret that you’ll need for the next step, you’ll need to open the new service that you just created by clicking View Details to get those values. Keep this open so that you can copy and paste it into AEP shortly.
Now, the last step in Marketo is to use the IMS organization ID that we copied from Adobe Experience Cloud earlier. Let’s do that again. From the homepage of Experience Cloud, scroll to the very bottom. At the bottom right, you’ll find the org ID located here. Take everything except for the @AdobeOrg suffix at the end.
Please be aware that you’ll need to be an admin for your Adobe organization in order to complete the mapping. Now, go to Adobe Organization Mapping and click Edit. Enter your org ID.
Click OK, then Confirm. And you may be prompted to log back in to Adobe Experience Cloud to finalize the mapping.
Now let’s go back to the custom service that you created so that we can grab the client ID and client secret.
I’ll first grab the client ID.
From here, let’s go back and launch Experience Platform, then head to the Sources menu. Along with other Adobe applications, you’ll see the Marketo Engage connector located here. If you haven’t authenticated before, you’ll select the configure button. Otherwise, you may see the text, “Add data.” In which case, you can select an account that’s already been connected.
If you haven’t authenticated, you’ll land on the option to set up a new account. We’ll enter some details for our new connection. And from of the custom service that we just created, we’ll grab the client ID and client secret, and paste it here.
To find your Munchkin ID, navigate back to Marketo to the My Account tab.
Copy the Munchkin ID from the bottom.
I can go ahead and click Connect to source to move on to the next step.
You’ll now be presented with the 10 data sets that can be imported from Marketo Engage. For this demo, we’ll ingest accounts, or rather, companies from the Marketo API. By clicking the data set, you should expect to see a preview of the data on the right-hand side.
After you click Next, you’ll set up a name related to accounts for your data set.
Then go ahead and select the schema related to accounts, once again, that was auto-generated from the utility.
If you create a custom schema to include custom field groups and attributes, make sure that you select that one instead. After selecting the schema, the mapping from the Marketo source data set and the AEP target dataset will appear. For most standard attributes, Marketo has included predefined mappings that can be modified to your preference. For any custom attributes, you’ll want to make sure that you revisit those to make sure that the mappings are correct. Lastly, make sure to enable the dataset for profile. Then click Next.
The final step here is to name a data flow to make it identifiable to reference later on. In this case, it’s already been auto-populated to include the name of the entity. We would also suggest keeping error diagnostics enabled so you can find any issues with data ingestion. Also, enable the partial data ingestion. Keeping the default threshold at 5%, meaning that batches with less than 5% of errors will be considered successful. Although, you’ll still be able to retrieve the air logs. Now after you click Next, you’ll be presented with a quick review of your setup for the specific data set. And the ingestion will begin once you click Save & ingest. You’ll notice that unlike other connectors, there’s no scheduling step. That’s because Marketo’s data ingestion automatically runs multiple batches to ingest all of your historical data. And once that’s complete, Marketo will automatically switch over to stream any new records or updated records by scanning for anything that’s been modified. You’ll want to make sure that you repeat these steps for all additional datasets that you’ll need for your B2B use cases. And that wraps it up for knowing how to use the Marketo Engage source data connector.
So in summary, within this video, we have successfully set up the data flows using the Marketo Source Connector, understood how to handle custom attributes, and learned how data ingestion works between Marketo and Experience Platform. Thank you for watching. -
For more information, please see the following documentation: