Before creating a Marketo Engage source connection and a dataflow, you must first ensure that you have mapped your Adobe Organization ID in Marketo. Furthermore, you must also ensure that you have completed auto-populating your Marketo B2B namespaces and schemas prior to creating a source connection and a dataflow.
This tutorial provides steps for creating a Marketo Engage (hereinafter referred to as “Marketo”) source connector in the UI to bring B2B data into Adobe Experience Platform.
This tutorial requires a working understanding of the following components of Adobe Experience Platform:
In order to access your Marketo account on Platform, you must provide the following values:
||The Munchkin ID is the unique identifier for a specific Marketo instance.|
||The unique client ID of your Marketo instance.|
||The unique client secret of your Marketo instance.|
For more information on acquiring these values, refer to the Marketo authentication guide.
Once you have gathered your required credentials, you can follow the steps in the next section.
In the Platform UI, select Sources from the left navigation bar to access the Sources workspace. The Catalog screen displays a variety of sources with which you can create an account.
You can select the appropriate category from the catalog on the left-hand side of your screen. Alternatively, you can find the specific source you wish to work with using the search bar.
Under the Adobe applications category, select Marketo Engage. Then, select Add data to create a new Marketo dataflow.
The Connect Marketo Engage account page appears. On this page, you can either use a new account or access an existing account.
To create a dataflow with an existing account, select Existing account and then select the Marketo account you want to use. Select Next to proceed.
If you are creating a new account, select New account. On the input form that appears, provide an account name, an optional description, and your Marketo authentication credentials. When finished, select Connect to source and then allow some time for the new connection to establish.
After creating your Marketo account, the next step provides an interface for you to explore Marketo datasets.
The left half of the interface is a directory browser, displaying the 10 Marketo datasets. A fully-functioning Marketo source connection requires the ingestion of the nine different datasets. If you are also using the Marketo account-based marketing (ABM) feature, then you must also create a 10th dataflow to ingest the Named Accounts dataset.
For the purposes of brevity, the following tutorial uses Opportunities as an example, but the steps outlined below apply to any of the 10 Marketo datasets.
Select the dataset you wish to ingest first, then select Next.
The Dataflow detail page allows you to select whether you want to use an existing dataset or a new dataset. During this process, you can also configure settings for Profile dataset, Error diagnostics, Partial ingestion, and Alerts.
To ingest data into an existing dataset, select Existing dataset. You can either retrieve an existing dataset using the Advanced search option or by scrolling through the list of existing datasets in the dropdown menu. Once you have selected a dataset, provide a name and a description for your dataflow.
To ingest into a new dataset, select New dataset and then provide an output dataset name and an optional description. Next, select a schema to map to using the Advanced search option or by scrolling through the list of existing schemas in the dropdown menu. Once you have selected a schema, provide a name and a description for your dataflow.
Next, select the Profile dataset toggle to enable your dataset for Profile. This allows you to create a holistic view of an entity’s attributes and behaviors. Data from all Profile-enabled datasets will be included in Profile and changes are applied when you save your dataflow.
Error diagnostics enables detailed error message generation for any erroneous records that occur in your dataflow, while Partial ingestion allows you to ingest data containing errors, up to a certain threshold that you manually define. See the partial batch ingestion overview for more information.
The Marketo source uses batch ingestion to ingest all historical records and uses streaming ingestion for real-time updates. This allows the source to continue streaming while ingesting any erroneous records. Enable the Partial ingestion toggle and then set the Error threshold % to maximum to prevent the dataflow from failing.
You can enable alerts to receive notifications on the status of your dataflow. Select an alert from the list to subscribe to receive notifications on the status of your dataflow. For more information on alerts, see the guide on subscribing to sources alerts using the UI.
When you are finished providing details to your dataflow, select Next.
When creating a dataflow to ingest data from the companies dataset, you can configure Exclude unclaimed accounts to either exclude or include unclaimed accounts from ingestion.
When individuals fill out a form, Marketo creates a phantom account record based on the Company Name that contains no other data. For new dataflows, the toggle to exclude unclaimed accounts is enabled by default. For existing dataflows, you can enable or disable the feature, with changes applying to newly ingested data and not existing data.
The Mapping step appears, providing you with an interface to map the source fields from your source schema to their appropriate target XDM fields in the target schema.
Each Marketo dataset has its own specific mapping rules to follow. See the following for more information on how to map Marketo datasets to XDM:
Based on your needs, you can choose to map fields directly, or use data prep functions to transform source data to derive computed or calculated values. For comprehensive steps on using the mapping interface, see the Data Prep UI guide.
Once your mapping sets are ready, select Next and allow for a few moments for the new dataflow to be created.
The Review step appears, allowing you to review your new dataflow before it is created. Details are grouped within the following categories:
Once you have reviewed your dataflow, select Save & ingest and allow some time for the dataflow to be created.
Once your dataflow has been created, you can monitor the data that is being ingested through it to see information on ingestion rates, success, and errors. For more information on how to monitor dataflows, see the tutorial on monitoring dataflows in the UI.
Custom attributes in datasets cannot be retroactively hidden or removed. If you want to hide or remove a custom attribute from an existing dataset, then you must create a new dataset without this custom attribute, a new XDM schema, and configure a new dataflow for the new dataset that you create. You must also disable or delete the original dataflow that consists of the dataset with the custom attribute you want to hide or remove.
You can delete dataflows that are no longer necessary or were incorrectly created using the Delete function available in the Dataflows workspace. For more information on how to delete dataflows, see the tutorial on deleting dataflows in the UI.
By following this tutorial, you have successfully created a dataflow to bring in Marketo data. Incoming data can now be used by downstream Platform services such as Real-Time Customer Profile and Data Science Workspace. See the following documents for more details:
The following sections provide additional guidelines that you may follow when using the Marketo source.
The following error messages are displayed in the UI when Platform detects issues with your setup:
Authentication will be denied if your Munchkin ID is not mapped to the Platform organization that you are using. Configure the mapping between your Munchkin ID and your organization using the Marketo interface.
A dataflow will fail to save and ingest if a primary identity is missing. Ensure that a primary identity exists within your XDM schema, before attempting to configure a dataflow.