Create a Marketo Engage source connection and dataflow in the UI

IMPORTANT
Before creating a Marketo Engage source connection and a dataflow, you must first ensure that you have mapped your Adobe Organization ID in Marketo. Furthermore, you must also ensure that you have completed auto-populating your Marketo B2B namespaces and schemas prior to creating a source connection and a dataflow.

This tutorial provides steps for creating a Marketo Engage (hereinafter referred to as “Marketo”) source connector in the UI to bring B2B data into Adobe Experience Platform.

Getting started

This tutorial requires a working understanding of the following components of Adobe Experience Platform:

  • B2B namespaces and schema auto-generation utility: The B2B namespaces and schema auto-generation utility allows you to use Postman to auto-generate values for your B2B namespaces and schemas. You must complete your B2B namespaces and schemas first, before creating a Marketo source connection and dataflow.
  • Sources: Experience Platform allows data to be ingested from various sources while providing you with the ability to structure, label, and enhance incoming data using Platform services.
  • Experience Data Model (XDM): The standardized framework by which Experience Platform organizes customer experience data.
  • Identity namespaces: Identity namespaces are a component of Identity Service that serve as indicators of the context to which an identity relates. A fully qualified identity includes an ID value and a namespace.
  • Real-Time Customer Profile: Provides a unified, real-time consumer profile based on aggregated data from multiple sources.
  • Sandboxes: Experience Platform provides virtual sandboxes which partition a single Platform instance into separate virtual environments to help develop and evolve digital experience applications.

Gather required credentials

In order to access your Marketo account on Platform, you must provide the following values:

Credential
Description
munchkinId
The Munchkin ID is the unique identifier for a specific Marketo instance.
clientId
The unique client ID of your Marketo instance.
clientSecret
The unique client secret of your Marketo instance.

For more information on acquiring these values, refer to the Marketo authentication guide.

Once you have gathered your required credentials, you can follow the steps in the next section.

Connect your Marketo account

In the Platform UI, select Sources from the left navigation bar to access the Sources workspace. The Catalog screen displays a variety of sources with which you can create an account.

You can select the appropriate category from the catalog on the left-hand side of your screen. Alternatively, you can find the specific source you wish to work with using the search bar.

Under the Adobe applications category, select Marketo Engage. Then, select Add data to create a new Marketo dataflow.

catalog

The Connect Marketo Engage account page appears. On this page, you can either use a new account or access an existing account.

Existing account

To create a dataflow with an existing account, select Existing account and then select the Marketo account you want to use. Select Next to proceed.

existing

New account

If you are creating a new account, select New account. On the input form that appears, provide an account name, an optional description, and your Marketo authentication credentials. When finished, select Connect to source and then allow some time for the new connection to establish.

new

Select a dataset

After creating your Marketo account, the next step provides an interface for you to explore Marketo datasets.

The left half of the interface is a directory browser, displaying the 10 Marketo datasets. A fully-functioning Marketo source connection requires the ingestion of the nine different datasets. If you are also using the Marketo account-based marketing (ABM) feature, then you must also create a 10th dataflow to ingest the Named Accounts dataset.

NOTE
For the purposes of brevity, the following tutorial uses Opportunities as an example, but the steps outlined below apply to any of the 10 Marketo datasets.

Select the dataset you wish to ingest first, then select Next.

select-data

Provide dataflow details provide-dataflow-details

The Dataflow detail page allows you to select whether you want to use an existing dataset or a new dataset. During this process, you can also configure settings for Profile dataset, Error diagnostics, Partial ingestion, and Alerts.

dataflow-details

Use an existing dataset

To ingest data into an existing dataset, select Existing dataset. You can either retrieve an existing dataset using the Advanced search option or by scrolling through the list of existing datasets in the dropdown menu. Once you have selected a dataset, provide a name and a description for your dataflow.

existing-dataset

Use a new dataset

To ingest into a new dataset, select New dataset and then provide an output dataset name and an optional description. Next, select a schema to map to using the Advanced search option or by scrolling through the list of existing schemas in the dropdown menu. Once you have selected a schema, provide a name and a description for your dataflow.

new-dataset

Enable Profile and error diagnostics

Next, select the Profile dataset toggle to enable your dataset for Profile. This allows you to create a holistic view of an entity’s attributes and behaviors. Data from all Profile-enabled datasets will be included in Profile and changes are applied when you save your dataflow.

Error diagnostics enables detailed error message generation for any erroneous records that occur in your dataflow, while Partial ingestion allows you to ingest data containing errors, up to a certain threshold that you manually define. See the partial batch ingestion overview for more information.

IMPORTANT
The Marketo source uses batch ingestion to ingest all historical records and uses streaming ingestion for real-time updates. This allows the source to continue streaming while ingesting any erroneous records. Enable the Partial ingestion toggle and then set the Error threshold % to maximum to prevent the dataflow from failing.

profile-and-errors

Enable alerts

You can enable alerts to receive notifications on the status of your dataflow. Select an alert from the list to subscribe to receive notifications on the status of your dataflow. For more information on alerts, see the guide on subscribing to sources alerts using the UI.

When you are finished providing details to your dataflow, select Next.

alerts

Skip unclaimed accounts when ingesting companies data

When creating a dataflow to ingest data from the companies dataset, you can configure Exclude unclaimed accounts to either exclude or include unclaimed accounts from ingestion.

When individuals fill out a form, Marketo creates a phantom account record based on the Company Name that contains no other data. For new dataflows, the toggle to exclude unclaimed accounts is enabled by default. For existing dataflows, you can enable or disable the feature, with changes applying to newly ingested data and not existing data.

unclaimed accounts

Map your Marketo dataset source fields to target XDM fields

The Mapping step appears, providing you with an interface to map the source fields from your source schema to their appropriate target XDM fields in the target schema.

Each Marketo dataset has its own specific mapping rules to follow. See the following for more information on how to map Marketo datasets to XDM:

Based on your needs, you can choose to map fields directly, or use data prep functions to transform source data to derive computed or calculated values. For comprehensive steps on using the mapping interface, see the Data Prep UI guide.

mapping

Once your mapping sets are ready, select Next and allow for a few moments for the new dataflow to be created.

Review your dataflow

The Review step appears, allowing you to review your new dataflow before it is created. Details are grouped within the following categories:

  • Connection: Shows the source type, the relevant path of the chosen source entity, and the amount of columns within that source entity.
  • Assign dataset & map fields: Shows which dataset the source data is being ingested into, including the schema that the dataset adheres to.

Once you have reviewed your dataflow, select Save & ingest and allow some time for the dataflow to be created.

review

Monitor your dataflow

Once your dataflow has been created, you can monitor the data that is being ingested through it to see information on ingestion rates, success, and errors. For more information on how to monitor dataflows, see the tutorial on monitoring dataflows in the UI.

Delete your attributes

Custom attributes in datasets cannot be retroactively hidden or removed. If you want to hide or remove a custom attribute from an existing dataset, then you must create a new dataset without this custom attribute, a new XDM schema, and configure a new dataflow for the new dataset that you create. You must also disable or delete the original dataflow that consists of the dataset with the custom attribute you want to hide or remove.

Delete your dataflow

You can delete dataflows that are no longer necessary or were incorrectly created using the Delete function available in the Dataflows workspace. For more information on how to delete dataflows, see the tutorial on deleting dataflows in the UI.

Next steps

By following this tutorial, you have successfully created a dataflow to bring in Marketo data. Incoming data can now be used by downstream Platform services such as Real-Time Customer Profile and Data Science Workspace. See the following documents for more details:

Appendix appendix

The following sections provide additional guidelines that you may follow when using the Marketo source.

Error messages in the UI error-messages

The following error messages are displayed in the UI when Platform detects issues with your setup:

Munchkin ID is not mapped to the appropriate organization

Authentication will be denied if your Munchkin ID is not mapped to the Platform organization that you are using. Configure the mapping between your Munchkin ID and your organization using the Marketo interface.

An error message displaying that the Marketo instance is not correctly mapped to the Adobe organization.

Primary identity is missing

A dataflow will fail to save and ingest if a primary identity is missing. Ensure that a primary identity exists within your XDM schema, before attempting to configure a dataflow.

An error message displaying that the primary identity is missing from the XDM schema.

recommendation-more-help
337b99bb-92fb-42ae-b6b7-c7042161d089