On-demand ingestion for sources dataflows in the UI

You can use on-demand ingestion to trigger a flow run iteration of an existing dataflow using the sources workspace in the Adobe Experience Platform user interface.

This document provides you with steps on how to create dataflows on demand for sources, as well as how to retry flow runs that have been processed or have failed.


What is a flow run?

Flow runs represent an instance of dataflow execution. For example, if a dataflow is scheduled to run hourly at 9:00 AM, 10:00 AM, and 11:00 AM, then you would have three instances of a flow run. Flow runs are specific to your particular organization.

Getting started

In order to create a flow run, you must first have the flow ID of a dataflow that is scheduled for one-time ingestion.

This document requires a working understanding of the following components of Experience Platform:

  • Sources: Experience Platform allows data to be ingested from various sources while providing you with the ability to structure, label, and enhance incoming data using Platform services.
  • Dataflows: A dataflow is a representation of data jobs that move data across Platform. Dataflows are configured across different services, helping move data from source connectors to target datasets, to Identity Service and Real-Time Customer Profile, and to Destinations.
  • Sandboxes: Experience Platform provides virtual sandboxes that partition a single Platform instance into separate virtual environments to help develop and evolve digital experience applications.

Create a dataflow on demand create-a-dataflow-on-demand

Navigate to the Dataflows tab of the sources workspace. From here, find the dataflow that you want to run on demand, and then select the ellipses (...) beside your dataflow name.

A list of dataflows in the sources workspace.

Next, select Run on-demand from the dropdown menu that appears.

A dropdown menu with the Run on-demand option selected.

Configure the schedule of your on-demand ingestion. Select the Ingestion start time, the Date range start time, and the Date range end time.

Scheduling configuration
Ingestion start time
The scheduled time of when the on-demand flow run will begin.
Date range start time
The earliest date and time that data will be retrieved from.
Date range end time
The date and time that data will be retrieved up to.

Select Schedule and allow a few moments for your on-demand dataflow to trigger.

The scheduling configuration window for on-demand ingestion.

Select your dataflow name to view your dataflow activity. Here you will see a list of your dataflow runs that have been processed. Select a dataflow run, then select Retry from the right rail to retry ingestion for a selected dataflow run iteration.

A list of processed flow runs for a selected dataflow.

Select Scheduled to see a list of dataflow runs that are scheduled for future ingestion.

A list of scheduled flow runs for a selected dataflow.

Next steps

By reading this document, you have learned how to create flow runs on demand for existing sources dataflows. For more information on sources, read the sources overview