Query schedules

You can automate query runs by creating query schedules. Scheduled queries run on a custom cadence to manage your data based on frequency, date, and time. You can also choose an output dataset for your results if required. Queries that have been saved as a template can be scheduled from the Query Editor.

Account requirements for scheduled queries technical-account-user-requirements

To help scheduled queries run reliably, Adobe recommends that administrators provision a technical account (using OAuth Server-to-Server credentials) for creating scheduled queries. Scheduled queries can also be created with a personal user account, but queries created this way will stop running if that user’s access is removed or disabled.

For details on setting up technical accounts and assigning the required permissions, see the Credentials guide prerequisites and API authentication.

For additional guidance on creating and configuring a technical account, refer to:

If you only use the Query Service UI, ensure you have the necessary permissions or coordinate with an administrator who manages technical accounts. Any scheduled queries are added to the list in the Scheduled queries tab, where you can monitor the status, schedule details, and error messages for all scheduled query jobs, as well as subscribe to alerts. For more information on monitoring and managing your queries, see the monitor scheduled queries document.

This workflow covers the scheduling process in the Query Service UI. To learn how to add schedules using the API, refer to the scheduled queries endpoint guide.

Create a query schedule create-schedule

To schedule a query, select a query template from either the Templates tab or the Template column of the Scheduled Queries tab. Selecting the template name navigates you to the Query Editor.

If you access a saved query from the Query Editor, you can create a schedule for the query or view the query’s schedule from the details panel.

Select Add schedule to navigate to the schedule details page.

Alternatively, select the Schedules tab below the query’s name.

The schedules workspace appears. The UI displays a list of any scheduled runs that the template is associated with. Select Add Schedule to create a schedule.

Add schedule details schedule-details

The schedule details page appears. Use this page to configure the schedule settings for the scheduled query. Details include the frequency and weekday of the scheduled query run, the start and end date, the dataset to export the results to, and query status alerts.

Scheduled query frequency scheduled-query-frequency

You can choose the following options for Frequency:

Provide dataset details dataset-details

Manage the query results by either appending the data to an existing dataset or creating a new dataset and appending the data to it.

Select Create and append into new dataset to create a data set when you execute a query for the first time. Subsequent executions continue to insert data into that data set. Lastly, provide a name and description for the dataset.

Alternatively, select Append into existing dataset followed by the dataset icon ( ).

The Select output dataset dialog appears.

Next, either browse the existing datasets or use the search field to filter the options. Select the row of the dataset that you wish to use. The dataset details are displayed in the panel on the right. Select Done to confirm your choice.

Quarantine queries if they continuously fail quarantine

When creating a schedule, you can enroll your query in the quarantine feature to safeguard system resources and prevent potential disruptions. The quarantine feature automatically identifies and isolates queries that repeatedly fail by placing them in a Quarantined state. By quarantining queries after ten consecutive failures, you can intervene, review, and rectify issues before allowing further executions. This helps to maintain your operational efficiency and data integrity.

Once a query is enrolled for the quarantine feature, you can subscribe to alerts for this query status change. If a scheduled query is not enrolled in quarantine, it does not appear as an option on the Alerts dialog.

You can also enroll a scheduled query into the quarantine feature from the inline actions of the Scheduled Queries tab. See the monitor queries documentation for more details.

Set alerts for a scheduled query status alerts-for-query-status

You can also subscribe to query alerts as part of your scheduled query settings. You can configure your settings to receive notifications for a variety of situations. Alerts can be set for a quarantined state, delays in query processing, or a change in status of your query. The available query-state alert options include start, success, and failure. Alerts can be received either as pop-up notifications or emails. Select the checkbox to subscribe to alerts for that status of scheduled query.

The table below explains the supported query alert types:

For an overview of alerts in Adobe Experience Platform, including the structure of how alert rules are defined, see the alerts overview. For guidance on managing alerts and alert rules within the Adobe Experience Platform UI, see the Alerts UI guide.

Set parameters for a scheduled parameterized query set-parameters

If you are creating a scheduled query for a parameterized query, you must now set the parameter values for these query runs.

After confirming your schedule details, select Save to create a schedule. You are returned to your template’s schedules tab. This workspace displays details of the newly created schedule, including the schedule ID, the schedule itself, and the schedule’s output dataset.

View scheduled query runs scheduled-query-runs

From your template’s Schedules tab, select the schedule ID to navigate to the list of query runs for your newly scheduled query.

Alternatively, to view a list of a query template’s scheduled runs, navigate to the Scheduled queries tab and select a template name from the list available.

The list of query runs for that scheduled query appears.

Compute hours at job level compute-hours

Track compute hours consumed at the query execution level for your CTAS/ITAS batch queries. This feature offers insights into compute usage, helping you optimize resource allocation and improve query performance.

The following table provides descriptions of each column available in the details section that lists scheduled query runs.

See the monitor scheduled queries guide for complete information on how to monitor the status of all query jobs through the UI.

Select a Query run ID from the list to navigate to the query run overview. For a full breakdown of the information available on the query run overview, see the monitor scheduled queries documentation.

To monitor scheduled queries using the Query Service API, see the scheduled query run endpoints guide.

Edit a schedule edit-schedule

You can access the schedule editor from the Scheduled Queries workspace, the schedule details page, or the Query Editor. You can modify supported configuration settings for an existing scheduled query without recreating the schedule. Editing a schedule only updates the schedule configuration. It does not modify the underlying SQL query definition.

Access the edit workflow access-edit-workflow

Use one of the following paths from the Scheduled Queries tab to access the edit workflow for an eligible scheduled query.

Either, select the ellipsis (…) for the schedule you want to modify, then Edit schedule from the inline actions menu. This opens the schedule editor directly. See Manage scheduled queries with inline actions for more information.

Or select a schedule name from the table to open the schedule details page. On the schedule details page, select Edit schedule from the top-right corner of the page to open the schedule editor.

You can also edit an existing qualifying schedule from the Query Editor.

Edit schedule settings edit-schedule-settings

Use the schedule editor to review the current schedule configuration and update supported settings.

The following settings can be modified:

Save schedule changes save-schedule-changes

After updating the schedule settings, select Save to apply the changes. A confirmation message appears when the schedule updates successfully.

Verify updated schedule information verify-schedule-updates

After saving, verify that the updated schedule configuration appears in the following locations:

Enable, disable, or delete a schedule delete-schedule

You can enable, disable, or delete a schedule from the schedules workspace of a particular query or from the Scheduled Queries workspace that lists all the scheduled queries.

To access the Schedules tab of your chosen query, you must select the name of a query template from either the Templates tab or the Scheduled Queries tab. This navigates to the Query Editor for that query. Form the Query Editor, select Schedules to access the schedules workspace.

Select a schedule from the rows of available schedules to populate the details panel. Use the toggle to disable (or enable) the scheduled query.

Delete disabled queries

A confirmation dialog appears. Select Disable to confirm the action.

Select Delete a schedule to delete the disabled schedule.

Alternatively, the Scheduled Queries tab offers a collection of inline actions for each scheduled query. The available inline actions include Disable schedule or Enable schedule, Delete schedule, and Subscribe to alerts for the scheduled query. For complete instructions on how to delete or disable a scheduled query through the scheduled Queries tab, please see the monitor scheduled queried guide.