Query Service packaging

Last update: 2024-02-27
  • Topics:
  • Queries
    View more on this topic
  • Created for:
  • User
    Developer

This document outlines the different types of packaging and query execution capabilities available in Query Service.

Adobe Experience Platform Query Service can be divided into two capabilities based on the query patterns that can be executed:

  • Ad hoc queries are SQL queries used to explore ingested datasets for verification, validation, experimentation, and so on. These queries do not write data back into the Platform data lake.
  • Batch queries are SQL queries used to perform the post-ingestion processing of ingested datasets. These queries clean, shape, manipulate, and enrich data, the results of which are written back to the Platform data lake. These queries can be scheduled, managed, and monitored as batch jobs.

Query Service capabilities are packaged with the following products and add-ons:

  • Platform-based applications (Adobe Real-Time Customer Data Platform, Adobe Customer Journey Analytics, and Adobe Journey Optimizer): Query Service access to execute ad hoc queries is provided from the outset with every variation and tier of Platform-based applications.
  • Data Distiller (add-on package that can be purchased with Adobe Real-Time CDP, Customer Journey Analytics and Adobe Journey Optimizer): Query Service access to execute batch queries is provided with Data Distiller.

Entitlements

The following table outlines the key Query Service entitlements based on how they are packaged:

Query Service Entitlement Packaged with Platform-based applications Packaged with Data Distiller
Query Pattern Supported Ad hoc queries only Batch query
Use Case Supported
  • Exploration​
  • Data Discovery​
  • Data Validation
  • Experimentation
  • Cleaning
  • Shaping
  • Manipulating
  • Enriching
Semantics Supported
  • SELECT queries
  • CTAS and ITAS queries
Maximum Execution Time 10 minutes 24 hours
License Metric Query User Concurrency:
  • 1 concurrent user (Real-Time CDP, Adobe Journey Optimizer)​
  • 5 concurrent users (Customer Journey Analytics)​
Query Concurrency:
  • 1 concurrent running query (all applications)​
An additional ad hoc query users pack add-on can be purchased to increase your authorized ad hoc query entitlement.
  • +5 additional concurrent users per pack
  • +1 additional concurrent running query per pack
Compute Hours:
  • Variable (scoped based on your application entitlement)
Compute Hours is a measure of the amount of time taken by the Query Service engine to read, process, and write data back into the data lake when a batch query is executed.
With the Data Distiller SKU, you also get an additional user and query concurrency, which can be used towards the execution of ad hoc queries. The Data Distiller SKU includes:
  • +5 additional concurrent users
  • +1 additional concurrent running query
Accelerated query and reporting usage No Yes - Concurrent accelerated queries allow you to read data from the accelerated store and display within your dashboards. A dedicated entitlement for storing reporting models and datasets in the accelerated store is also provided.
Data lake storage capacity Your total storage entitlement is dependant on your platform-based applications licenses. For example, Real-Time CDP, AJO, CJA, and so on. Yes - An additional storage entitlement is provided to persist your raw and derived datasets for Data Distiller use cases beyond a seven-day data expiration date.
Your data lake storage capacity is measured in terabytes (TB) and depends on the quantity of Compute hours that you have bought. Please check the product description for more details.
Data export allowance Your total export entitlement is dependant on your platform-based applications licenses. For example, Real-Time CDP, AJO, CJA, and so on. Yes - An additional export entitlement is provided to allow for the export of derived datasets created using Data Distiller.
Your annual data export allowance is measured in terabytes (TB) and depends on the quantity of Compute hours that you have bought. Please check product description for more details.
Query Execution Interface
  • Query Service UI
  • Third-party client UI
  • PostgresSQL client UI
  • Query Service UI
  • Third-party client UI
  • PostgresSQL client UI
  • REST APIs
Query Results Returned Via Client UI Derived dataset stored in data lake
Result Limit
  • Query Service UI - The number of output rows can be configured with a UI setting to between 50-500 rows.
  • Third-party clients - 50,000
  • PostgresSQL client - 50,000
CTAS and ITAS queries only generate success messages as the query output is stored in derived datasets.
Read Dataset Capacity Yes Yes
Write Dataset Capacity No Yes
Scheduling Capacity No Yes
Monitoring Capacity Yes Yes
Query Alert Setup Capacity No Yes

Access control

Access control for Experience Platform is administered through the Adobe Admin Console where product profiles link users with permissions and sandboxes. See the access control overview for more information.

In order to use Query Service, the Manage Queries permission must be enabled within Admin Console. This permission allows users to execute ad hoc and batch queries. Detailed instructions for requesting access to the product profile Manage Queries permission have been outlined in the manage permissions for a product profile and manage users for a product profile documents.

After purchasing the Data Distiller add-on, the Write Dataset permission must be granted. This permission allows Data Distiller users to execute batch queries.

The following table outlines the effects of the Manage Queries permission:

Permission Function
Manage Queries (without write data permission) Provides access to execute ad hoc queries
Manage Queries (with write data permission) Provides access to execute batch queries

Sandbox support

Sandboxes are virtual partitions within a single instance of Experience Platform. Each Platform instance supports multiple production and non-production sandboxes, each maintaining its own library of Platform resources. Non-production sandboxes allow you to test features, run experiments, and make custom configurations without impacting your production sandboxes. For more information on sandboxes, see the sandboxes overview. All Query Service entitlements are shared across all sandboxes.

Next steps

By reading this document, you should have a better understanding of the different packaging types and query execution capabilities available in Query Service. To learn more about Query Service, such as well-known industry use cases, read the use case documentation. For more general information, visit the Query Service overview.

On this page