XDM System overview
- Topics:
- Schemas
CREATED FOR:
- Developer
Standardization and interoperability are key concepts behind Adobe Experience Platform. Experience Data Model (XDM), driven by Adobe, is an effort to standardize customer experience data and define schemas for customer experience management.
XDM is a publicly documented specification designed to improve the power of digital experiences. It provides common structures and definitions that allow any application to use to communicate with Platform services. By adhering to XDM standards, all customer experience data can be incorporated into a common representation that can deliver insights in a faster, more integrated way. You can gain valuable insights from customer actions, define customer audiences through segments, and express customer attributes for personalization purposes.
XDM is the foundational framework that allows Adobe Experience Cloud, powered by Experience Platform, to deliver the right message to the right person, on the right channel, at exactly the right moment. The methodology on which Experience Platform is built, XDM System, operationalizes Experience Data Model schemas for use by Platform services.
Learn about the role of XDM System within Experience Platform.
XDM schemas
Experience Platform uses schemas to describe the structure of data in a consistent and reusable way. By defining data consistently across systems, it becomes easier to retain meaning and therefore gain value from data.
Before data can be ingested into Platform, a schema must be composed to describe the data’s structure and provide constraints to the type of data that can be contained within each field. Schemas consist of a base class and zero or more schema field groups.
For more information on the schema composition model, including design principles, and best practices, see the basics of schema composition.
Standard XDM components
XDM provides a robust collection of standard field groups and data types, which are intended to capture common concepts and use cases across different industries. Experience Platform allows you to filter these components by industry, enabling you to quickly and confidently construct schemas that best support your particular business needs.
When constructing schemas in the Experience Platform UI, listed field groups are shown with a popularity metric. This metric is determined by how often other Platform users employ the field group in their schemas. The higher the number, the more popular the field group. By default, results are displayed from most popular to least popular, keeping you informed of data modeling trends in your industry.
Schema Library
Experience Platform provides a user interface and RESTful API from which you can view and manage all schema-related resources in the Experience Platform Schema Library. The Schema Library contains standard XDM components made available to you by Adobe, as well as resources from Experience Platform partners and vendors whose applications you use.
You can also create and manage new schemas and resources that are unique to your organization using the Schema Registry API, or the Schemas workspace in the Platform UI.
For more information on how to manage and interact with schemas in Platform, refer to the following documentation:
Data behaviors in XDM System
Data intended for use in Experience Platform is grouped into three behavior types:
- Record: Provides information about the attributes of a subject. A subject could be an organization or an individual.
- Time-series: Provides a snapshot of the system at the time that an action was taken either directly or indirectly by a record subject.
- Ad-hoc: Captures fields that are namespaced for usage only by a single dataset. Ad-hoc schemas are used in various data ingestion workflows for Experience Platform, including ingesting CSV files and creating certain kinds of source connections.
All XDM schemas describe data that can be categorized as record or time series. The data behavior of a schema is defined by the schema’s class, which is assigned to a schema when it is first created. XDM classes describe the smallest number of properties a schema must contain in order to represent a particular data behavior.
Although you are able to define your own classes within the Schema Registry, you are recommended to use the standard classes XDM Individual Profile and XDM ExperienceEvent for record and time-series data, respectively. These classes are outlined in more detail below.
NOTEXDM Individual Profile
XDM Individual Profile is a record-based class that forms a singular representation of the attributes of both identified and partially identified subjects. Profiles that are highly identified may be used for personal communications or targeted engagements. Highly identified profiles can contain detailed personal information such as name, gender, date of birth, location, and contact information including phone numbers and email addresses.
Less-identified profiles may consist only of anonymous behavioral signals like browser cookies. In this case, the sparse profile data is used to build an information base into which the interests and preferences of the anonymous profile are collated and stored. These identifiers may become more detailed over time as the subject signs up for notifications, subscriptions, purchases, and so on. This increase in profile attributes may eventually result in an identified subject and allow for a higher degree of targeted engagement.
As a profile continues to grow, it becomes a robust repository of an individual’s personal information, identification information, contact details, and communication preferences.
See the XDM Individual Profile reference guide for more information on the structure and use-case of the fields provided by the class.
XDM ExperienceEvent
XDM ExperienceEvent is a time-series-based class used to capture the state of the system when an event (or set of events) occurred, including the point in time and identity of the subject involved. Experience Events are immutable, factual records of what occurred at that point in time, representing what happened without aggregation or interpretation. They are critical for time-domain analytics as they can be used to analyze changes that occur in a given window of time, and to compare between multiple windows of time to track trends.
Experience Events can be either explicit or implicit. Explicit events are directly observable human actions taking place during a point in a journey. Implicit events are events that are raised without a direct human action, but still relate to an individual. Examples of implicit events can include the scheduled sending of email newsletters or battery voltage reaching a certain threshold.
While not all events are easily categorized across all data sources, it is extremely valuable to harmonize similar events into similar types where possible for processing.
See the XDM ExperienceEvent reference guide for more information on the structure and use-case of the fields provided by the class.
XDM schemas and Experience Platform services
Experience Platform is schema-agnostic, meaning that any schema that conforms to the XDM standard is made available to Platform services. The ways in which different Platform services use schemas are outlined in more detail below.
Catalog Service, Data Ingestion, and data lake
Catalog Service is the system of record for Experience Platform assets and their related schemas. Catalog does not contain the actual data files or directories, but rather it holds the metadata and descriptions of those files and directories.
Catalog data is stored in the data lake, a highly granular data store containing all data managed by Platform, regardless of origin or file format.
To begin ingesting data into Experience Platform, you can use Catalog Service to create a dataset. The dataset references an XDM schema that describes the structure of the data to be ingested. If a dataset is created without a schema, Experience Platform derives an “observed schema” by inspecting the type and content of ingested data fields. Datasets are then tracked in the Catalog Service and stored in the data lake alongside the schemas and observed schemas on which they are based.
See the Catalog Service overview for more information. See the Data Ingestion overview for more information on Adobe Experience Platform Data Ingestion.
Query Service
You can use standard SQL to query Experience Platform data to support many different use cases with Adobe Experience Platform Query Service.
After a schema has been composed and a dataset has been created which references that schema, data is then ingested and stored in the data lake. You can then use Query Service to join any datasets in the data lake and capture the query results as a new dataset for use in reporting, machine learning, or ingestion into Real-Time Customer Profile.
Refer to the Query Service overview for more information on the service.
Real-Time Customer Profile
Real-Time Customer Profile provides a centralized consumer profile for targeted and personalized experience management. Each profile contains data that is aggregated across all systems and include actionable timestamped accounts of events that involve the profile subject. These events may have taken place in any of the systems you use with Experience Platform.
Real-Time Customer Profile consumes schema-formatted data based on the XDM Individual Profile and XDM ExperienceEvent classes, and responds to queries based on that data.
The system maintains an instance of each customer profile, merging data together to form a “single source of truth” for the individual. This unified data is represented using what is known as a “union schema” (sometimes referred to as a “union view”). A union schema aggregates the fields of all schemas that implement the same class into a single schema. When composing a schema using the UI or API, you can enable the schema for use with Real-Time Customer Profile and tag it for inclusion in the union. The tagged schema will then participate in the schema definition being fed to Profile.
As XDM Individual Profile and XDM ExperienceEvent data is ingested into the data lake, Real-Time Customer Profile ingests any data that has been enabled for its use. The more interactions and details that are ingested, the more robust individual profiles become.
XDM Individual Profile data helps inform and empower actions across any channel or Adobe product integration. When paired with a rich history of behavioral and interaction data, this data can be used to power machine learning. The Real-Time Customer Profile API can also be used to enrich the functionality of third-party solutions, CRMs, and proprietary solutions.
See the Real-Time Customer Profile overview for more information.
Data Science Workspace
NOTEAdobe Experience Platform Data Science Workspace uses machine learning and artificial intelligence to gain insights from data stored within Experience Platform. Data Science Workspace allows data scientists to build recipes based on XDM Individual Profile and XDM ExperienceEvent data about customers and their activities. These recipes facilitate predictions such as buying propensity and recommended offers that the individual is likely to appreciate and use.
With Data Science Workspace, data scientists can easily create intelligent service APIs powered by machine learning. These services work with other Adobe solutions, including Adobe Target and Adobe Analytics Cloud, to help you automate personalized, targeted digital experiences.
For more information on using Experience Platform data to power insights, see the Data Science Workspace overview.
Next steps and additional resources
Now that you better understand the role of schemas throughout Experience Platform, you are ready to start composing your own.
To learn design principles and best practices for composing schemas to be used with Experience Platform, begin by reading the basics of schema composition. For step-by-step instructions on how to create a schema, see the tutorials on creating a schema using the API or using the user interface.
To reinforce your understanding of XDM System in Experience Platform, watch the following video:
I’m going to give you an overview of an important concept in Adobe Experience Platform called Experience Data Model, or XDM, the data model for describing customer experiences. When you use XDM to build schemas and ingest data adhering to these schemas, you can begin to use the XDM System. XDM System represents the broader application of XDM throughout the platform ecosystem, supporting customer journey analysis, data science and marketing activation use cases.
Experience Platform implementations begin by laying this XDM foundation. You compose schemas that describe your company’s data in a structured, reusable way. Once you’ve defined the structure of your data, you can stream or batch that data into datasets. Then, you can use that data to generate insights with downstream platform services. And, to activate customer profiles across marketing channels with intelligent decisioning and segmentation. Schemas are created by data architects, but they’re fundamental to the workflows of almost every experienced platform user. So schemas are a crucial foundation of Experience Platform. Let’s dive into the different components that go into schemas to help you better understand their composition. Let’s start with a really simple modeling exercise. What is an experience? In a business to consumer context, it’s basically just somebody doing something. How would we model that in XDM? Let’s cut a word and essentialize it even a little further. Putting my elementary school grammar hat on, there are two main components here, a noun and a verb, who they are and what they’re doing. Experience data can be classified into two distinct categories, record and time series. Record data collects attributes about a subject, typically an individual person. In our example, you can think of record attributes as adjectives for our noun, like name, address, loyalty program status.
The other type of experience data is time series data, which you can think of as being related to our verb describing what this subject did with context around it. To build schemas for these two types of data, our first step is to add a class. For this example, we’ll use our two most common standard classes XDM Individual Profile class for individual attributes and XDM ExperienceEvent for the actions.
If we want to get fancy, we can use another class for additional product details, which would be of record type because it’s for attributes of that subject. In that case, a product. A class imparts the basic concept of what schema describes. In this case, a person, an event and a product. Classes describe the smallest number of common properties that all schemas based on that class will need. For example, all ExperienceEvents will need a timestamp field for when an event occurred.
Next, we need to add the additional fields to our schemas. This is done with field groups. Field groups are reusable components of structured fields, organized around concepts. So in our example, we would use the loyalty details field group for the gold member status and demographic details for the name and address. For our experience event schema, the web SDK field group could collect a lot of the standard web fields and commerce details could capture the order details. Field groups are reusable, so by using the same commerce details field group in a schema for in-store purchases would ensure we’re ingesting those order details with the same field definitions and structure as our web transactions, allowing marketers to easily create segments independent of the purchase method. Experience platform includes many standard classes and field groups that cover common industry use cases. You can also define your own classes and field groups to represent data for your particular business needs. When building real time customer profiles with Experience Platform, another key component of are identities. Identities are what link together data from different data sets. In our example, both a profile and ExperienceEvent of schema would need to share a common identity field and pass in a common value for those records to come together. The product schema would also need to share a common field and value with the ExperienceEvent schema for that record to connect. Typically the product SKU. Understanding your business use cases and building solid schemas with the right identity fields is critical. But once this foundation is in place, it will help ensure high integrity data and platform, providing insights into the customer’s activities and preferences. As marketers, data scientists and analysts, we can move beyond simply identifying the customer to understanding the environment in which they operate and how it may have influenced their actions. We can also explore similarities between subjects and express those similarities as segments. So that’s a quick overview of an Experience data model in the XDM system. Thanks for watching. -