Convert your data model to XDM

Transcript

When you ingest data into Adobe Experience Platform, it has to use an Experience Data Model or XDM. The modeling is a little different from what you might be used to with relational databases. So, in this video, I want to walk you through an example of how data architects can take their existing data model and convert it into an Experience Data Model. Just a heads up, this video is going to be a little bit longer than the ones I usually make. This is a big topic and I’m just going to cover it in a single video. Let’s convert the data model of our sample branded, Luma, a fitness retailer. Although we’ll use a retail example, the concepts are industry agnostic and you should be able to apply them to your own industry. The product documentation contains the examples of other common industry models, which you can review as well. We’ll look at the difference in modeling approaches using entity relationship diagrams. Luma starts with this relational database model containing a very normalized data structure with data split into multiple dimension tables. XDM uses a de-normalized design. This enhances speed and performance, allowing experience platform to support real-time customer profiles and streaming segmentation. After converting it to XDM, it will look like this. All of the fields are organized around classes, with an individual profile class for customer details and experience event class for customer events in store, product and promotions classes. We’ve collapsed all of the product related entities into one and we did the same for the store and promotion details. Fewer relationships and joins means that profiles can load faster, enabling Real-time Customer Profiles and streaming segmentation. The schemes that were actually build in the interface will look like this. I recommend a separate schema for each data source to help ensure integrity when mapping incoming data to your schemas. Behind the scenes, real time customer profile will automatically merge schemas of the same class to a union schema, which is essentially the previous diagram I just showed you. Now, let’s dive into how to convert our relational database model to an XDM Model. Here are the steps. We’ll identify the source system of each entity, categorize our entities into record or time series data, denormalize our entities by putting them into logical groups, map each logical group to standard XDM classes, map fields in the entity to standard field groups, create new classes and field groups for any remaining entities in fields, list the identities that can be used for each source, identify namespaces for each identity and finally, create new namespaces for any that are needed. We’ll start by listing out the source system of each entity. I’ll convert the entities for my ERD into a spreadsheet. This first step is mostly to support that best practice of building separate schemas for each data source. Next, we’ll categorize our entities. Now, look at each entity and determine if it’s record data or time series data. Are they attributes or something or are they things that happened? You might remember this example from the overview video. Next, we’ll de normalize our entities with logical groupings. The store transactions and web events represent actions of a customer. Our customer CRM and loyalty entities are all attributes of a customer. Store, store location and location are all attributes of a store so we’ll group those. We have a bunch of product attributes we can also group and promotions. Now, we’re onto mapping the groups to standard classes. A standard class is one that Adobe provides to you out of the box. A great way to explore the standard classes is by opening up the platform interface and going to the schema’s area. You can go to the classes tab and see all of the available classes. An even better way to explore them is to begin the process of creating a schema. Here, I can use the industry filters and preview icons to quickly see the most relevant classes and preview their fields. XDM ExperienceEvent is used for most time series data. We’ll use that for both our store transaction and web event data. Note how the class will contribute some of our fields like event type and timestamp. XDM Individual Profile is used for attributes of an individual. We’ll use that for our customer data. There’s also a promotion class which we can use for our promotions. There are no standard classes for stores and products so I’ll mark those as not available. Next, we need to look at our fields. Fields come from both classes and field groups. The rest of my fields will come from field groups. Field groups are just groups of fields organized around a theme. Back in the interface, I’ll select the XDM ExperienceEvent class so I can see the compatible field groups. Again, I can use the industry filter to shorten the list. Using the preview, I can see that the commerce details group contains the fields I need for purchases and products. I can also search for specific fields to locate the field groups that contain them. As you can tell this step might take some time. I’m going to jump ahead and complete the column in my spreadsheet, indicating areas where standard field groups met my needs. After examining all of this standard field groups and classes, we know which custom classes and field groups we’ll need to build for our model, which I’ll note in the next column.

Now, we’re on to identities. Identity fields are used for multiple purposes. For example, to construct identity graphs, which are critical to creating real-time customer profiles from multiple data sources to honor GDPR requests and related privacy functions and they’re also used as the person ID in customer journey analytics. So, they’re important to have for basically every usage of platform. I’d recommend that you specify them in every schema, especially the ones used for data ingestion. Identities are similar, but different from keys in relational databases. We have two categories of identities, primary and secondary. Primary identities mostly play a role when using real-time customer profile. They should exist in every record you ingest into the relevant dataset and should be unique. It’s likely that the primary key values you use now will be what you want to use as primary identities. Secondary identities are other fields that can uniquely identify an individual, but which may or may not be present. In the Luma’s loyalty system, they would use the loyalty ideas, their primary identity, which I’ve indicated with the asterisks in the ERD and the CRM ID as a secondary identity. By being present as an identity in both the loyalty and CRM data, those two data sources can merge individual profiles on that value. Since CRM ID can also be set from the website when a customer logs in, that will merge the website events as well. Get the idea? Be careful with fields such as phone numbers and email addresses, which might not always be unique to an individual user. I should also mention that identity fields must use this string type. So, if you include the data types in your ERD, make sure to label these as strings. Identity graph doesn’t use the field names to see the identity relationships. They use an abstraction called the identity namespace. There are some out of-the-box namespaces which you can see in the platform interface. Again, if you don’t see what you need, note that, and then list out what custom namespaces you’ll need to create. Oh, so we got through it all. Looking back at our ERD, you can see with the color coding, how the classes are used to organize the types of data and how the identity fields help join customer data together from various sources to create real time customer profiles. Each entity now models a specific schema you’ll build out in platform for each of your data sources, good luck. -