Model data in schemas

In this lesson, you will model Luma’s data into schemas. This is one of the longest lessons in the tutorial, so get a glass of water and buckle up!

Standardization and interoperability are key concepts behind Adobe Experience Platform. Experience Data Model (XDM) is an effort to standardize customer experience data and define schemas for customer experience management.

XDM is a publicly documented specification designed to improve the power of digital experiences. It provides common structures and definitions for any application to use to communicate with Platform services. By adhering to XDM standards, all customer experience data can be incorporated into a common representation that can deliver insights in a faster, more integrated way. You can gain valuable insights from customer actions, define customer audiences through segments, and express customer attributes for personalization purposes.

XDM is the foundational framework that allows Adobe Experience Cloud, powered by Experience Platform, to deliver the right message to the right person, on the right channel, at exactly the right moment. The methodology on which Experience Platform is built, XDM System, operationalizes Experience Data Model schemas for use by Platform services.

Data Architects will need to create schemas outside of this tutorial, but Data Engineers will work closely with the schemas created by the Data Architect.

Before you begin the exercises, watch this short video to learn more about schemas and the Experience Data Model (XDM):

Permissions required

In the Configure Permissions lesson, you set up all the access controls required to complete this lesson.

Create Loyalty Schema via UI

In this exercise, we will create a schema for Luma’s loyalty data.

1. Go to the Platform user interface and ensure that your sandbox is selected.
2. Go to Schemas in the left navigation
3. Select the Create Schema button on the top right
4. From the dropdown menu, select XDM Individual Profile, since we will be modeling attributes of an individual customer (points, status, and so on).
   

Add standard field groups

Next you will be prompted to add field groups to the schema. All fields must be added to schemas using groups. You can choose from a large set of industry-standard field groups provided by Adobe or create your own. As you start modeling your own data in Experience Platform, it is good to become familiar with the industry-standard field groups provided by Adobe. Whenever possible, it is a best practice to use them as they sometimes power downstream services, such as Customer AI, Attribution AI, and Adobe Analytics.

When working with your own data, a big step will be to determine which of your own data should be captured in Platform and how it should be modeled. This large topic is discussed in more depth in the course Model Your Customer Experience Data with XDM. In this tutorial, I will just be guiding you through the implementation of some pre-determined schemas.

To add field groups:

1. In the Add Field groups modal, select following field groups:

   1. Demographic Details for basic customer data like name and birthdate
   2. Personal Contact Details for basic contact details like email address and phone number

2. You can preview the fields contributed in the field group by selecting the icon on the right side of the row.
   

3. Check the Industry > Retail box to expose industry-specific field groups.

4. Select Loyalty to add the loyalty program fields.

5. Select Add field group to add all three field groups to the schema.
   

Now take some time to explore the current state of the schema. The field groups have added standard fields related to a person, their contact details, and loyalty program status. You may find these two field groups useful when you create schemas for your own company’s data. Select a specific field group row or check the box next to the field group name to see how the visualization changes.

To save the schema:

1. Select the top node of the schema.
2. Enter Luma Loyalty Schema as the Display Name.
3. Select Save.
   

Add a custom field group

Now let’s create a custom field group.

While the loyalty field group contained a loyaltyID field, Luma would like to manage all of their system identifiers in a single group to help ensure consistency across their schemas.

Field groups must be created in the schema workflow. To create the field group:

1. Select Add under the Schema Field Groups heading
   
2. Select Create new field group
3. Use Luma Identity profile field group as the Display name
4. Use system identifiers for XDM Individual Profile class as the Description
5. Select Add field groups
   

The new, empty field group is added to your schema. The + buttons can be used to add new fields to any location in the hierarchy. In our case we want to add fields at the root level:

1. Select + next to the name of the schema. This adds a new field under your tenant id namespace to manage conflicts between your custom fields and any standard fields.

2. In the Field properties sidebar add the details of the new field:

   1. Field name: systemIdentifier
   2. Display name: System Identifier
   3. Type: Object
   4. Select Apply

   

Now add two fields under the systemIdentifier object:

1. First field
   1. Field name: loyaltyId
   2. Display name: Loyalty Id
   3. Type: String
2. Second field
   1. Field Name: crmId
   2. Display Name: CRM Id
   3. Type: String

Your new field group should look like this. Select the Save button to save your schema, but leave the schema open for the next exercise.

Create a data type

Field groups, such as your new Luma Identity profile field group, can be reused in other schemas, allowing you to enforce standard data definitions across multiple systems. But they can only be reused in schemas that share a class, in this case the XDM Individual Profile class.

The data type is another multi-field construct which can be reused in schemas across multiple classes. Let’s convert our new systemIdentifier object into a data type:

With the Luma Loyalty Schema still open, select the systemIdentifier object and select Convert to new data type

If you Cancel out of the schema and navigate to the Data types tab, you will see your newly created data type. We will use this data type later in the lesson.

Create CRM Schema via API

Now we will create a schema using the API.

First we create the empty schema:

1. Open Postman

2. If you haven’t made a request in the last 24 hours, your authorization tokens have probably expired. Open the request Adobe I/O Access Token Generation > Local Signing (Non-production use-only) > IMS: JWT Generate + Auth via User Token and select Send to request new JWT and Access Tokens.

3. Open your environment variables and change the value of CONTAINER_ID from global to tenant. Remember, you must use tenant whenever you want to interact with your own custom elements in Platform, such as creating a schema.

4. Select Save
   

5. Open the request Schema Registry API > Schemas > Create a new custom schema.

6. Open the Body tab and paste the following code and select Send to make the API call. This call creates a new schema using the same XDM Individual Profile base class:

   {
     "type": "object",
     "title": "Luma CRM Schema",
     "description": "Schema for CRM data of Luma Retail ",
     "allOf": [{
       "$ref": "https://ns.adobe.com/xdm/context/profile"
     }]
   }
   

   NOTE

   The namespace references in this and subsequent code samples (for example https://ns.adobe.com/xdm/context/profile), can be obtained by using list API calls with the CONTAINER_ID and accept header set to the correct values. Some are also easily accessible in the user interface.

7. You should get a 201 Created response

8. Copy meta:altId from Response body. We will use it later in another exercise.
   

9. The new schema should be visible in the user interface but without any field groups
   

Add standard field groups

Now it’s time to add the field groups to the schema:

1. In Postman, Open the request Schema Registry API > Schemas > Update one or more attributes of a custom schema specified by ID.

2. In the Params tab, paste the meta:altId value from the previous response as the SCHEMA_ID

3. Open the Body tab and paste the following code and select Send to make the API call. This call adds the standard field groups to your Luma CRM Schema:

   [{
       "op": "add",
       "path": "/allOf/-",
       "value": {
         "$ref": "https://ns.adobe.com/xdm/context/profile-personal-details"
       }
     },
     {
       "op": "add",
       "path": "/allOf/-",
       "value": {
         "$ref": "https://ns.adobe.com/xdm/context/profile-person-details"
       }
     }
   ]
   

4. You should get a 200 OK status for the response and the field groups should be visible as part of your schema in the UI

   

Add custom field group

Now let’s add our Luma Identity profile field group to the schema. First, we need to find the id of our new field group, using a list API:

1. Open the request Schema Registry API > Field groups > Retrieve a list of field groups within the specified container.

2. Select the Send button to retrieve a list of all of the custom field groups in your account

3. Grab the $id value of the Luma Identity profile field group (yours will be different from the value in this screenshot)
   

4. Open the request Schema Registry API > Schemas > Update one or more attributes of a custom schema specified by ID. again

5. The Params tab should still have the $id of your schema

6. Open the Body tab and paste the following code, replacing the $ref value with the $id of your own Luma Identity profile field group:

   [{
     "op": "add",
     "path": "/allOf/-",
     "value": {
       "$ref": "REPLACE_WITH_YOUR_OWN_FIELD_GROUP_ID"
     }
   }]
   

7. Select Send
   

Verify that the field group has been added to the schema by checking both the API response and in the interface.

Create Offline Purchase Events Schema

Now let’s create a schema based on the XDM ExperienceEvent class for Luma’s offline purchase data. Since you are now getting familiar with the schema editor user interface, I will reduce the number of screenshots in the instructions:

1. Create a schema with the XDM ExperienceEvent class
2. Add the standard field group Commerce Details to capture common order details. Spend a few minutes exploring the objects inside.
3. Search for Luma Identity profile field group. It is not available! Remember that field groups are tied to a class, and since we are using a different class for this schema we can’t use it. We we need to add a new field group for the XDM ExperienceEvent class containing the identity fields. Our data type will make that really easy!
4. Select the Create new field group radio button
5. Enter the Display name as Luma Identity ExperienceEvent field group and select the Add field groups button
6. Make sure the + buttons appear on in the Structure section so you can add new fields
7. In Structure section, select + at top level of the schema
8. As the Field Name, enter systemIdentifier
9. As the Display Name, enter System Identifier
10. As the Type, select System Identifier which is the custom data type you created earlier
11. Select the Apply button
12. Name your schema Luma Offline Purchase Events Schema
13. Select the Save button

Note how the data type added all of the fields!

Also, select XDM ExperienceEvent under the Class heading and inspect some of the fields contributed by this class. Note that _id and timestamp fields are required when using the XDM ExperienceEvent class—these fields must be populated for every record you ingest when using this schema:

Create Web Events Schema

Now we are going to create one more schema for Luma’s website data. By this point you should be an expert at creating schemas! Build the following schema with these properties

Select the Consumer Experience Event field group. This field group contains the commerce and productListItems objects that were also in Commerce Details. Indeed Consumer Experience Event is a combination of several other standard field groups that are also available separately. AEP Web SDK ExperienceEvent Mixin field group also contains other field groups, including some of the same ones in Consumer Experience Event. Fortunately, they blend together seamlessly.

Notice that we didn’t add the Luma Identity ExperienceEvent field group to this schema. This is because the Web SDK has a different way of collecting identities. If you select the XDM ExperienceEvent class in the Composition section of the schema editor, you will notice that one of the fields it adds by default is called IdentityMap. IdentityMap is used by various Adobe applications to link to Platform. You will see how identities are sent to Platform via identityMap in the streaming ingestion lesson.

Create Product Catalog Schema

By using the Commerce Details and Consumer Experience Event field groups, Luma reports some details of product-related events via the standard productListItems data type. But they also have additional product detail fields that they would like to send to Platform. Instead of capturing all of these fields in their point-of-sale and e-commerce systems, Luma would prefer to ingest these fields directly from their product catalog system. A “schema relationship” allows you to define a relationship between two schemas for the purposes of classification or lookups. Luma will use a relationship to classify their product details. We will begin the process now and complete it at the end of the next lesson.

First we must create a schema for Luma’s product catalog using a custom class:

1. Select the Create schema button and select the Browse option from the dropdown
   
2. Select the Create new class radio button
3. Name it Luma Product Catalog Class
4. Leave the Behavior as Record
5. Select the Assign Class button
   
6. Create a new field group called Luma Product Catalog field group with the following fields:
   1. productName: Product Name: String
   2. productCategory: Product Category: String
   3. productColor: Product Color: String
   4. productSku: Product SKU: String | Required
   5. productSize: Product Size: String
   6. productPrice: Product Price: Double
7. Name the schema Luma Product Catalog Schema (be sure to update the correct field and not update the class name)
8. Save the schema

Your new schema should look like this. Note how the productSku field is listed in the Required fields section:

The next step is to define the relationship between the two ExperienceEvent schemas and the Luma Product Catalog Schema, however there are a few additional steps that we must take in the next lesson before we can do that.

Additional Resources

• Experience Data Model (XDM) System documentation
• Schema Registry API

Now that you have your schemas you can map identities!

Business.Adobe.com resources