Ingest the data
-
In the Platform user interface, select Datasets in the left navigation
-
Open your
Luma Loyalty Dataset
-
Scroll down until you see the Add Data section in the right column
-
Upload the
luma-loyalty.json
file. -
Once the file uploads, a row for the batch will appear
-
If you reload the page after a few minutes, you should see that the batch has successfully uploaded with 1000 records and 1000 profile fragments.
- Enabling error diagnostics generates data about the ingestion of your data, which you can then review using the Data Access API. Learn more about it in the documentation.
- Partial ingestion allows you to ingest data containing errors, up to a certain threshold which you can specify. Learn more about it in the documentation
Validate the data
There are a few ways to confirm that the data was successfully ingested.
Validate in the Platform user interface
To confirm that the data was ingested into the dataset:
-
On the same page where you have ingested the data, select the Preview dataset button on top-right
-
Select the Preview button and you should be able to see some of the ingested data.
To confirm that the data landed in Profile (may take a few minutes for the data to land):
- Go to Profiles in the left navigation
- Select the icon next to the Select identity namespace field to open the modal
- Select your
Luma Loyalty Id
namespace - Then enter one of the
loyaltyId
values from your dataset,5625458
- Select View
Validate with data ingestion events
If you subscribed to data ingestion events in the previous lesson, check your unique webhook.site URL. You should see three requests show up in the following order, with some time in between them, with the following eventCode
values:
ing_load_success
—the batch as ingestedig_load_success
—the batch was ingested into identity graphps_load_success
—the batch was ingested into profile service
See the documentation for more details on the notifications.
Ingest data in batches with Platform API
Now let’s upload data using the API.
Download and prep the data
- You should have already downloaded and unzipped luma-data.zip into your
Luma Tutorial Assets
folder. - Open
luma-crm.json
in a text editor and replace all instances of_techmarketingdemos
with your own underscore-tenant id, as seen in your schemas - Save the updated file
Get the dataset id
First we let’s get the id of the dataset id of the dataset into which we want to ingest data:
- Open Postman
- If you don’t have an access token, open the request OAuth: Request Access Token and select Send to request a new access token, just like you did in the Postman lesson.
- Open your environment variables and make sure the value of CONTAINER_ID is still
tenant
- Open the request Catalog Service API > Datasets > Retrieve a list of datasets. and select Send
- You should get a
200 OK
response - Copy the id of the
Luma CRM Dataset
from the Response body