Set up the source

Upload sample file

You need to upload a sample data file to your Data Landing Zone via Azure Storage Explorer so that you can use it during the lab. To do so, do the following:

  1. Download the Sample Files
  2. Drag 'n drop and/or upload the Lab_Customer_Account.csv file to the Data Landing Zone you saved from the previous step.

When uploaded your screen should look like the below screenshot.

WARNING
Make sure you do not upload the file into the project folder. It contains preloaded data that you don’t use in our labs.

Data Landing Zone file browser showing the uploaded LabCustomerAccount.csv file, not the project folder

  1. Go to Adobe Experience Platform and navigate to: Sources -> Catalog -> Cloud storage
  2. Click on Setup / Add Data for the Data Landing Zone

Setup or Add data action for the Data Landing Zone cloud storage source

NOTE
If at least one connection exists for that source, you see Add data as the default action. If no connections exist for that source, you see Setup as the default action

Preview the file

  1. Select the Lab_Customer_Account.csv

    Selecting the LabCustomerAccount.csv file to preview in Azure Storage Explorer

  2. In the preview pane, look at the following attributes and observe the following:

    • sms_optIn is a consent field that has several missing values (shown in preview as - )
    • account_create_date does not have the proper date format. It has string values along with date and time values in one string.
    • account_end_date has the proper date format.

    smsoptIn field with several missing values shown in the file preview

    accountcreatedate and accountenddate fields shown in the file preview

    note
    NOTE
    You will need to deal with the missing values, dates and improperly formatted fields in the mapping steps later in this lab
  3. Click Next in the upper right corner of the screen to continue to the next step

Set up the dataflow

  1. In the Dataflow detail screen, choose New dataset.
  2. Name the output dataset as Customer Account - <Your Initials>
  3. Select the dep: Customer Account schema from the dropdown list.
  4. Turn on the Profile dataset toggle box.
    (If you do not turn this on, the Profile Store is not able to monitor for new data entering this dataset and hence does not ingest this data into Profile)
  5. Turn on the Enable partial ingestion.
    (If you do not turn this on, the ingestion may fail if one of the records has errors)
  6. Set the Dataflow name as Customer Account Batch Ingestion – <Your Initials>
  7. Turn on all the alerts Sources Dataflow Start/Success/Failure

Dataflow detail screen with new dataset, profile toggle, and partial ingestion settings configured

CAUTION
Ensure you have enabled the dataset for both profile and partial ingestion.

Click Next in the upper right corner of the screen to continue to the next step.

NOTE
Enable partial ingestion specifies the number of errors (INGEST and DCVS) as a percentage of the total number of records that can fail before the entire dataflow is declared a failure.
recommendation-more-help
blueprints-learn-help-blueprints