AEP - Automatic mapping failure on boolean fields during data ingestion
Description
In this example, a CRM data ingestion includes the client’s flow with the customers’ master record. The record includes boolean flags about the customers’ privacy consents.
- The ingestion process is based on a batch import from an S3 bucket.
- On the CRM side, such fields can arrive as integers (0,1) or as strings (“0”, “1”).
- When ingesting the data all the boolean type fields have been imported in the target dataset as “FALSE”, no matter the original fields contains integer value 1 or string value “1”.
Expected behaviour:
- the schema-based import from an S3 bucket should properly map integer 1 and string “1” to “TRUE” boolean value.
Resolution
Boolean 0 or 1 is not supported. You would need to use the mapper functions to convert this to boolean values.