Mapping XDM types to other formats
The sections below describe how each XDM type maps to other common serialization formats:
Among the standard XDM types listed in the tables below, the Map type is also included. Maps are used in standard schemas when data is represented as keys that map to certain values, or where keys cannot reasonably be included in a static schema and must be treated as data values.
Many standard XDM components use map types, and you can also define custom map fields if desired. The map type’s inclusion in the tables below is intended to help you determine how to map your existing data to XDM if it is currently stored in any of the formats listed below.
Parquet, Spark SQL, and Java
XDM type | Parquet | Spark SQL | Java |
---|---|---|---|
String | Type: BYTE_ARRAY Annotation: UTF8 | StringType | java.lang.String |
Number | Type: DOUBLE | LongType | java.lang.Double |
Long | Type: INT64 | LongType | java.lang.Long |
Integer | Type: INT32 Annotation: INT_32 | IntegerType | java.lang.Integer |
Short | Type: INT32 Annotation: INT_16 | ShortType | java.lang.Short |
Byte | Type: INT32 Annotation: INT_8 | ByteType | java.lang.Short |
Date | Type: INT32 Annotation: DATE | DateType | java.util.Date |
DateTime | Type: INT64 Annotation: TIMESTAMP_MILLIS | TimestampType | java.util.Date |
Boolean | Type: BOOLEAN | BooleanType | java.lang.Boolean |
Map | MAP -annotated group( <key-type> must be STRING ) | MapType ( keyType must be StringType ) | java.util.Map |