Amazon S3 V2 Connector > Data type reference > Parquet Amazon S3 file data types and transformation data types
  

Parquet Amazon S3 file data types and transformation data types

Parquet Amazon S3 file data types map to transformation data types that the Secure Agent uses to move data across platforms.
The following table lists the Parquet Amazon S3 file data types that the Secure Agent supports and the corresponding transformation data types:
Parquet Amazon S3 File Data Type
Transformation Data Type
Range and Description
Boolean
Integer
TRUE (1) or FALSE (0)
Date
Date/Time
January 1,0001 to December 31,9999
Decimal
Decimal
For mappings- Precision 18 and 28 digits. Scale 0 to 28.
If you specify a precision less than 18 or 28 digits, 18 or 28 is considered as the precision.
For elastic mappings- Precision 18, 28, and 38 digits. Scale 0 to 38.
Double
Double
Precision 15
Float
Double
Precision 15
Int32
Integer
-2,147,483,648 to +2,147,483,647
Int64
Bigint
-9,223,372,036,854,775,808 to +9,223,372,036,854,775,807
8-byte signed integer
Int96
Binary
12-byte signed integer
Map
Map
Unlimited number of characters.
Struct
Struct
Unlimited number of characters.
String
String
-1 to 104,857,600 characters
Time
Date/Time
Time of the day. Precision to microsecond.
Timestamp
Date/Time
January 1,0001 00:00:00 to December 31,9999 23:59:59.997997. Precision to microsecond.
group(LIST)
Array
Unlimited number of characters.
Note: Specify the correct precision and scale in the source file. Otherwise, the decimal point is shifted when you write the source data to a target.
Amazon S3 V2 Connector supports the Parquet file type Date, Decimal, Time, and Timestamp only on Cloudera 6.1 distribution.
The Parquet schema that you specify to read or write a Parquet file must be in lower case. Parquet does not support case-sensitive schema.

Parquet Timestamp data type support

The following table lists the Timestamp data type support for Parquet file format:
Timestamp Data type
Mapping
Elastic Mapping
Timestamp_micros
Yes
No
Timestamp_millis
Yes
No
Time_millis
Yes
No
Time_micros
Yes
No
int96
Yes
Yes
Date
Yes
Yes
The Secure Agent does not support the following Parquet data types: