Transformations > Python transformation > Python transformation fields
  

Python transformation fields

A Python transformation has incoming fields and output fields. Use incoming fields and output fields as variables in the Python code snippets that you define on the Python tab.
Add output fields on the Output Fields tab.
Add output fields for the output data that you want to pass to the downstream transformation. To add a field, click Add Field, and then enter the field name, data type, precision, and scale. You can also create output fields on the Outputs tab of the Python editor by clicking Create New Field.
After you add fields to the transformation, you can use the field names as variables in the Python code.
Use the following guidelines when you create output fields:
You can configure one or more input fields as partition keys. Data Integration uses the partition keys to repartition the data before the code runs. If you do not add an incoming fields as a partition key, the data is processed using its default partitioning scheme.

Data type conversion

A Python transformation converts transformation data types to Python data types based on the Python transformation field type.
When a Python transformation reads input rows, it converts incoming field data types to Python data types. When a Python transformation writes output rows, it converts Python data types to output field data types.
For example, the following processing occurs for an input field with the double data type:
  1. 1. The Python transformation converts the double data type in the incoming field to the Python float data type.
  2. 2. The Python transformation uses the value in the incoming field as the value for the Python float data type.
  3. 3. When the transformation generates the output row, it converts the Python float data type to the double data type.
The following table shows how the Python transformation maps transformation data types to Python data types:
Transformation data type
Python data type
Integer
Int
Decimal
Float
Double
Float
Timestamp
Datetime
String
Str
text
Text

Data types in input and output fields

In the Python transformation, the data types in corresponding incoming fields and output fields must be the same. If the data types are not the same, convert them in the Python code.
For example, you create an incoming field with the integer data type and an output field with the string data type. You define the Python code to process the data in the incoming field and write the data to the output field. In the Python code, you can use the function str() to convert the integer data type in the incoming field and write the output as a string data type in the output field.

Partition keys

In an elastic mapping, you can use partition keys to define how to group data into partitions before the Python code runs.
You can configure one or more input fields as partition keys. Data Integration Elastic uses the partition keys to repartition the data before the code runs. If you do not add an incoming field as a partition key, the data is processed using its default partitioning scheme.