Transformations > Hierarchy Processor transformation > Adding incoming fields to output groups
  

Adding incoming fields to output groups

You can add incoming fields to the output individually or you can add an entire input group. You can add incoming fields to an output group or to an array or struct field.
To add output fields, click the Add link on the Hierarchy Processor tab next to the incoming field or input group that you want to add.
You can add fields in the following ways based on the input field data type and the output type:
Add incoming field
Adds the selected incoming field to the selected output group or field.
You can select this option when you add incoming fields with primitive data types to the output.
Add input group
Adds all incoming fields in the group to the selected output group or field.
You can select this option when you add incoming fields with primitive data types to the output.
Add all descendants
Adds all children under the field to the output, including all arrays and structs. If the incoming field contains arrays, Data Integration creates a separate output group for each array.
You can select this option when you select an incoming struct or array field, and the output is relational.
Add single occurring children
Adds all single occurring children under the field to the output, including single occurring child fields that are nested under a struct. Does not add single occurring children that are nested under arrays.
You can select this option when you select an incoming struct or array field, and the output is relational.
Add primitive single occurring children
Adds all primitive single occurring children under the field to the output, including the primitive, single occurring child fields that are nested under a struct. Does not add single occurring children that are nested under arrays.
You can select this option when you select an incoming struct or array field, and the output is hierarchical.
Preserve incoming field
Preserves the hierarchical structure of the selected field in the output. For example, if you add an array of structs to the output group, then the output group contains an array of structs with the same structure.
You can select this option when you select an incoming struct or array field, and the output is hierarchical.
Flatten selected array
Flattens an array of primitives into a primitive field. Creates one output record for each element in the array.
You can select this option when you select an incoming struct or array field, and the output is hierarchical.
Add selected array as struct
Flattens an array of structs into a struct field. Creates one output record for each element the array.
You can select this option when you select an incoming struct or array field, and the output is hierarchical.
After you add the incoming fields to the output group, you can rename or delete output fields as needed.

Add all descendants

When you add incoming struct and array fields to the output, you can choose to add all descendants. This option adds all children under the field to the output group, including all arrays and structs. You can select this option when the output is relational.
Hierarchical data with arrays cannot be mapped to the same output group when you select this option. If you add an incoming field that contains hierarchical data with arrays to the output group, Data Integration creates a separate output group for each array.

Example

You want to add all fields in the Input field to relational output groups. The incoming fields contain the arrays Customer and Order.
Add the Input field to the output fields by selecting Add All Descendants.
The following image shows the transformation input and output:
The list of incoming fields on the left includes a top-level struct: Customers, which contains an array: Customer. The next struct is Orders, which contains an array: Order. The output groups on the right, after Add All Descendants, are Customer and Order.
Data Integration maps the fields in the Customer array to the first output group. It maps all fields in the Order array to the second output group.

Add single occurring children

When you add incoming struct and array fields to the output, you can choose to add single occurring children. This option adds all single occurring children under the field to the output group, including the single occurring child fields that are nested under a struct. You can select this option when the output is relational.
When you select this option, the incoming field must have child objects.
If the field you select contains an array, the array and its children are not added because an array can contain multiple elements.

Example

You want to extract customer information from the Customer array. Each output record should contain information about one customer.
Add the Customer array to the output group and choose Add Single Occurring Children.
The following image shows the incoming and output fields:
The list of incoming fields on the left includes a top-level struct: Customers. Below that struct is an array: Customer. The Customer array contains five string fields and a struct: FullAddress. The struct FullAddress contains three string fields. The output group on the right contains the eight string fields.
All single occurring children under Customer, including the children under the struct FullAddress, map to the output group.
If you choose Add Single Occurring Children on the Customers struct, an error message displays because there are no single occurring children under that top-level struct.

Add primitive single occurring children

When you add incoming struct and array fields to the output, you can choose to add primitive single occurring children. This option adds all primitive single occurring children under the field to the output group, including the primitive, single occurring child fields that are nested under a struct. You can select this option when the output is hierarchical.
When you select this option, the incoming field must have child objects.
If the field you select contains an array, the array and its children are not added because an array can contain multiple elements.

Example

You want to extract the make, model, company, and policy number from an array of vehicle records. Each output record should contain information about one vehicle.
Add the vehicle array to the output group and choose Add Primitive Single Occurring Children.
The following image shows the incoming and output fields:
The image shows the Hierarchy Processor tab of the Hierarchy Processor transformation. The Incoming Fields panel contains one input group with the following field: vehicle (array). The vehicle array contains the following fields: make (string), model (string) insurance (struct), and maintenance (array). The insurance struct contains the following fields: company (string) and policy_num (string). The maintenance array contains the following fields: date (string) and description (array of strings). The Output Fields panel contains the following primitive fields: make (string), model (string), company (string), and policy_num (string).
Note that the date field is not added to the output group because it is under a child array of the selected field and is not single occurring.

Preserve incoming field

When you add incoming struct and array fields to the output, you can choose to preserve the incoming field. This option copies the field to the output without changing the structure. You can select this option when the output is hierarchical.
When you add a nested array and select this option, the records that get created vary based on how you configure the data source for the output group.

Example

You want to extract the description information from a nested array of maintenance records. The description information is in an array of strings. You want the output data to also be in an array of strings.
Add the description array to the output group and choose Preserve Incoming Field.
The following image shows the incoming and output fields:
The image shows the Hierarchy Processor tab of the Hierarchy Processor transformation. The Incoming Fields panel contains one input group with the following field: vehicle (array). The vehicle array contains the following fields: make (string), model (string) insurance (struct), and maintenance (array). The insurance struct contains the following fields: company (string) and policy_num (string). The maintenance array contains the following fields: date (string) and description (array of strings). The Output Fields panel contains the following field: description (array of strings).
When Data Integration creates the output array, it sets the data source for the description array to Input.vehicle.vehicle.maintenance.maintenance.desc.elem, indicating that the information for the output array comes from the elements in the desc array in the Input group. The data source for the Output group determines the structure of the output records.
By default, Data Integration sets the data source for the Output group to Input. When you run the mapping, Data Integration collates all descriptions into one output record. To write the descriptions to separate records for each vehicle, set the data source to Input.vehicle.vehicle. To write the descriptions to separate records for each maintenance record, set the data source to Input.vehicle.vehicle.maintenance.maintenance. For more information about data source configuration, see Data source configuration.

Flatten selected array

When you add an incoming array of primitives to the output, you can choose to flatten the selected array. This option flattens the array into a field of the same data type. You can select this option when the output is hierarchical.
This option creates one record for each element in the array.

Example

You want to extract the description information from a nested array of maintenance records. The description information is in an array of strings. You want to flatten the output into a string field.
Add the description array to the output group and choose Flatten Selected Array.
The following image shows the incoming and output fields:
The image shows the Hierarchy Processor tab of the Hierarchy Processor transformation. The Incoming Fields panel contains one input group with the following field: vehicle (array). The vehicle array contains the following fields: make (string), model (string) insurance (struct), and maintenance (array). The insurance struct contains the following fields: company (string) and policy_num (string). The maintenance array contains the following fields: date (string) and description (array of strings). The Output Fields panel contains the following field: description (string).
The output contains one record for each occurrence of description in the incoming data.
For example, the incoming data contains the following record:
[
{
"vehicle": [
{
"make": "Toyota",
"model": "Corolla",
"insurance": {
"company": "Allstate",
"policy_num": "AS12876"
},
"maintenance": [
{
"date": "01/01/2020",
"description": ["oil filter1", "oil filter2"]
},
{
"date": "01/08/2020",
"description": ["tire rotation1", "tire rotation2"]
}
]
},
{
"make": "Toyota",
"model": "RAV4",
"insurance": {
"company": "Allstate",
"policy_num": "AS2033"
},
"maintenance": [
{
"date": "01/02/2020",
"description": ["air filter replacement1", "air filter replacement2"]
},
{
"date": "01/08/2020",
"description": ["battery replacement1", "battery replacement2"]
}
]
}
]
}
]
Data Integration creates the following output records, one for each occurrence of description in the incoming data:
{"description":"oil filter1"}
{"description":"oil filter2"}
{"description":"tire rotation1"}
{"description":"tire rotation2"}
{"description":"air filter replacement1"}
{"description":"air filter replacement2"}
{"description":"battery replacement1"}
{"description":"battery replacement2"}

Add selected array as struct

When you add an incoming array of primitives to the output, you can choose to add the array as a struct. This option flattens the array into a struct field without flattening any descendant arrays. You can select this option when the output is hierarchical.
This option creates one record for each element in the selected array.

Example

You want to flatten an array of vehicle records into a struct without flattening the child array of maintenance records.
Add the vehicle array to the output group and choose Add Selected Array as Struct.
The following image shows the incoming and output fields:
The image shows the Hierarchy Processor tab of the Hierarchy Processor transformation. The Incoming Fields panel contains one input group with the following field: vehicle (array). The vehicle array contains the following fields: make (string), model (string) insurance (struct), and maintenance (array). The insurance struct contains the following fields: company (string) and policy_num (string). The maintenance array contains the following fields: date (string) and description (array of strings). The Output Fields panel contains the following field: vehicle (struct). The vehicle struct contains the following fields: make (string), model (string) insurance (struct), and maintenance (array). The insurance struct contains the following fields: company (string) and policy_num (string). The maintenance array contains the following fields: date (string) and description (array of strings).
The output contains one record for each element in the vehicle array.
For example, the incoming data contains the following record which contains data about two vehicles:
[
{
"vehicle": [
{
"make": "Toyota",
"model": "Corolla",
"insurance": {
"company": "Allstate",
"policy_num": "AS12876"
},
"maintenance": [
{
"date": "01/01/2020",
"description": ["oil filter1", "oil filter2"]
},
{
"date": "01/08/2020",
"description": ["tire rotation1", "tire rotation2"]
}
]
},
{
"make": "Toyota",
"model": "RAV4",
"insurance": {
"company": "Allstate",
"policy_num": "AS2033"
},
"maintenance": [
{
"date": "01/02/2020",
"description": ["air filter replacement1", "air filter replacement2"]
},
{
"date": "01/08/2020",
"description": ["battery replacement1", "battery replacement2"]
}
]
}
]
}
]
Data Integration creates the following output records, one for each vehicle:
{
"vehicle":{
"make":"Toyota",
"model":"Corolla",
"insurance":{
"company":"Allstate",
"policy_num":"AS12876"
},
"maintenance":[
{
"date":"01/08/2020",
"description":["tire rotation2","tire rotation1"]
},
{
"date":"01/01/2020",
"description":["oil filter2","oil filter1"]
}
]
}
}
{
"vehicle":{
"make":"Toyota",
"model":"RAV4",
"insurance":{
"company":"Allstate",
"policy_num":"AS2033"
},
"maintenance":[
{
"date":"01/08/2020",
"description":["battery replacement2","battery replacement1"]
},
{
"date":"01/02/2020",
"description":["air filter replacement2","air filter replacement1"]
}
]
}
}