Amazon S3 V2 Connector > Amazon S3 V2 sources and targets > Amazon S3 V2 targets
  

Amazon S3 V2 targets

You can use an Amazon S3 V2 object as a target in a mapping, elastic mapping, or mapping task.
Specify the name and description of the Amazon S3 V2 target. Configure the Amazon S3 V2 target and advanced properties for the target object.

Data encryption in Amazon S3 V2 targets

To protect data, you can encrypt the Amazon S3 files when you write the files to the target. Do not use the master symmetric key and customer master key together.
Select the type of the encryption in the Encryption Type field under the Amazon S3 V2 advanced target properties.
You can select the following types of encryption:
None
The data is not encrypted.
Server-side encryption
Select Server Side Encryption as the encryption type if you want Amazon S3 to encrypt the data using Amazon S3-managed encryption keys when you write to the target.
If you do not specify the customer master key ID in the connection properties, you must select Server Side Encryption as the encryption type.
Server-side encryption with KMS
Select Server Side Encryption with KMS as the encryption type if you want Amazon S3 to encrypt the data using AWS KMS-managed customer master key encryption keys when you write to the target.
The AWS KMS-managed customer master key specified in the connection property must belong to the same region where Amazon S3 is hosted.
For example, if Amazon S3 is hosted in the US West (Oregon) region, you must use the AWS KMS-managed customer master key enabled in the same region.
Client-side encryption
Select Client Side Encryption as the encryption type if you want the Secure Agent to encrypt the data when you write to the target. Client-side encryption uses a master symmetric key, which is a 256-bit AES encryption key in Base64 format or a customer master key.
When you use a serverless runtime environment, you cannot configure client-side encryption for Amazon S3 V2 targets.
Informatica encryption
Select Informatica Encryption as the encryption type if you want to encrypt the data using Informatica crypto libraries when you write to a target. Informatica encryption is applicable only when you run mappings on the Secure Agent machine.
To encrypt a file using Informatica Encryption method, perform the following tasks:
  1. 1. Ensure that the organization administrator has permission to Informatica crypto libraries when you create an Amazon S3 V2 connection.
  2. 2. Select Informatica Encryption as the encryption type in the advanced target properties.
The following table lists the encryption type supported for various file types:
Encryption Type
Avro File
Binary File1
Delimited
JSON File2
ORC File
Parquet File
Client-side encryption
No
Yes
Yes
No
No
No
Server-side encryption
Yes
Yes
Yes
Yes
Yes
Yes
Server-side encryption with KMS
Yes
Yes
Yes
Yes
Yes
Yes
Informatica encryption
No
Yes
Yes
No
No
No
1Applies only to mappings.
2Applies only to elastic mappings.
The remaining properties are applicable for both mappings and elastic mappings.
Rules and guidelines for data encryption in Amazon S3 V2 targets
Consider the following rules and guidelines when you configure data encryption in Amazon S3 V2 targets:
For information about the Amazon S3 client encryption policy, see the Amazon S3 documentation.

Overwriting existing files

You can choose to overwrite the existing target files.
Select the Overwrite File(s) If Exists option in the Amazon S3 V2 target advanced properties to overwrite the existing files. By default, the value of the Overwrite File(s) If Exists check box is true.
If you select the Overwrite File(s) If Exists option, the Secure Agent deletes the existing files with same file name and creates a new files with the same file name in the target directory.
If you do not select the Overwrite File(s) If Exists option, the Secure Agent does not delete the existing files in the target directory. The Secure Agent adds time stamp at the end of each target file name in the following format: YYYYMMDD_HHMMSS_millisecond. For example, the Secure Agent renames the target file name in the following format: output.txt-20171220_091900_69844051

Target partitioning

You can configure partitioning to optimize the mapping performance at run time when you write data to a file of flat format type. You can configure target partitioning only on mappings.
The partition type controls how the agent distributes data among partitions at partition points. You can define the partition type as passthrough partitioning. With partitioning, the Secure Agent distributes rows of target data based on the number of threads that you define as partition.
You can configure the Merge Partition Files options in the advanced target properties. You can specify whether the Secure Agent must merge the number of partition files as a single file or maintain separate files based on the number of partitions specified to write data to the Amazon S3 V2 targets.
If you do not select the Merge Partition Files option, separate files are created based on the number of partitions specified. The file name is appended with a number starting from 0 in the following format: <file name>_<number>
For example, the number of threads for the Region.csv file is three. If you do not select the Merge Partition Files option, the Secure Agent writes three separate files in the Amazon S3 V2 target in the following format:
<Region_0>
<Region_1>
<Region_2>
If you configure the Merge Partition Files option, the Secure Agent merges all the partitioned files as a single file and writes the file to Amazon S3 V2 target.

Distribution column

You can write multiple flat files to Amazon S3 target from a single source in a mapping. Configure the Distribution Column option in the advanced target properties.
You can specify one column name in the Distribution Column field to create multiple target files during run time. When you specify the column name, the Secure Agent creates multiple target files in the column based on the column values that you specify in Distribution Column.
Each target file name is appended with the Distribution Column value in the following format:
<Target_fileName>+_+<Distribution column value>+<file extension>
Each target file contains all the columns of the table including the column that you specify in the Distribution Column field.
For example, the name of the target file is Region.csv that contains the values North America and South America. The following target files are created based on the values in the Distribution Column field:
Region_North America.csv
Region_South America.csv
You cannot specify two column names in the Distribution Column field. If you specify a column name that is not present in target field column, the task fails.
When you specify a column that contains value with special characters in the Distribution Column field, the Secure Agent fails to create target file if the corresponding Operating System do not support the special characters.
For example, the Secure Agent fails to create target file if the column contains date value in the following format: YYYY/MM/DD

Writing to multiple target objects

When you import target objects, the Secure Agent appends a FileName field to the imported target object. When you map the FileName field in the target object to an incoming field, the Secure Agent creates the folder structure and the target files based on the FileName field. For example:
Syntax:
<tgt_filename_folder>/<tgt_filename=incoming_value_folder>/part_file
Sample:
emp_tgt.parquet/emp_tgt.parquet=128000/part-0000-e9ca8-6af-efd43-455c-8709.c000.parquet
The FileName field is applicable to the following file formats:
Consider the following guidelines when using the target FileName field in mappings:

Object tag

You can add a tag to the object stored on the Amazon S3 bucket. Each tag contains a key value pair. You can use an object tag for flat files.
Tagging an object helps to categorize the storage. You can add the object tags in the Object Tags field under the advanced target properties. Enter the object tag in the Key=Value format. You can also enter multiple object tags in the following format:
key1=Value1;key2=Value2
You can either enter the key value pairs or the specify the file path that contains the key value pairs. For example, you can specify the file path in the C:\object\tags.txt format. You can specify any file path on which the Secure Agent is installed.
When you upload new objects in the Amazon S3 bucket, you can add tags to the new objects or add tags to the existing objects. If the Secure Agent overrides a file that contains a tag in the Amazon S3 bucket, the tag is not retained. You must add a new tag for the overridden file. If you upload multiple files to the Amazon S3 bucket, each file that you upload must have the same set of tags associated with the multiple objects.
To add tags in the Amazon S3 V2 target object, you must add the s3:PutObjectTagging permission in the Amazon S3 policy. Following is the sample policy:
{
"Version": "2012-10-17",
"Id": "Policy1500966932533",
"Statement": [
{
"Sid": "Stmt1500966903029",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject",
"s3:PutObjectTagging"
],
"Resource": [
"arn:aws:s3:::<bucket_name>/*",
"arn:aws:s3:::<bucket_name>"
]
}
]
}
The following table lists the special characters that Amazon S3 V2 Connector supports during entering the key value pair:
Special Characters
Support
+
Yes
-
Yes
=
No
.
Yes
_
Yes
:
Yes
/
Yes

Rules and guidelines for tagging an object

Use the following rules and guidelines for tagging an object: