IICS Fall '20 October Release - Potential change in task behavior due to Pushdown Optimization

Version 1

     

    Description of the change

    With the upcoming release of IICS Fall ’20, we have enabled Pushdown Optimization (PDO) for various use cases involving data flows of mainly two categories:

    1. Loads from a cloud storage system to a cloud data warehouse
    2. Loads within a cloud data warehouse.

     

    Steps to take

     

    It is strongly recommended to check this and take actions below prior to the release. You can do this any time before the release.

     

    If you have a mapping for which you have enabled Pushdown Optimization (PDO) in the Mapping Task (see screen shot below), Informatica will now attempt to run it in Pushdown mode. In such cases there is a possibility that the result of the mapping may not match the result you have been observing so far. You can take either of the steps below to avoid any unexpected behavior:

    • If you had intentionally configured this with the expectation of a pushdown behavior you can leave it as-is. However, once your org is updated please review the results a few times to ensure those are as expected.
    • Otherwise, edit the tasks and delete the “Pushdown Optimization” entry from the Advanced Session Properties section on the “Schedule” page of the Mapping Task wizard, and save the task again. You can always reconfigure it back in future if needed.

     

    How to determine if you could be impacted due to this change

    Your tasks are likely to be impacted only if ALL of the conditions below are met:

    1. You have a mapping with a combination of sources and targets listed below in the “Source- Target Combinations” table
    2. The mapping is configured with target as insert. No updates/upserts/deletes.
    3. You are not using any “Variable” fields in any Expression transforms.
    4. You have created one or more Mapping Tasks for any of such mappings
    5. You have configured Advanced Session Properties for these tasks and set the specific Pushdown Optimization property to “Full” within it. Refer to the screen shot.

     

    You can ignore this for the mappings/tasks for which any of the above is not true.

     

    Note that in near future even other actions on targets will be pushed down (updates, upserts, deletes). We suggest reviewing all such mappings regardless of whether point #2 above applies.

     

    Source-Target Combinations

    Mappings that have these combinations are potentially impacted by this change. Not all mappings will be actually impacted but it is advisable to check regardless. If you have a large no of mappings and want to shortlist using a more detailed analysis, please refer to the Pushdown Optimization section of the connector guide to check specific coverage of transformations.

     

    Source Connector used

    Target Connector used

    Amazon S3 V2

    Amazon Redshift V2

    Amazon S3 V2

    Snowflake Cloud Data Warehouse V2

    Snowflake Cloud Data Warehouse V2

    Snowflake Cloud Data Warehouse V2

    Azure Data Lake Store V3

    Azure Data Warehouse V3

    Google Cloud Storage V2

    Google Big Query V2

     

     

    You can ignore this in the conditions below

    1. For all the mappings that have any other connector used than the ones in table above. This includes any mappings using ODBC, whether configured for Pushdown Optimization or not.
    2. All mappings that have update, upsert or delete configured for targets. Note that this can change in near future
    3. All mappings that are run directly from within the mapping canvas, and not using a Mapping Task
    4. All mappings with mapping tasks not configured for Pushdown Optimization
    5. Elastic mappings and Mass Ingestion tasks

     

     

     

    Our recommendation

    If you had configured a task with an intention to run it in Pushdown mode, you might want to leave it unchanged and let it run. However, make sure to validate the behavior as if you are running it for the first time to make sure it’s as per your expectations.

     

    For all other mappings that you don’t want to get pushed down or impacted any other way, remove the entry below.

     

    Sample screen shot: This is in the “Schedule” page of Mapping Task configuration

      

    Screenshot

            

    For more information on the upcoming release please refer:

     

    https://network.informatica.com/docs/DOC-18766

     

     

    Subscribe to Status Updates

     

     

    Please subscribe to status updates for your pod here. If you don’t know your pod, login to IICS and once logged in, check the address bar and you will see the pod name. Subscribe only to the pod you are in, and the services you use, so you don’t receive unnecessary updates