1 Reply Latest reply on Jul 12, 2021 6:06 AM by user126898

    Synapse Pushdown Optimization

    David Cribbs Guru

      I am not really super surprised that I need to ask this question as the documentation on this has been unclear and spotty from the start.

       

      1st let's look at the history.

      In connector V1, the pushdown optimization ability was 'there' but I do not think it worked, though I was told by INFA that it did.  I never noticed any difference in speed.

      In V2 the same behavior was observed.

      Then came the Synapse named connector (V3?) and the same behavior was observed, but at some point there was an update that caused any of our mapping tasks with pushdown as an option to fail, even though the update was supposed to make pushdown possible (wait wasn't it already).

      Then an update later talked a bit more about pushdown optimization, but made the option greyed out.

      Now, the most recent July 2021 major release talks all about Pushdown optimization now being possible (again wasn't it already supposed to be possible), including an email that talks about what connector source to target combinations work with pushdown where V3 Data Warehouse (I still am not sure if the Synapse connector is what is really meant).  I sign in today to check out the new hotness for pushdown optimization and it is still greyed out for the Synapse connector, though the documentation for this specific connector talks all about it.

       

      Can someone spell it out for me?  Is pushdown optimization possible, or what?

       

      I really think that what is going on is that you may have to pay for a pushdown license, but Informatica is a bit shy in telling you that and you have to ask the question, which is a little silly in my opinion.

        • 1. Re: Synapse Pushdown Optimization
          user126898 Guru

          There are two types of pushdown in the cloud world.  Standard PDO and Advance PDO.

           

          Standard PDO is what PowerCenter had.  It is ODBC based and you can do source,target or full.  For Full the data had to be on the same database.

           

          Advance PDO or ecosystem PDO is unique to IICS in that you can do a full PDO but the data resides on different storage structures.  ADLS -> SYNAPSE,  ADLS -> SNOWFLAKE.  This is because we are tapping into the native connectors and not ODBC and since synapse/snowflake/redshift/bigquery have the ability to copy directly from the respective cloud storages.

           

          Now you are absolutely correct you need to pay for PDO in older versions of the cloud license structure since the work was being do by the database and not the secure agent so we needed to meter the number of records you processed. 

           

          In the new IPU based model you dont license it directly since we charge based on the consumption rate you use.

           

          A simple shipping request to GCS to request the feature will turn it on or they will tell you based on the existing license if you need to license it.

           

          Thanks,

          Scott