1 Reply Latest reply on Dec 14, 2020 4:23 AM by user126898

    Dynamic mapping from File share ADLS Gen2  to ADLS Gen2 Blob container

    Ravindra M New Member

      We have a requirement from client where we have dynamic flat files in .txt  format and vendors are dropping them into "File share" of Azure ADLS Gen2.

       

      1. Do we have a connection from IICS to connect to File share ?

      Note: We have connection of ADLS Gen2 which is able to connect to Blob container but not file share.

       

      2. As the source files keep changing in schema and data we need to build dynamic mapping which will fetch data from File share of ADLS Gen2 and drop new updated data into ADLS Gen2 Blob container so that Databricks tables can be built on top of it ?

       

      Please let me know if this possible in IICS or even if this feasible in DEI then please send us details.

        • 1. Re: Dynamic mapping from File share ADLS Gen2  to ADLS Gen2 Blob container
          user126898 Guru

          Hello,


          1) There is not a native connection.  But you can mount the file share as a network drive in either windows/linux such that the secure agent can read the files like they were sitting on the local file system which you then could read and write to ADLS.

           

          2) cloud mappings are dynamic by nature.  At run time we grab the structure and process.  You can parameterize the source dir and file name so if it changes no issues.   Since you don't have hard ports and have port rules it is mimicking the dynamic map functionality from DEI.

           

          Everything you want to do is available in CDI.

           

           

          thanks,

          Scott