7 Replies Latest reply on Aug 19, 2019 9:33 PM by Anusha P

    Dynamic mapping with flatfiles

    inuser481045 New Member

      Hi All ,

      We are trying to create a BDM mapping (10.2 version) where different flat file data objects (varying number of columns) need to be parameterized and for each flat file data object the source file name & path needs to be parameterized .

       

      ie we want to "assign  parameters to determine the flat file name and directory of source"  and "assign a parameter to determine the flat file data object" . When we do both together in one mapping,workflow is failing as parameter for flat file name and path are not getting values assigned from parameter file .

       

      Can any one please help  to find a solution?

        • 1. Re: Dynamic mapping with flatfiles
          Tim Dexter New Member

          Did you ever get this working?

          • 2. Re: Dynamic mapping with flatfiles
            inuser481045 New Member

            No.  We ended up hard-coding  file path and file name in all data objects . Seems  parameterizing flat file data object names and the corresponding source directories/names  at the same time is not possible

            • 3. Re: Dynamic mapping with flatfiles
              Tim Dexter New Member

              Seems like a pretty regular use case!

              We have lots of file names that are variations on a theme that need to be processed, moved and loaded to hive.

              I'll keep digging cos I can not believe that the #1 ETL tool can not work on dynamic source file names!

              • 4. Re: Dynamic mapping with flatfiles
                Nico Heinze Guru

                PowerCenter can. The Informatica platform cannot, as far as I know.

                So please don't blame ETL tool #1, it's "only" #2 which obviously is not able to work with dynamic file names.

                 

                Sorry for being so mean.

                 

                I don't know the platform well enough. Would it be possible to issue some pre-session command (that's a standard feature of PowerCenter) to rename the files such that each source file has the same name, then invoke the platform mapping to read and process it, afterwards return the file to its original name, and finally repeat all these steps until all input files have been processed?

                Just an idea. As mentioned, I don't know the platform good enough.

                 

                Regards,

                Nico

                • 5. Re: Dynamic mapping with flatfiles
                  R V Active Member

                  Can you try the below,

                   

                  1) Create an empty FF data object

                  2) In the Run-time: Advanced properties of the FF data object:

                       For Generate Run-time Column Names: select 'From Control File.

                       Parameterize the Control file Name, Control file path, Output file name Output file directory

                        Header Options (under write): Select Output Field Names

                  4) Create a mapping with this (FF) as source and check the option"At runtime, get data object columns from data source

                  5) For the write object, Under Ports - Check Columns defined by: Mapping flow

                  6) Expose all the parameters which have been created at the FF (Physical data object) level as mapping parameters and then execute the mapping with different parameters/control files.

                  • 6. Re: Dynamic mapping with flatfiles
                    user140848 Guru

                    Something else you could try - I have mostly experience with PowerCenter, so I don't know if this will work on the Platform - is to only use the common part of the path as the directory, and use the parameterised part of the path plus the filename as the filename.

                     

                    e.g. instead of directory being /my/infa/directory/$dir_param and file being $filename_param, try setting directory /my/infa/directory and file $subdir_and_filename_param.

                     

                    Regards,

                    Sacha Roscoe

                    • 7. Re: Dynamic mapping with flatfiles
                      Anusha P Active Member

                      Hi,

                       

                      In order to paremeterize the flat file names and source directory at run time, please follow below KB articles:

                      https://kb.informatica.com/howto/6/Pages/20/521213.aspx

                      https://kb.informatica.com/howto/6/Pages/1/145797.aspx