6 Replies Latest reply on Feb 12, 2019 11:00 AM by Mitesh Joshi

    Thrift File Using Informatica

    Mitesh Joshi New Member

      Hi Team,

       

      Is it possible to create a thrift file using Informatica PowerCenter or Informatica Developer.

       

      We are currently using the version 10.2

       

      Regards,

      Mitesh Joshi

        • 1. Re: Thrift File Using Informatica
          Nico Heinze Guru

          What exactly do you mean? The Thrift definition file of the highest-level layer of an Apache Thrift stack or something else?

          What exactly do you want to do in the end?

           

          Regards,

          Nico

          • 2. Re: Thrift File Using Informatica
            Mitesh Joshi New Member

            Hi Nico,

             

            Thanks for reverting.

             

            I am new to this thrift concept. I am not sure what the highest-level layer means. Could you please explain a bit on it.

             

            Also, how does an end to end process using thrift file looks like in informatica? Could you please provide an example.

             

            Regards,

            Mitesh Joshi

            • 3. Re: Thrift File Using Informatica
              Nico Heinze Guru

              No, for sure I'm the wrong person to do that. I've just read a bit of the Wikipedia page today after you've posted your question.

              The reason why I've asked for the highest-level definition file is that the Wiki article mentions it, and it was the only mentioning of something like a file. That's why I've asked whether you mean this.

               

              As far as my (very thin!) understanding goes, Thrift provides an abstraction layer (an API layer, maybe?) for web-based services. Nothing about files in particular. That's why I've asked. This concept is completely new to me as well.

               

              Regards,

              Nico

              • 4. Re: Thrift File Using Informatica
                Mitesh Joshi New Member

                Hi Nico,

                 

                Thanks for reverting.

                 

                I have figured out what I need, however, not sure how to achieve it.

                 

                I need to create a schema of whatever file I have been provided. Suppose, if there is a csv file as input, the output should be the schema of that input csv file.

                 

                For Example:

                Input CSV is like

                ID,Name,Company,State

                123,ABC,XYZ,DEF

                 

                Output File should look like

                ID string

                Name String

                Company String

                State String

                 

                I am not sure how to achieve it. Could you please guide on it.

                 

                Regards,

                Mitesh Joshi

                • 5. Re: Thrift File Using Informatica
                  Nico Heinze Guru

                  If that's all, then things may be easy:

                  Assuming that each CSV file has a header line, you can just read that CSV file and split up the header line into individual lines, each one naming the name of the respective column in the CSV file and indicating "String" as the data type. That can be fully automated with pretty plain PowerCenter mappings.

                   

                  Of course things may become much more complex. For example, how to find out what data type each CSV column has? In fact the answer is: you cannot. That's information to be delivered by the people providing the CSV file. You should not even try to guess the data types, that's eventually bound to severe errors and problems.

                   

                  Never try to "programmatically" find out data types; this information must be delivered by those people cerating the source files. Period.

                   

                  Regards,

                  Nico

                  • 6. Re: Thrift File Using Informatica
                    Mitesh Joshi New Member

                    Hi Nico,

                     

                    I created a simple mapping which takes the header line from the csv file using a command input to flat file.

                    head -1 temp.csv

                     

                    Once i get the line, i have used Java Transformation to split the comma separated values and created a row for each column. This helps me to input a file with n number of columns.

                     

                    So even if the file has 10 cols or n cols, this mapping will work. As well, the input can be parameterized.

                     

                    As of now, we are not worried about the data type, however, I will think of it as well.

                     

                    Again, thanks for your help for guiding.

                     

                    Regards,

                    Mitesh Joshi

                    1 of 1 people found this helpful