1 2 Previous Next 15 Replies Latest reply on May 31, 2018 1:53 AM by mayur marathe

    Connect Active java transformation

    mayur marathe Seasoned Veteran

      Hi All,

      I need one help , i have mapping like below

       

      Source - SQ - JavaTx1 ( Active) - JavaTX2 (Active) - Target

       

      I need to connect  output of one jtx to another java transformation as input but when i try to connect it gives me error concatenation disallowed on active transformation java1 and java2.

       

      did anybody faced this issue before what solution you have applied to get rid of it i dont want to make 2 seperate workflows

       

      Please help this is kind of urgent

        • 1. Re: Connect Active java transformation
          Nico Heinze Guru

          In principle it is very well possible to do what you describe: feed the output of one JTX into another active JTX and then feed the output of the second one to target(s). No problem at all.

           

          If the Designer tells you that concatenation is disallowed, then there must be something else wrong.

          One common potential mistake is that some ports of JTX1 are forwarded to the target and some of its output ports are forwarded to the active JTX2 and that people are trying to pass output ports from JTX2 to the same target. That can't work (at least not with "simple" relational or flat file targets).

          Is that the case here?

           

          In this case it would be helpful if you could describe:

          - what output each JTX produces,

          - some info about the structure of the target,

          - how output of JTX1 and of JTX2 should populate the target.

          Then we probably will be able to help you.

           

          Regards,

          Nico

          • 2. Re: Connect Active java transformation
            mayur marathe Seasoned Veteran

            Hi All

             

            thanks I have resolved this issue , as we can not connect output of one active java transformation to another active java transformation i have given the output of java transformation to expression txn and then used joiner with the join on the port created in expression transformation

            • 3. Re: Connect Active java transformation
              mayur marathe Seasoned Veteran

              Hi Nico , I have tried to solve this issue with joiner but joiner has limitation of cache it skips the row if we have more number of records , the details are as below of mapping

              we have one configuration table where all the static data is kept like URL and certificate token etc so input to first java txn ill be

              URL (string)

              fromdate (date/time)

              todate(date/time)

              authtoken (string)

              sleep(integer)

              max_retry(integer)

               

              Output of first java txn  - party_id (integer)

               

              This output of 1st txn is fed to another java txn which gets the data according to the that party_id like name of party ,etc

               

              input to second java txn

              Output of first java txn  - party_id (integer)

              URL (string)

              authtoken (string)

              sleep(integer)

              max_retry(integer)

               

              the authtoken ,sleep,max_retry are common for both txn , i have different instance of same source

               

              Output of 2nd java txn

              Party_id  (integer )

              name ( string)

              hashcode (string)

               

              target table details are

               

              Party_id  (number )  15

              name ( varchar2)    1000

              hashcode (varchar2)  4000

              creation _date  timestamp 26

              modified_date  timestamp 26

               

              created_date and modified_date is provided from expression transformation (sysdate)

               

              can you please help to resolve above issue because the joiner option is not working

               

              Thanks

              Mayur

              • 4. Re: Connect Active java transformation
                Nico Heinze Guru

                First the Joiner option does work if used correctly. I've tried that often enough. And it's not a matter of cache size or so.

                Second you can simply pass values through an active JTX without problem, but of course you may just as well set all output ports explicitly if you separate input ports from output ports.

                Could you please post the input-only, output-only, and input-output ports plus Java code snippets of the first JTX?

                 

                Regards,

                Nico

                • 5. Re: Connect Active java transformation
                  mayur marathe Seasoned Veteran

                  Hi Nico , Thanks

                  Joiner works properly  and fetches records but less no of records only i mean it pass less no of records only , i have tested this when i pass the output of first java transformation to dummy target table it give around 25000 records when i join these two data the final output table has only 2000 records. i have created expression txn before join where i created dummy ports and assigned value 1 and then joined those ports in the joiner , with normal join.but the records passed is less and they have unique records.

                  input only port for first java txn is  a

                  URL (string)

                  fromdate (date/time)

                  todate(date/time)

                  authtoken (string)

                  sleep(integer)

                  max_retry(integer)

                  and output port is party_id

                   

                  then for second java txn the input will be (o/p of first java txn)

                   

                  Output of first java txn  - party_id (integer)

                  URL (string)

                  authtoken (string)

                  sleep(integer)

                  max_retry(integer)

                   

                   

                  output will be

                  Party_id  (number )  15

                  name ( varchar2)    1000

                  hashcode (varchar2)  4000

                  creation _date  timestamp 26

                  modified_date  timestamp 26

                   

                   

                  not sure what you exactly mean by "but of course you may just as well set all output ports explicitly if you separate input ports from output ports"

                  I am not able to put java code exactly becuase of the policy.

                   

                  can you please help

                  • 6. Re: Connect Active java transformation
                    Nico Heinze Guru

                    No, I can't. What I need (and in fact everyone would need, e.g. someone from Informatica Global Customer Support will need just as well) is first a screenshot of the mapping (so that we can see what goes where) plus the Java in the JTX. Without these two I doubt that anyone will be able to help you.

                     

                    No offense intended, but for me the information in this post is simply not sufficient to really understand what's going on and what's going wrong. Sorry for that, I'm just bloody honest.

                     

                    Regards,

                    Nico

                    • 7. Re: Connect Active java transformation
                      mayur marathe Seasoned Veteran

                      Hi Nico

                      I have attached the Image file of the mapping in 2 parts , 2 instances of source and  java transformation here in this case i have joined the o/p java transformation by joiner by creating dummy columns in expression transformation and joined that dummy columns (d_port and d_port 1)  in joiner as condition by assigning them value as 1

                      joiner details

                      Normal Join ( tried all the types of join ) by creating master join as the second pipeline has only one row

                      the first pipeline will be output of first java transformation which has many rows but second has one static row need this static row for all the output rows generated by first java transformation e.g

                      say first java txn has given party_id as 1,2,3,4,5... then url created at second java transformation will be like

                      http://api_url?party_id=1

                      http://api_url/?party_id=2

                      http://api_url/?party_id=3

                      http://api_url/?party_id=4

                      output of each url (java txn) is passed to the target.now the issue is when i use joiner then number of rows passed to target is less and i dont find any other way to join of pass the output of one java transformation to second java transformation

                       

                      Please help

                       

                      • 8. Re: Connect Active java transformation
                        Nico Heinze Guru

                        If I understand the mapping right, you have a "dummy join" (in effect a Cartesian Join) between the master stream (one record from the upper SQ) and the detail stream (first JTX). So in case the master SQ delivers 1 record and the JTX delivers 5 records, you will get 5 records from the Joiner. And that's the correct result for a Joiner set up this way.

                        So what do you mean that there are not "enough" records?

                         

                        It might help if you would please describe from the business side what exactly this mapping is supposed to do. Please do NOT write about any transformations, just describe what the mapping shall do, then we can work on some useful implementation, ok?

                         

                        Thanks,

                        Nico

                        • 9. Re: Connect Active java transformation
                          mayur marathe Seasoned Veteran

                          Thanks Nico

                          The requirement is we have 2 different urls from which we get the data first url get the data like party_id for e.g and the second url will take this party_id as input parameter and gets remaining parameters from second url like name,hashcode etc , so i have created 2 separate java transformation first will bring the party_id and another JTX will bring related details to the partyID received by first JTX and i was unable to join the party_id from first JTX to another one because both JTX are active txn so i have used the joiner

                           

                          Issue with the joiner

                          When i run the Workflow the counts which i get in the target table is less, i came to this conclusion because i just fed the output of First JTX(party_id) to one temporary table created and checked in temporary table i received around 45000 unique rows (party_id) but in target table i have received only 2000 rows(details for the particular party_id) , means some rows are skipped in joiner only because there i am joining the 2 pipelines.

                              Firstly i tried with the normal join with joiner cache as Auto but then i tried multiple combination with join type and cache also still not able to figure out.

                           

                          In short i need to connect output of first active JTX to another in same mapping so that REST urls will get all the data without missing any records in single workflow only , is there any way i can achieve this

                           

                          Thanks

                          Mayur

                          • 10. Re: Connect Active java transformation
                            Nico Heinze Guru

                            Of course there is a way: simply set up the second JTX such that it accepts all output ports from the first JTX (no matter whether they are needed or not), and that's it. No need to join anything, you can simply forward the party ID from the first JTX to the second JTX and be done with it.

                             

                            Second there's a very easy way to find out where records go wrong in the mapping:

                            Simply set the Tracing Override property on the Config Object tab of the session to Verbose Initialization; this will make sure that for each and every transformation in the mapping you get the exact numbers of records going into and out of this transformation.

                            This way you can check which transformation processes how many records, and in the end you will find out this way where those 43,000 records seem to "vanish".

                            Then the next step is to find out why these records are "vanishing", and for that there are several possibilities as well (we can discuss these later on).

                             

                            No offense intended, but your description is still too technical. Please let me repeat what I understood.

                            From some source you get some source data.

                            For each source record you are trying to retrieve a party ID using a JTX.

                            Now for each individual party ID (can there be duplicates?) you want to use the second JTX in order to get some additional details for each single party ID.

                            Finally you want to write those records (n records per party ID?) to your target.

                             

                            Is that correct?

                             

                            Regards,

                            Nico

                            • 11. Re: Connect Active java transformation
                              mayur marathe Seasoned Veteran

                              Thanks Nico

                              your understanding is correct , sorry for sharing again technical details we have one configuration table as source where we placed all the static data like url, authentication token etc 2 separate urls for 2 JTX so i have chosen two pipeline from same source.I have updated the Joiner txn for verbose data now to check what exactly happening let me run and inform  you once it runs.

                               

                              i am not clear about below statement

                              simply set up the second JTX such that it accepts all output ports from the first JTX (no matter whether they are needed or not), and that's it.

                               

                              I have 2 different urls from same source table one needed for first TXN and another needed for second TXN , second TXN url needs to be concatenate with the output of first TXN(party_id)

                              e.g

                              http://second url?party_id=1

                              http://second url?party_id=2

                               

                              so how will i make the second TXN to accept output port as output from first txn, because when i tried doing that when i connect o/p first TXN to another it connects but when i try to connect rest parameters from second SQ it throws error

                              please help may be i am missing something here

                              • 12. Re: Connect Active java transformation
                                Nico Heinze Guru

                                You can pass the URL for the second JTX through the first JTX as a simple pass-through port (an input-output port which is not touched anywhere in the Java code). That should work. I've used that often enough in the past.

                                 

                                What do you mean by "when I try to connect REST parameters from second SQ it throws error"? Which transformation throws what error message?

                                Ah, ok, one more look at the mapping, now I hope it's clear to me:

                                 

                                You should first join the data from the two SQs with a Cartesian Join.

                                Then forward these records to the first JTX to get the party ID.

                                Now forward each output record from the first JTX (including the REST URL which is passed through, see above) to the second JTX and use them there.

                                Should be straightforward.

                                 

                                Regards,

                                Nico

                                • 13. Re: Connect Active java transformation
                                  mayur marathe Seasoned Veteran

                                  Thank nico

                                  I have removed joiner TXN from my mapping now with only one source qualifier , i have passed all the ports to first JTX . and also Forwarded the ports which are used only in second JTX as  passed through ports and connected to second JTX with the output of first JTX as one of the input parameter and ran it but till i am getting the full rows i mean full data .

                                   

                                  not sure what is wrong in the mapping

                                   

                                  Thanks

                                  Mayur

                                  • 14. Re: Connect Active java transformation
                                    Nico Heinze Guru

                                    OK, back to the start.

                                     

                                    First please post a screenshot of the mapping as it is now.

                                    Second please set the session log to Tracing Override = Verbose Initialisation, re-run the session, and post the session log here (don't worry, it doesn't contain any data with Verbose Initialisation).

                                    Third what is the problem that you get? Earlier you have written that you only get 2000 out of 45000 expected rows, and now you get too many data records? Or what? Sorry, but I am simply puzzled. Please explain what's the problem.

                                     

                                    Regards,

                                    Nico

                                    1 2 Previous Next