10 Replies Latest reply on Aug 31, 2018 8:50 AM by Rajan Rath

    How to configure RabbitMQ in informatica powercenter and what are all the softwares required

    Sekhar B New Member

      How to configure RabbitMQ in informatica powercenter and what are all the softwares required

        • 1. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
          mark reedman New Member

          HI Sekhar,

          Did you ever get an answer to this question?

          Thanks

          Mark

          • 2. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
            Rajan Rath Guru

            The complete details on configuring RabbitMQ with Informatica is documented in the below article.

            http://kb.informatica.com/howto/6/Pages/17/333913.aspx

             

            However, kindly be informed that RabbitMQ is not supported by Informatica which means the specific application is not tested and certified; which might (not) work.

             

            Regards,

            Rajan

            • 3. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
              mark reedman New Member

              Hi Rajan,

               

              I've followed the link you gave before and could not get it to work.  I have a test environment setup and I'm able to connect to RabbitMQ with my Informatica workflow.  I can see that it's an active connection but as soon as I publish a message I get various errors.  My assumption is configuration issues but the article doesn't provide enough details for testing this and the debug is pretty vague.  Really struggling with this.

               

              Thanks

              Mark

              • 4. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
                Rajan Rath Guru

                There could be certainly configuration issues as the steps mentioned in the article has worked earlier with Informatica PowerCenter.

                You may use the JMSConnectTest tool to test the JMS connectivity to the queue and confirm if the configuration is correct or if that requires any changes. Only when the connection works at the connection test tool, you can proceed to Informatica.

                 

                Regards,
                Rajan

                • 5. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
                  mark reedman New Member

                  Hi again,

                   

                  I've used the JMSConnectTest and it works fine, returning the desired results. 

                   

                  C:\JMSConnect>java -jar JMSCOnnectTest.jar -c com.sun.jndi.fscontext.RefFSContextFactory -p file:Z:\InfraSRV\jndi -j ConnectionFactory -d MyQueue -ju guest -jx guest -t 2

                   

                  Loading jar files:

                  C:\JMSConnect\javalib\adapter-common.jar

                  C:\JMSConnect\javalib\adapter-imf-metadata.jar

                  C:\JMSConnect\javalib\adapter-interface-pvt.jar

                  C:\JMSConnect\javalib\adapter-interface.jar

                  C:\JMSConnect\javalib\amqp-client-3.6.2.jar

                  C:\JMSConnect\javalib\amqp-client-5.3.0.jar

                  C:\JMSConnect\javalib\ant-weblogic.jar

                  C:\JMSConnect\javalib\basemodel-core.jar

                  C:\JMSConnect\javalib\basemodel-expression.jar

                  C:\JMSConnect\javalib\basemodel-mapping.jar

                  C:\JMSConnect\javalib\cmnframeworks-com.infa.products.frameworks.exceptionframew

                  ork.jar

                  C:\JMSConnect\javalib\collections-10.1.0.33.364-SNAPSHOT.jar

                  C:\JMSConnect\javalib\com.infa.products.imf.runtime-10.1.0.35.364-SNAPSHOT.jar

                  C:\JMSConnect\javalib\com.infa.products.imf.utils-10.1.0.35.364-SNAPSHOT.jar

                  C:\JMSConnect\javalib\commons-cli-1.1.jar

                  C:\JMSConnect\javalib\commons-codec.jar

                  C:\JMSConnect\javalib\commons-httpclient.jar

                  C:\JMSConnect\javalib\commons-io-1.2.jar

                  C:\JMSConnect\javalib\commons-lang-2.4.jar

                  C:\JMSConnect\javalib\commons-logging.jar

                  C:\JMSConnect\javalib\decodeDTService.jar

                  C:\JMSConnect\javalib\derby.jar

                  C:\JMSConnect\javalib\dq-classifier-common-ct_impl.jar

                  C:\JMSConnect\javalib\dq-model-classifier-common.jar

                  C:\JMSConnect\javalib\dq-model-classifier.jar

                  C:\JMSConnect\javalib\dq-model-content.jar

                  C:\JMSConnect\javalib\dq-model-nlp-common.jar

                  C:\JMSConnect\javalib\dq-model-nlp.jar

                  C:\JMSConnect\javalib\dq-nlp-common-ct_impl.jar

                  C:\JMSConnect\javalib\dqstrategy.jar

                  C:\JMSConnect\javalib\exceptionframework-10.1.0.25.364-SNAPSHOT.jar

                  C:\JMSConnect\javalib\exceptionframework-63.jar

                  C:\JMSConnect\javalib\filetransferUtil.jar

                  C:\JMSConnect\javalib\fscontext.jar

                  C:\JMSConnect\javalib\geronimo-jms_1.1_spec-1.1.1.jar

                  C:\JMSConnect\javalib\imf-utils-63.jar

                  C:\JMSConnect\javalib\jlmapi.jar

                  C:\JMSConnect\javalib\jms.jar

                  C:\JMSConnect\javalib\jndi.jar

                  C:\JMSConnect\javalib\Mallet-2.0.7.jar

                  C:\JMSConnect\javalib\pmclassifier.jar

                  C:\JMSConnect\javalib\pmgenericserver.jar

                  C:\JMSConnect\javalib\pmimghierbtree.jar

                  C:\JMSConnect\javalib\pmjmsplugin.jar

                  C:\JMSConnect\javalib\pmjtx.jar

                  C:\JMSConnect\javalib\pmnlp.jar

                  C:\JMSConnect\javalib\pmserversdk.jar

                  C:\JMSConnect\javalib\pmsfdcserver.jar

                  C:\JMSConnect\javalib\pmwebm.jar

                  C:\JMSConnect\javalib\providerutil.jar

                  C:\JMSConnect\javalib\rabbitmq-client-tests.jar

                  C:\JMSConnect\javalib\rabbitmq-client.jar

                  C:\JMSConnect\javalib\rabbitmq-jms-1.4.7.jar

                  C:\JMSConnect\javalib\runtime-10.1.0.35.364-SNAPSHOT.jar

                  C:\JMSConnect\javalib\runtime-63.jar

                  C:\JMSConnect\javalib\slf4j-api-1.7.12.jar

                  C:\JMSConnect\javalib\slf4j-api-1.8.0-beta2.jar

                  C:\JMSConnect\javalib\slf4j-nop-1.7.25.jar

                  C:\JMSConnect\javalib\StanfordNER-3.4.jar

                  C:\JMSConnect\javalib\Trove-2.0.2.jar

                  C:\JMSConnect\javalib\xercesImpl-2.9.1.jar

                  C:\JMSConnect\javalib\xml-apis-2.0.2.jar

                  Loading completed.

                   

                  A numeric value must be set for JMSConnectTest -n

                  The value of -n is set to 1

                   

                  A numeric value must be set for JMSConnectTest -s

                  The value of -s is set to 1000

                   

                  Receiving message: NULL

                  Received 1 message(s).

                   

                  The issue I think is to do with the JMS message being sent from RabbitMQ.  I don't think it is using the JMS Topic Exchange correctly and the message format is returning in a non-JMS structure.  I'm guessing of course.

                   

                  Thanks

                  Mark

                  • 6. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
                    Rajan Rath Guru

                    Hello Mark,

                     

                    Do you receive any specific error when using the same with Informatica i.e. in the session?

                    Are all the libraries (JAR files) from "C:\JMSConnect\javalib" added to the CLASSPATH environment variable or copied to the $INFA_HOME/server/bin/javalib directory?

                     

                    If the connection from Informatica to the JMS server succeeds and the error/issue is observed when reading the data from the source, then your guess could be correct and the issue could be with the message in the queue or the format in which the message is present/loaded at the queue.

                     

                    Regards,

                    Rajan

                    • 7. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
                      mark reedman New Member

                      Thanks for replying.

                       

                      I'm pretty new to this and it's a lot of black box stuff - meaning it's difficult to debug. 

                       

                      I've copied all the relevant Jar files into the $INFA_HOME/server/bin/javalib directory.  If I set the source to a TextMessage then I get the following:

                       

                      Severity Timestamp Node Thread Message Code Message

                      INFO 8/9/2018 12:31:24 PM node01 WRITER_1_*_1 WRT_8226 Target Load Order Group [1] is set for real-time flushing. Target based commit is switched to source based commit and target based commit interval is used as source based commit interval.

                      ERROR 8/9/2018 12:31:25 PM node01 READER_1_1_1 JMS_2028 [ERROR] The message received by Source Qualifier [SQ_MyQueue_To_Oracle] does not match the body definition.

                      ERROR 8/9/2018 12:31:25 PM node01 READER_1_1_1 JMS_2091 [ERROR] The mismatched body type row is : [com.rabbitmq.jms.client.message.RMQBytesMessage@1f]

                      ERROR 8/9/2018 12:32:04 PM node01 READER_1_1_1 JMS_2028 [ERROR] The message received by Source Qualifier [SQ_MyQueue_To_Oracle] does not match the body definition.

                      ERROR 8/9/2018 12:32:04 PM node01 READER_1_1_1 JMS_2091 [ERROR] The mismatched body type row is : [com.rabbitmq.jms.client.message.RMQBytesMessage@1f]

                       

                      This makes sense that it needs to be BytesMessage.  However, when I set it to BytesMessage I get the following message:

                       

                      Severity Timestamp Node Thread Message Code Message

                      ERROR 8/13/2018 2:05:39 PM node01 READER_1_1_1 JAVA PLUGIN_1762 [ERROR] java.lang.NullPointerException

                      ERROR 8/13/2018 2:05:39 PM node01 READER_1_1_1 JAVA PLUGIN_1762 [ERROR] at com.informatica.powerconnect.jms.server.reader.JMSReaderPartitionDriver.onMessage(JMSReaderPartitionDriver.java:2086)

                      ERROR 8/13/2018 2:05:39 PM node01 READER_1_1_1 JAVA PLUGIN_1762 [ERROR] at com.informatica.powerconnect.jms.server.reader.JMSReaderPartitionDriver.run(JMSReaderPartitionDriver.java:1752)

                      ERROR 8/13/2018 2:05:39 PM node01 READER_1_1_1 JMS_2050 [ERROR] Source Qualifier [SQ_MyQueue_To_Oracle] encountered an error while receiving the JMS message. Reason: [null].

                      ERROR 8/13/2018 2:05:39 PM node01 READER_1_1_1 JAVA PLUGIN_1762 [ERROR] Source Qualifier [SQ_MyQueue_To_Oracle] encountered an error while receiving the JMS message. Reason: [null].

                      ERROR 8/13/2018 2:05:39 PM node01 READER_1_1_1 JAVA PLUGIN_1762 [ERROR] at com.informatica.powerconnect.jms.server.reader.JMSReaderPartitionDriver.run(JMSReaderPartitionDriver.java:1858)

                      ERROR 8/13/2018 2:05:39 PM node01 READER_1_1_1 SDKS_38200 Partition-level [SQ_MyQueue_To_Oracle]: Plug-in #300800 failed in run().

                      INFO 8/13/2018 2:05:39 PM node01 READER_1_1_1 CMN_1053 SQ_MyQueue_To_Oracle:Exchange: Control=[EOF]

                      INFO 8/13/2018 2:05:39 PM node01 TRANSF_1_1_1 CMN_1053 SQ_MyQueue_To_Oracle:Leaf Exchange Output : Control=[EOF]

                       

                      My .bindings file looks like this:

                      # -----.bindings file------------------------------------------------------------------

                      ConnectionFactory/ClassName=javax.jms.QueueConnectionFactory

                      ConnectionFactory/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory

                       

                      ConnectionFactory/RefAddr/0/Content=jms/ConnectionFactory

                      ConnectionFactory/RefAddr/0/Type=name

                      ConnectionFactory/RefAddr/0/Encoding=String

                       

                      ConnectionFactory/RefAddr/1/Content=javax.jms.ConnectionFactory

                      ConnectionFactory/RefAddr/1/Type=type

                      ConnectionFactory/RefAddr/1/Encoding=String

                       

                      ConnectionFactory/RefAddr/2/Content=com.rabbitmq.jms.admin.RMQObjectFactory

                      ConnectionFactory/RefAddr/2/Type=factory

                      ConnectionFactory/RefAddr/2/Encoding=String

                       

                      # Change this line accordingly if the broker is not at localhost

                      ConnectionFactory/RefAddr/3/Content=localhost

                      ConnectionFactory/RefAddr/3/Type=host

                      ConnectionFactory/RefAddr/3/Encoding=String

                       

                      #myQueue/ClassName=com.rabbitmq.jms.admin.RMQDestination

                      myQueue/ClassName=javax.jms.Queue

                      myQueue/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory

                       

                      myQueue/RefAddr/0/Content=jms/Queue

                      myQueue/RefAddr/0/Type=name

                      myQueue/RefAddr/0/Encoding=String

                       

                      myQueue/RefAddr/1/Content=javax.jms.Queue

                      myQueue/RefAddr/1/Type=type

                      myQueue/RefAddr/1/Encoding=String

                       

                      myQueue/RefAddr/2/Content=com.rabbitmq.jms.admin.RMQObjectFactory

                      myQueue/RefAddr/2/Type=factory

                      myQueue/RefAddr/2/Encoding=String

                       

                      myQueue/RefAddr/3/Content=MyQ

                      myQueue/RefAddr/3/Type=destinationName

                      myQueue/RefAddr/3/Encoding=String

                       

                      myQueue/RefAddr/4/Content=true

                      myQueue/RefAddr/4/Type=amqp

                      myQueue/RefAddr/4/Encoding=String

                       

                      myQueue/RefAddr/5/Content=MyQ

                      myQueue/RefAddr/5/Type=amqpQueueName

                      myQueue/RefAddr/5/Encoding=String

                       

                      myQueue/RefAddr/6/Content=jms.durable.queues

                      myQueue/RefAddr/6/Type=amqpExchangeName

                      myQueue/RefAddr/6/Encoding=String

                       

                      myQueue/RefAddr/7/Content=MyQ

                      myQueue/RefAddr/7/Type=amqpRoutingKey

                      myQueue/RefAddr/7/Encoding=String

                       

                      Any help would be appreciated?

                      Thanks

                      Mark

                      • 8. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
                        Rajan Rath Guru

                        Hello Mark,

                         

                        The analysis you've performed is pretty good. It is certain that the message type at the source is different than the one set at Informatica which returns the first error.

                        For the second error received, I presume that the source datatype is changed to ByteMessage. However, is the datatype at the source qualifier also changed to that of ByteMessage? Could you try removing the source and source qualifier and recreate the mapping to see if that helps?

                         

                        It is also certain that the connectivity to the JMS source works perfect and there are no issues with it. The issues are mostly either with the message type (text,byte etc) or the libraries used to fetch the data from the source.

                         

                        Regards,

                        Rajan

                        • 9. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
                          mark reedman New Member

                          Morning Rajan,

                           

                          I have no control of the source definition.  I post a payload - a simple message.  I'm not sure what you mean by "Could you try removing the source and source qualifier and recreate the mapping to see if that helps?" Do you mean remove and recreate?

                           

                          I've changed the  Body to be BytesMesage which is all I have control over:

                           

                          This sits in a very simple mapping that pushes to a raw column in an Oracle database:

                           

                          I have used Textmessage as I mentioned but that indicated it should be BytesMessage.

                           

                          Thanks

                          Mark

                          • 10. Re: How to configure RabbitMQ in informatica powercenter and what are all the softwares required
                            Rajan Rath Guru

                            Mark,

                             

                            Yes, can you try deleting the source and source qualifier transformation completely from the mapping and create it again using the JMS source?

                            Also does the JMS source have any message header? If so, you will need to create the message headers at the source.

                             

                            Regards,

                            Rajan