6 Replies Latest reply on May 12, 2020 7:45 AM by Jani Painokallio

    mq.data.change.threads does not work?

    Jani Painokallio Active Member

      Hi, have anybody else experience about the mq.data.change.threads parameter? 570072

      When putting that in place - repackage - and then debug of com.delos.cmx.server.messaging. I only see one thread working on those messages, not 20 as I set on that parameter. Also no performance boost. Is there something else which should be set?

        • 1. Re: mq.data.change.threads does not work?
          Anuvinda Kulkarni Guru

          Hello,

           

          Have you applied all the solutions as suggested in the KB?

           

          Thanks.

          • 2. Re: mq.data.change.threads does not work?
            Jani Painokallio Active Member

            Have tried:

            1) https://kb.informatica.com/faq/4/Pages/142115.aspx and

            2) https://kb.informatica.com/solution/23/Pages/69/570072.aspx

            When enabling debuging, I can see that only one Thread is working with the messages, and clearly see that data used by the messages is returned in 0.004 seconds

            However: This takes 1.562 seconds, and Thread-117 is exactly the one which is taking care of messages

            [2020-05-11 12:18:03,632] [Thread-117] [DEBUG] com.delos.cmx.server.datalayer.access.DataAccessObjectDirectorForDatabaseAdapter: Retrieved 1 com.delos.cmx.server.datalayer.access.ReposSifOrsConfigDataAccessObject: SVR1.43F5J2 1.562

            SVR1.43F5J2 is JMS_EVENT type of Configuration

            select

              generated_object_type

              , length(cast(mapping_file as varchar))

              , length(cast(schema_file as varchar))

              from  MDMORS.C_REPOS_SIF_ORS_CONFIG

            where  ROWID_SIF_ORS_CONFIG = 'SVR1.43F5J2'

            This Query takes 0.005 second, so where it is consuming rest - taking quick beer ?

            Query 1 of 1, Rows read: 1, Elapsed time (seconds) - Total: 0,007, SQL query: 0,005, Reading results: 0,002

            • 3. Re: mq.data.change.threads does not work?
              Jani Painokallio Active Member

              So two questions

              1) Why only one thread is working

              2) What is com.delos.cmx.server.datalayer.access.ReposSifOrsConfigDataAccessObject: SVR1.43F5J2 1.562 which takes 1.562 seconds?

              • 4. Re: mq.data.change.threads does not work?
                Anuvinda Kulkarni Guru

                Can you try adding all these 3 properties in cmxserver.properties? For eg:

                mq.data.change.threads=4

                mq.data.change.event.batch.size=1000

                max.data.change.events.per.poll=500

                • 5. Re: mq.data.change.threads does not work?
                  Jani Painokallio Active Member

                  Let's try these ones:

                  mq.data.change.threads=4

                  mq.data.change.event.batch.size=1000

                  max.data.change.events.per.poll=500

                   

                  These I allready have on my cmxserver.properties file:

                  # Number of threads to use to process JMS messages during the publish process. Default is 1.

                  mq.data.change.threads=50

                  # Number of JMS messages to process in each batch for the publish process. Default is 500.

                  mq.data.change.batch.size=5000

                  # Amount of time in seconds that is allowed to process the JMS messages. Default is 120.

                  mq.data.change.timeout=120

                   

                  Let's try this one

                  • 6. Re: mq.data.change.threads does not work?
                    Jani Painokallio Active Member

                    Interesting, still seeing only one Thread and one worker:

                    [2020-05-12 16:56:13,500] [Thread-116] [DEBUG] com.delos.cmx.server.messaging.DataChangeMessengerSupervisor: Worker http://<ip>:<port>, rowid=SVR1.1I0K, key=<ORS-ID>~SVR1.1I0K successfully pinged.

                    [2020-05-12 16:56:13,500] [Thread-116] [DEBUG] com.delos.cmx.server.messaging.DataChangeMessengerSupervisor: Assigning id=1000 to worker http://<ip>:<port>, rowid=SVR1.1I0K, key=<ORS-ID>~SVR1.1I0K

                     

                    Also I can see some interesting error message at the log, I do not know if this is an issue or not:

                     

                    2020-05-12 17:26:22,501] [Thread-116] [DEBUG] com.delos.cmx.server.messaging.DataChangeMessengerSupervisor: SQL for fetching timed out mq data -> SELECT DISTINCT rowid_job, rowid_object, rowid_mq_rule, change_type, rowid_mq_data_change, SENT_STATE_ID, LAST_UPDATE_DATE FROM C_REPOS_MQ_DATA_CHANGE WHERE SENT_STATE_ID >= 1000 AND SENT_STATE_ID <= 10000 AND ROWNUM <= 5000 ORDER BY LAST_UPDATE_DATE

                    [2020-05-12 17:26:22,537] [Thread-116] [ERROR] com.delos.cmx.server.messaging.DataChangeMessengerSupervisor: Error finding timed out records. sql=SELECT DISTINCT rowid_job, rowid_object, rowid_mq_rule, change_type, rowid_mq_data_change, SENT_STATE_ID, LAST_UPDATE_DATE FROM C_REPOS_MQ_DATA_CHANGE WHERE SENT_STATE_ID >= 1000 AND SENT_STATE_ID <= 10000 AND ROWNUM <= 5000 ORDER BY LAST_UPDATE_DATE

                    com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-206, SQLSTATE=42703, SQLERRMC=ROWNUM, DRIVER=3.69.66

                     

                    Running this SQL manually gives:

                    "ROWNUM" is not valid in the context where it is used.. SQLCODE=-206, SQLSTATE=42703, DRIVER=4.11.77 SQL Code: -206, SQL State: 42703.

                    ROWNUM I think is Oracle syntax, and at least in DB2 you can get similiar result:

                    SELECT DISTINCT rowid_job, rowid_object, rowid_mq_rule, change_type, rowid_mq_data_change, SENT_STATE_ID, LAST_UPDATE_DATE

                      FROM C_REPOS_MQ_DATA_CHANGE

                    WHERE SENT_STATE_ID >= 1000

                       AND SENT_STATE_ID <= 10000

                    ORDER BY LAST_UPDATE_DATE

                    FETCH FIRST 5000 ROWS ONLY