3 Replies Latest reply on May 14, 2020 2:23 AM by Utkarsh Pandey

    Trace the reason behind EDC job failure

    Bhupendra Baraiya Seasoned Veteran

      I am monitoring EDC job execution using Rest APIs.

       

      API response just says failed and gives a link to logs.

       

      In Logs I couldn't find what is the error why it failed.

       

      Kindly let me know how can I see what is the exact error either using API or how should I search for an error in logs (using keywords like error, failed or exception doesn't help)

        • 1. Re: Trace the reason behind EDC job failure
          Utkarsh Pandey Active Member

          Hi Bhupendra ,

           

          While clicking on the log location you need to look for the error in stdout > click here full logs .

           

          Also you can get the aggregated logs of a particular resource using yarn command as below :-

           

          yarn logs -applicationId [ YARN application ID] -appOwner [User] > /tmp/[Application ID].log

           

          ( You can get applicationID for scanner application from YARN Resource Manager UI)

                                        

          Regards,

          Utkarsh

          • 2. Re: Trace the reason behind EDC job failure
            Likhitha Jami New Member

            If possible, please get the complete scanner log and check for the following keywords for finding the reason for failure.

            1. If Metadata Load was enabled, then search for keywords like "Import completed successfully" and "Export completed successfully". It helps us to know whether the metadata load was successful or not.

            2. If Data Discovery was enabled, then search for keyword like "validation" and check 'Profile validation summary' in the log.

             

            If you could not find any of the above keywords/reason for failure, then please rerun the scanner from the LDM Admin with JVM option: -Dscannerloglevel=DEBUG

             

            Note:

            To get the Scanner log, please execute below command on Hadoop node of External/Embedded cluster.

            Command: yarn logs -applicationId <applicationId> >> Full_Scanner_Log.txt

            • 3. Re: Trace the reason behind EDC job failure
              Venkatesh Srinivasan Active Member

              Hi Bhupendra,

               

              Without looking at the log we cannot tell you the reason for failure.

              A scanner can fail due to multiple reasons during different stages. Most of the times you should be able to capture the errors with keywords like 'errors', 'failure', or 'exception' etc.

               

              As the log does not contain such keywords, I strongly recommend you to open a case with Informatica Technical support team with the complete scanner log to analyze further on this issue.

               

              You can get the aggregated scanner log of the problematic resource by running the following command from the hadoop gateway node.

              "yarn logs -applicationId <applicationID> -appOwner <ldmappowner>"

              Example-  yarn logs -applicationId application_1539074599894_0079 > logs.txt

              You can get the application ID from the resource manager UI.

               

              As an alternate method, you can collect the scanner log by logging into LdmAdmin >open  problematic resource > 'Monitoring' tab > click on the Log URL > Under log type 'stdout', select the hyperlink that says "Click here for the full log."

               

              Thanks!