2 Replies Latest reply on Jul 28, 2021 9:03 PM by Jharana Patra

    *********** FATAL ERROR : An unexpected condition occurred in file [/export/home/data/jenkins_workspace/workspace/agent_linux64_Florence_Patch/powrmart/common/utils/blkdesc.cpp] line [1114].  Aborting the DTM process.  Contact Informatica Global Customer

    Praful Bhise New Member

      Hi,

       

      I am trying to parse the json file and write it as parquet stored in ADLS.

      I created Hierarchical schema to read the json contents, but encountered the below error.

      It seems to be some memory issue, but it keeps on coming even increasing the size.

      Any advise?

       

       

       

       

       

       

       

      Task Name: m_json_processor

      Agent Group Id: 01002025000000000016

      Agent Group Name: SAG-SPK-ANL-***

      Agent Id: 01002008000000000012

      Agent Name: SA-SPK-ANL-EMEA

      07/27/2021 12:57:33 **** Importing Connection: Conn_0100200B0000000000BL ...

      07/27/2021 12:57:33 **** Importing Source Definition: onesource##45_OUT##45_dev##45_GBAPTACLUB##45_full_20210705_2312##46_json ...

      07/27/2021 12:57:33 **** Importing Target Definition: json_parquet_output ...

      07/27/2021 12:57:33 **** Importing SessionConfig: default_session_config ...

          <Warning> :  The Error Log DB Connection value should have Relational: as the prefix.

          <Warning> :  Invalid value  for attribute Error Log DB Connection. Will use the default value

          Validating Source Definition  onesource##45_OUT##45_dev##45_GBAPTACLUB##45_full_20210705_2312##46_json...

          Validating Target Definition  json_parquet_output...

      07/27/2021 12:57:33 **** Importing Mapping: Mapping0 ...

      Validating transformations of mapping Mapping0...

      Validating mapping variable(s).

          <Warning> :  Invalid table attribute: Exec SQL

      07/27/2021 12:57:34 **** Importing Workflow: wf_mtt_0100200Z0000000004CF ...

          <Warning> :  Invalid value false for attribute Recursive Directory Read. Will use the default value NO    [Session< s_mtt_0100200Z0000000004CF >  --> adlsgen2_adlsg2 Reader< adlsgen2_adlsg2 Reader > ]

          <Warning> :  Invalid value false for attribute Allow Wildcard Characters. Will use the default value NO    [Session< s_mtt_0100200Z0000000004CF >  --> adlsgen2_adlsg2 Reader< adlsgen2_adlsg2 Reader > ]

          <Warning> :  The value entered is not a valid integer.

          <Warning> :  Invalid value NO for attribute Fail task after wait time. Will use the default value

      [Summary for Application Source Qualifier Source]

           Is a new repartition point.

           Partition type changed from None to Pass Through.

      Successfully extracted session instance [s_mtt_0100200Z0000000004CF].  Starting repository sequence id is [1635523758]

      ======================================================================

      DIRECTOR> VAR_27085 [2021-07-27 12:57:34.397] Parameter file [/opt/infacld/apps/Data_Integration_Server/data/parameters/s_mtt_0100200Z0000000004CF.param] is opened for [session [wf_mtt_0100200Z0000000004CF.s_mtt_0100200Z0000000004CF]].

      DIRECTOR> VAR_27062 [2021-07-27 12:57:34.397] Warning! Cannot find section for worklet [wf_mtt_0100200Z0000000004CF] and folder [] in parameter file [/opt/infacld/apps/Data_Integration_Server/data/parameters/s_mtt_0100200Z0000000004CF.param].

      DIRECTOR> VAR_27028 [2021-07-27 12:57:34.398] Use override value [s_mtt_0100200Z0000000004CF_2_error.csv] for user-defined workflow/worklet variable:[$ErrorFileName].

      DIRECTOR> VAR_27028 [2021-07-27 12:57:34.398] Use override value [1970-01-01] for user-defined workflow/worklet variable:[$LastRunDate].

      DIRECTOR> VAR_27028 [2021-07-27 12:57:34.398] Use override value [1970-01-01 00:00:00] for user-defined workflow/worklet variable:[$LastRunTime].

      DIRECTOR> VAR_27028 [2021-07-27 12:57:34.398] Use override value [s_mtt_0100200Z0000000004CF_7_27_2021_12_57_success.csv] for user-defined workflow/worklet variable:[$SuccessFileName].

      DIRECTOR> TM_6014 [2021-07-27 12:57:34.398] Initializing session [s_mtt_0100200Z0000000004CF] at [Tue Jul 27 12:57:34 2021].

      DIRECTOR> TM_6683 [2021-07-27 12:57:34.398] Repository Name: [XMLRepository]

      DIRECTOR> TM_6684 [2021-07-27 12:57:34.398] Server Name: [rDTM]

      DIRECTOR> TM_6686 [2021-07-27 12:57:34.398] Folder: []

      DIRECTOR> TM_6685 [2021-07-27 12:57:34.398] Workflow: [wf_mtt_0100200Z0000000004CF] Run Instance Name: [] Run Id: [0]

      DIRECTOR> TM_6101 [2021-07-27 12:57:34.398] Mapping name: Mapping0.

      DIRECTOR> TM_6964 [2021-07-27 12:57:34.398] Date format for the Session is [MM/DD/YYYY HH24:MI:SS.US]

      DIRECTOR> TM_6703 [2021-07-27 12:57:34.408] Session [s_mtt_0100200Z0000000004CF] is run by 64-bit Integration Service  [], version [9.1.0 HotFix1], build [0627].

      MANAGER> PETL_24058 [2021-07-27 12:57:34.428] Running Partition Group [1].

      MANAGER> PETL_24000 [2021-07-27 12:57:34.428] Parallel Pipeline Engine initializing.

      MANAGER> PETL_24001 [2021-07-27 12:57:34.429] Parallel Pipeline Engine running.

      MANAGER> PETL_24003 [2021-07-27 12:57:34.429] Initializing session run.

      MAPPING> CMN_1569 [2021-07-27 12:57:34.437] Server Mode: [UNICODE]

      MAPPING> CMN_1570 [2021-07-27 12:57:34.437] Server Code page: [ISO 8859-1 Western European]

      MAPPING> TM_6151 [2021-07-27 12:57:34.437] The session sort order is [Binary].

      MAPPING> TM_6185 [2021-07-27 12:57:34.437] Warning. Code page validation is disabled in this session.

      MAPPING> CMN_65083 [2021-07-27 12:57:34.437] Current Timezone:[UTC +0:0]

      MAPPING> CMN_65085 [2021-07-27 12:57:34.437] Current Process ID:[24979]

      MAPPING> TM_6155 [2021-07-27 12:57:34.437] Using HIGH precision processing.

      MAPPING> TM_6180 [2021-07-27 12:57:34.437] Deadlock retry logic will not be implemented.

      MAPPING> TM_6187 [2021-07-27 12:57:34.437] Session target-based commit interval is [10000].

      MAPPING> PMJVM_42020 [2021-07-27 12:57:34.453] [INFO] Loaded library : /opt/infacld/jdk/jre/lib/amd64/server/libjvm.so.

      MAPPING> PMJVM_42014 [2021-07-27 12:57:34.453] [DEBUG] The value specified for option JVMOption5 is not valid and will be ignored.

      MAPPING> PMJVM_42009 [2021-07-27 12:57:34.621] [INFO] Created Java VM successfully.

      MAPPING> SDKS_38029 [2021-07-27 12:57:34.781] Loaded plug-in 603501: [plugin description].

      MAPPING> SDKS_38024 [2021-07-27 12:57:34.860] Plug-in 603501 initialization complete.

      MAPPING> SDKS_38016 [2021-07-27 12:57:34.860] Reader SDK plug-in intialization complete.

      MAPPING> SDKS_38029 [2021-07-27 12:57:34.868] Loaded plug-in 603501: [plugin description].

      MAPPING> SDKS_38024 [2021-07-27 12:57:36.215] Plug-in 603501 initialization complete.

      MAPPING> SDKS_38017 [2021-07-27 12:57:36.215] Writer SDK plug-in intialization complete.

      MAPPING> CONNECTOR_10000 [2021-07-27 12:57:55.845] [INFO] The source or target object at runtime is overridden with the parameter name google_analytics/json_parquet_output= table name google_analytics/json_parquet_output.

      MAPPING> SDKS_38509 [2021-07-27 12:57:56.367] SDK target and group initialization complete.

      MAPPING> TM_6307 [2021-07-27 12:57:56.367] DTM error log disabled.

      MAPPING> TE_7022 [2021-07-27 12:57:56.589] TShmWriter: Initialized

      MAPPING> pmudt_50096 [2021-07-27 12:57:57.070] [INFO] DT Service Contents were placed in Service DB successfully.

      MAPPING> TM_6007 [2021-07-27 12:57:57.070] DTM initialized successfully for session [s_mtt_0100200Z0000000004CF]

      DIRECTOR> PETL_24033 [2021-07-27 12:57:57.071] All DTM Connection Info: [<NONE>].

      MANAGER> PETL_24004 [2021-07-27 12:57:57.071] Starting pre-session tasks. : (Tue Jul 27 12:57:57 2021)

      MANAGER> PETL_24027 [2021-07-27 12:57:57.077] Pre-session task completed successfully. : (Tue Jul 27 12:57:57 2021)

      DIRECTOR> PETL_24006 [2021-07-27 12:57:57.077] Starting data movement.

      MAPPING> TM_6966 [2021-07-27 12:57:57.078] Reducing estimate of the amount of buffer pool memory required for high performance.

      MAPPING> TM_6660 [2021-07-27 12:57:57.078] Total Buffer Pool size is 83926080 bytes and Block size is 8388608 bytes.

       

       

      ***********

      FATAL ERROR : An unexpected condition occurred in file [/export/home/data/jenkins_workspace/workspace/agent_linux64_Florence_Patch/powrmart/common/utils/blkdesc.cpp] line [1114].  Aborting the DTM process.  Contact Informatica Global Customer Support.

      ***********