7 Replies Latest reply on Dec 22, 2020 12:43 AM by Krishna Gangwar

    REST APIs for Informatica Developer/Analyst

    Eugene Yeo New Member

      Hi,

       

      Are there any REST APIs available to perform the following on Informatica DQ (Developer/Analyst Version 10.2.1 )

      1. Import rule specifications, or view the existing rule specifications. If there are no REST APIs for this, are there any alternative such as importing the rule specification from a xml/json file.
      2. View profile configuration, runs and results.

       

      I could only find documentation on Business Glossary APIs.

       

      Regards,

      Eugene Yeo

        • 1. Re: REST APIs for Informatica Developer/Analyst
          Krishna Gangwar Active Member

          Hi Eugene Yeo,

           

          I would like to address point #1.

           

          There is no REST API to either import or view existing rule specifications in DEQ 10.2.1. However, you can import/export rule specifications either using developer client or infacmd command-line utility.

           

          1) From Developer Client

          When you create a rule specification via Analyst Tool, it can be seen through the developer client as well. To export the rule specification, right-click > export > Informatica > Export Object Metadata File. This will generate an XML file. Which can later be imported via developer client (import not possible through Analyst Tool).

           

          2) From infacmd command-line utility

          You can also export the rule specification via infacmd command line utility. The syntax of the command will be:

          $INFA_HOME/isp/bin/infacmd.sh oie exportObjects -dn <Informatica_domain_name> -un <Informatica_User> -pd <password> -rs <MRS_NAME> -pn <MRS_PROJECT_CONTAINING_RULESPEC> -fp <OUTPUT_XML_NAME> -cp <EXPORT_CONTROL_FILE_PATH>

           

          Where export control file is an XML file that defines what objects need to be exported from MRS project specified under -pn option. If the export control file is not used, all the content in the specified project will be exported. Basically, the export control file is used as a filter.

           

          Refer to below sample export control file for reference:

           

          <?xml version="1.0" encoding="UTF-8" standalone="yes" ?>

          <exportParams xmlns="http://www.informatica.com/oie/exportControl/9">

              <folders>

                  <!-- Consider exporting selected objects in the project. Recursively export from subfolders. -->

                  <folder recursive="true">

                      <!-- Export rule specification if created by the specified user. -->

                      <objectList type="Rule Specification" createdBy="admin">

                          <object name="Test_Rule_spec" />

                      </objectList>

                  </folder>

              </folders>

          </exportParams>

           

           

          refer to the documentation for the same:

          1) https://docs.informatica.com/data-quality-and-governance/data-quality/10-2/command-reference/infacmd-control-files/export-control-files/export-control-file-parameters-for-model-repository-objects/export-control-file-sample-for-model-repository-objects.html

           

          2) https://docs.informatica.com/data-quality-and-governance/data-quality/10-2/command-reference/infacmd-oie-command-reference/exportobjects.html

           

           

          Let me know if this helps.

           

          regards,

          Krishna

          1 of 1 people found this helpful
          • 2. Re: REST APIs for Informatica Developer/Analyst
            Namratha Dhoolipalla Guru

            Hi Eugene,

             

            In addition to Krishna's reply above, I would like to address point #2.

            There are no REST API's to view the Profile Configuration, runs and results.

            But you can make use of the Developer Tool and Analyst Tool to view this information.

             

            Thanks,

            Namratha

            1 of 1 people found this helpful
            • 3. Re: REST APIs for Informatica Developer/Analyst
              Eugene Yeo New Member

              Hi Namratha,

               

               

              The reason I'm looking to see if there are any REST APIs available, is because I'm looking to do something like below:

              1. Create the DQ rules (from an external tool instead of having to use the UI in analyst/developer to create it) and push it into Informatica DQ.

              2. Trigger profiling/scorecard runs programmatically instead of a timed schedule.

              3. Retrieve the profile + scorecard results from Informatica and display it in an external reporting tool.

               

               

              So far from the information that i have received/found:

              1) The rules can be exported/imported to/from an xml file through either Informatica Developer or using the infacmd command line program.

              Provided I understand wholly the rule syntax in the xml file, there isn't a simple way to create a DQ rule programmatically to be imported into Informatica DQ.

               

               

              2) Profiling and scorecard runs can be triggered using a schedule via the scheduler service. Calling it from an external script using the the infacmd ps command is also a possibility.

              https://docs.informatica.com/data-quality-and-governance/data-quality/10-2/command-reference/infacmd-ps-command-reference/executeprofile.html

              There is no REST API available to call to trigger the profiling/scorecard run.

              There is no functionality to detect data changes in the data source and automatically trigger profiling.

               

               

              3) Profile results can be viewed through the analyst or developer tool.

              There also database views available that can expose this information.

               

               

              https://docs.informatica.com/data-quality-and-governance/data-quality/10-2/database-view-reference/model_repository_mrx_views/model_repository_views_overview.html

              https://docs.informatica.com/data-quality-and-governance/data-quality/10-2/database-view-reference/profiling_warehouse_views.html

               

               

              Is there an easy way to restrict the data from the views to only yield data from specific projects? Or i assume i will have to build those queries by doing joins with multiple views.

               

               

              Regards,

              Eugene Yeo

              • 4. Re: REST APIs for Informatica Developer/Analyst
                Namratha Dhoolipalla Guru

                Hi Eugene,

                 

                Your understanding about the Rules in IDQ and Profiles is correct.

                It is recommended to create, edit, import and export rules only through the Developer Tool (create, edit, import, export) or Command line programs (import, export). Using any other text editors will cause the rule xml to be corrupted.

                 

                When you say changes in Data source for Profiles and Scorecards, can you please elaborate the issue further?

                Scorecard keeps track of the changes in the data on which it is running and creates trends for the same.

                Is this what you are looking for: Trend Charts ?

                 

                There are no views to restrict the data to a specific project. You can build queries on top of the views and tables.

                Do not modify the data in the tables from the back end as this can cause issues corruption with the results and the objects.

                 

                Thanks,

                Namratha

                • 5. Re: REST APIs for Informatica Developer/Analyst
                  Eugene Yeo New Member

                  Hi Namratha,

                   

                  To elaborate further on what I meant with regards to this  statement 'There is no functionality to detect data changes in the data source and automatically trigger profiling'.

                   

                  1. Lets say I create a profile on a relational database table (the table contains 100 rows). The profile is then run and the results are generated.

                   

                  2. An ETL/DB job then runs and inserts another 100 rows to the relational database table in (1).

                   

                  3. Rather than having to wait for a timed schedule to trigger the profile run again, is there a way for the data inserts (or updates/deletes of data stored in the table) to be detected, and automatically trigger the profile run for that object so that the latest statistics/results can be generated?

                   

                  Thanks.

                   

                  Regards,

                  Eugene Yeo

                  • 6. Re: REST APIs for Informatica Developer/Analyst
                    Namratha Dhoolipalla Guru

                    Hi Eugene,

                     

                    Yes, your understanding is correct. There is no way for the Profiling to be triggered automatically when the data volume increases at the source.

                     

                    Thanks,

                    Namratha

                    • 7. Re: REST APIs for Informatica Developer/Analyst
                      Krishna Gangwar Active Member

                      Hi Eugene Yeo,

                       

                       

                      There is no option to automatically trigger the profile on a table upon INSERT/UPDATE/DELETE operation as this is databse specific operation. In Informatica IQD, you first import a table as relational data object. Informatica only keeps the metadata in MRS and the latest data is read when running the mapping/data preview.

                       

                       

                      However, you can try creating a workflow which might help you achieving your use case:

                      1) A mapping task that load in to the target table. (Assuming a profile was already created on this table)

                      2) An exclusive gateway to check if the mapping task succeeded. If the mapping task succeeds, run the profile. If not, abort the workflow.

                      3) A command task to run the profile

                       

                       

                      Below is the screenshot for the proposed wf design:

                       

                       

                       

                      This way, you can simulate "automatic profile trigger" upon changes in the table.

                       

                       

                       

                      Let me know if this helps.

                       

                       

                       

                       

                      Regards,

                      Krishna