Enterprise Data Catalog: REST API

Version 3

    Representational State Transfer (REST) is a common framework to communicate between computers via internet. We can use a REST Web Service to interact with Web resources and perform updates on Web resources.REST APIs can be used in EDC to extract information from models and objects and update models and objects. The APIs expose the Model that defines the structure for the metadata that the scanner ingests in the catalog. It also exposes Data that represents the metadata ingested in the catalog.

     

    Open Platform

    No metadata lock-in; any metadata can be ingested and accessed from the repository

    Automated Curation

    Programmatic Curation of data assets to deal with metadata at scale

    Pervasive data Governance

    Analytics on Metadata Repository.

    Integrate with third party applications search, lineage and asset relationship services

     

    REST APIs for directly accessing metadata services to create custom analytics and metadata powered applications.

     

    REST API include the following functionalities

     

    REST API use cases

    Access

    • Search metadata
    • Read Objects and Relationships
    • Read System and Custom Attributes

    Annotations

    • Create and populate Custom Attributes on resources/assets.
    • Axon associations with technical assets.

     

    To access REST API, use the following URL to view the list of REST APIs exposed: <cataloghost>:<port>/access/swagger-ui.html#/

    • Use the following URL format to call the REST APIs: <cataloghost>:<port>/access/2/catalog/models/<REST API>
    • JAVA Client and Source are available here:
      • Java client: <cataloghost>:<port>/access/2/files/client.jar
      • Source files and Javadocs. <cataloghost>:<port>/access/2/files/client-sources.jar

     

     

    Common usage

     

    Extract all object with custom attributes X = Y

    • First step is to retrieve the custom attribute internal ID
      • Search for all CAs using:
        http://<server>:<port>/access/2/catalog/models/attributes?pageSize=<largevalue>
      • Iterate on each attribute in the response until finding the one you are looking for by comparing the name, find the attribute ID : Xid
      • Search for object with customer attribute value using:

        http://<server>:<port>/access/2/catalog/data/objects?q=${Xid}%3A%22${Y}%22

     

     

    More Information

     

    Documentation

     

    more documentation can be found here

     

    Java Client example

     

    REST API program examples can be found here. in these examples you can find:

    • FuzzyBGAssociater: This program uses the EIC REST API to match technical metadata (Column Names) against similar BG Terms. By using the Fuzzy Name Matching library, this program makes it easy for users to initialize BG term associations against large datasets.
    • BGAssociationReport: This program uses the EIC REST API to generate a coverage report of BG terms against specified resources. Using this program, data stewards can quickly get a report on # of columns not associated with BG terms yet.
    • BulkClassifier: This program uses the EIC REST API to add values to custom attributes in data assets.
    • UnrulyLinker: This program uses the EIC REST API to add lineage links between tables of two resources having same names.
    • UltimateColumnLineageReport: this program retrieves the relational columns which have lineage information along with the origin column of the object.
    • CustomAttributeValuesCopier: this program copies a custom attribute value to another custom attribute value for each Columns, Tables and resources
    • ObjectFilteredByCustomAttributeValueReport: this program extract all the objects filtered with a specified custom attribute value