Custom Scanner Framework: Overview & Configuration

Version 3

    Custom Scanners: Overview

    Custom scanners will enable the user to create a model for any data source within the catalog. This is possible by creating and registering custom models through an open framework.  Custom models can include classes, attributes and relationships to model any kind of data source. Once the model is created, it is possible to ingest metadata including object names, relationships and attributes through delimited files. All the key functionalities like search, lineage, relationships, custom attributes, Business Glossary Associations etc. are all available for custom objects. In addition, Custom Lineage scanner can be used to add lineage links for custom objects. Custom objects can be linked to any object in the catalog including those that come from native scanners.

    Follow the steps below to create Custom Scanners at a high level:  (for more details, please refer our documentation)


    1. Create a XML file for the custom model
      1. Create model using the XML file in Step-1 above
      2. Use any pre-packaged models as a baseline for model creation
    2. Create Resource type by selecting connection type based on the model created above
    3. Create a Custom resource by uploading the zip files (2 CSV files – objects & links)
    4. Run any dependent resources that need to be linked to the custom resource
    5. Create custom lineage resource to link the objects (please refer the documentation on how to create custom lineage scanner)


    Below is simple structure of a custom model: