2 Replies Latest reply on Aug 2, 2021 4:07 AM by Darren Wrigley

    Metadata Lost from Datacatalog

    Bharat Rallapati Active Member

      Hi Team,

       

      I have scenario like " we are scanning the metadata from snowflake on Daily basis (scheduled) and now the problem is some times due to improper authorization we lost some metadata. After revalidation got the proper authorizations and re scanned the metadata and everything is fine.

       

      Could you please suggest any other work around for this scenario other than maintaining  the data catalog backup, because it is not simple task to take Data Catalog backup everyday.

       

      Note : So far it's mandator to scan the metadata daily.

       

      Thanks,

      Bharat.

        • 1. Re: Metadata Lost from Datacatalog
          Mayank Raoka Seasoned Veteran

          Hi Bharat,

           

          If you want to scan the metadata daily,its better to use it from admin credentials or the authorization having access to all data. So that no loss of metadata occurs due to such kind of issues.

           

          Because the tool is designed in a way that every time it runs it deletes and re-fetch the data again.

           

          Regards,

          Mayank

          1 of 1 people found this helpful
          • 2. Re: Metadata Lost from Datacatalog
            Darren Wrigley Guru

            right, use a service account to execute the metadata scan & you can use a different account if needed for profiling.

             

            as a correction to the previous answer - the scanner will not delete and re-fetch data again, it will only delete objects that are no longer valid.  the staging process is used to identify objects which are added/updated/deleted

            1 of 1 people found this helpful