3 Replies Latest reply on Nov 29, 2019 10:22 AM by Srinivas Pai

    how to use regex in filtering directories in resource settings

    US ITS DATA CATALOG ADMINISTRATION New Member

      Hi,

       

      We have a resource created to filter all the files in path below.

      dtus/core/masterdata/costcenter

      There are 2 subfolders under 'costcenter' folder. One folder is named 'csv' and other folder named 'parquet'.

      In resource, under metadata load settings, I have an option to select first level directory as 'csv' and only files under that folder will be scanned.

       

      My goal is to include csv folder by using regex patterns.

       

      I have provided (csv) as regular expression in 'first level directory' selection. This regular expression will select strings whose name have 'csv' in them. However, this did not work and that folder is not included for scanning.

       

      Can some one please let me know how these regex section will work to filter directories?

      Also, can we filter the files in a folder using regular expressions or the filter happens only on directory names?