0 Replies Latest reply on Jun 18, 2020 10:56 AM by John Quillinan

    When using the File Resource Type, how do I force the scanner to pick up column names from the first row in a Delimited File?

    John Quillinan Seasoned Veteran

      I have created File resource pointing to a folder of files on a Windows file share (SMB protocol).

       

       

       

      The Metadata scanner picks up the 7 files in my folder, but does not return the column names from the first row in each file.

       

       

      How do I correct this so that EDC pick up the column names in the first row of my delimited file?

       

      I have looked into other options:

      1) Custom Resource Type using Delimited System Model:

      I looked into defining the source metadata using a custom resource type, but SFTP is the only protocol supported for profiling.  My files are on a SMB file share.

      2) Custom Resource Type to bring in Source Metadata, and using IDQ to create a flat file data object using Network Path and perform profiling in Analyst.  Analyst could not find the path.

       

       

      3.  Custom Resource Type to bring in Source Metadata, and using IDQ to create a flat file data object using Browse and Upload and perform profiling in Analyst.  This CON to this approach is that it requires the files to manually uploaded on a regular basis, not a sustainable solution.

       

      Finally, I am not confident that using IDQ to profile the data is an option for bringing in Profiling Metadata.  Here is why.  When I created the IDQ Resource, and select Profiled Schema Connections, I do not see my FAA project from the MRS, nor do I see any of the Profiled Flat File Data Objects.  I do however see the results from my Resource profiling. 

       

      Question 1: Are profiled data objects not associated with Connections (created in the Informatica Administrator console) even an option for an IDQ Resource? 

       

      Question 2: If I was working with Amazon S3 files profiled from a Physical Data Object using an Amazon S3 Connection, would I see these listed in the Profiled Schema Connections for the IDQ Resource?

       

      Main Question:  How do I resolve my original issue with bringing column names using the File resource for a Structured File?