6 Replies Latest reply on Aug 5, 2020 11:43 PM by niti rawat

    Data Masking in Data Quality 10.2.0

    niti rawat Seasoned Veteran

      Hi,

       

      We want to mask data captured in reject files (.bad files).

      Do we need to buy a separate license for data masking in Data Quality or how do we ensure if the existing DQ license also has @data masking capabilities?

       

      Regards,

      Niti

        • 1. Re: Data Masking in Data Quality 10.2.0
          user126898 Guru

          By default masking is NOT included in IDQ licenses you would need to purchase.  You may have already have and you can check in the admin console and look at the license options.

           

          thanks,

          Scott

          • 2. Re: Data Masking in Data Quality 10.2.0
            niti rawat Seasoned Veteran

            Hi Scott,

             

            Thanks for confirming that it is to be purchased separately along with the DQ license. When we checked the admin console we can see the license is listed there under general properties of DIS, but there is no field related to data masking. We have raised a case with Informatica asking about the data masking license. I just wanted to see if in the meantime we can find that out on our own with the configuration on admin console.

             

            Regards,

            Niti

            • 3. Re: Data Masking in Data Quality 10.2.0
              Nico Heinze Guru

              May I ask WHY you want to mask data in the bad files? What's the sense of this measure?

               

              If the sense is making it impossible to derive actual data from those files, then there's a much easier alternative:

              Simply set each bad file name to the "null device". On Windows that's <any directory>\NUL , on Unix/Linux it's /dev/null. Setting the bad file to the "bit bucket" essentially makes the data in the .bad files "vanish" completely; they will never be written to the hard disk.

               

              If you really need the data in the .bad files, then of course this approach is not useful for you, no question.

              But if it's only in order to keep prying eyes from looking at these files, writing the .bad data to the "bit bucket" is far easier and cheaper (both in terms of CPU consumption and disk space).

               

              Regards,

              Nico

              • 4. Re: Data Masking in Data Quality 10.2.0
                niti rawat Seasoned Veteran

                Hi Nico,

                 

                We have PII data in .bad files, and as per the business policy our client does not want this data to be readable even in the .bad files. We are capturing some rejections ( not meeting the business rules) in a DB table as well, to be used further for data correction and other purposes, and so the data in .bad files is to be masked. We are trying to convince the client against it cause it them makes the bad file content not-useful.

                 

                Regards,

                Niti

                • 5. Re: Data Masking in Data Quality 10.2.0
                  Nico Heinze Guru

                  Masking .bad files is completely useless and technically extremely nasty (the only way I see is to set up one Named Pipe for each single .bad file, and the "receiving end" of the Named Pipe for each .bad file would be a shell script which masks the data in the respective .bad file, an approach which causes extremely much administrative pain and brings absolutely no advantage which I can think of).

                  Writing the .bad files to /dev/null on Unix/Linux resp. $PMBadFileDir\NUL on Windows makes more sense. This way the .bad files would not even be created at all. Not very helpful in debugging, but at least the data cannot be compromised in any way.

                   

                  Regards,

                  Nico

                  • 6. Re: Data Masking in Data Quality 10.2.0
                    niti rawat Seasoned Veteran

                    Hi Nico,

                     

                    I agree with your what you said. We were also against the masking of .bad files and were finally able to convince the business to not mask .bad files. Instead we applied an EBF to not let the log files show any data in case of a failure and that suffices for now. Thank you for valuable inputs.

                     

                    Regards,

                    Niti