6 Replies Latest reply on Sep 13, 2017 8:16 AM by Ashok Gulagond

    how to calculate DTM Buffer size

    Somasekhar reddy New Member

      Hi All,

      how to calculate DTM Buffer Size.

       

      Thanks in Advance,

      Soma sekhar reddy

        • 1. how to calculate DTM Buffer size
          Guru

          Please take a look at the Performance Guide for general guidelines. For special questions you can always refer to this forum again, but repeating the basic explanations from the manuals here wouldn't make much sense.

           

          Regards,

          Nico

          • 2. how to calculate DTM Buffer size
            New Member

            Check out in Informatica Support portal. Informatica does provide you a freeware tool which calculates the DTM Buffer Size.

            • 3. how to calculate DTM Buffer size
              New Member

              For example, you create a session that contains a single partition using a mapping that contains 50 sources and

              50 targets. Then, you make the following calculations:

              1. You determine that the session requires a minimum of 200 memory blocks:

              [(total number of sources + total number of targets)* 2] = (session buffer blocks)

              100 * 2 = 200

              2. Based on default settings, you determine that you can change the DTM Buffer Size to 15,000,000, or you can

              change the Default Buffer Block Size to 54,000:

              (session Buffer Blocks) = (.9) * (DTM Buffer Size) / (Default Buffer Block Size) * (number of

              partitions)

              200 = .9 *

              14222222

              / 64000 * 1

              or

              200 = .9 * 12000000 /

              54000

              * 1

              Note:

               

              For a session that contains

              n partitions, set the DTM Buffer Size to at least

              n

              times the value for the session

              with one partition. The Log Manager writes a warning message in the session log if the number of memory blocks

              is so small that it causes performance degradation. The Log Manager writes this warning message even if the

              number of memory blocks is enough for the session to run successfully. The warning message also gives a

              suggestion for the proper value.

              If you modify the DTM Buffer Size, increase the property by multiples of the buffer block size.

              • 4. how to calculate DTM Buffer size
                Imran Khan Active Member

                Hey,

                 

                The easiest way to explain this in my opinion is as such:

                 

                go into the mapping and add up all the values from the precision column from ALL your sources and targets. once you get this number (which is the max amount any one record would require for processing), multiply it by 100. The result will be your DTM buffer size in bytes, optimized for 100 records at a time (this is ideal for processing as per the performance guide that Nico was referencing).

                 

                With this number, set the DTM Buffer in the session with it, and you should be great. Works wonders on performance. I've reduced a run time from 15 minutes to 30 seconds.

                • 5. Re: how to calculate DTM Buffer size
                  sridhar b Seasoned Veteran

                  Hi,

                   

                  Any idea about the total amount of Buffer memory that a host can hold? And what happen if a DTM memory required for a session is more than the total DTM memory available on host?

                   

                  Thanks,

                  sridhar.

                  • 6. Re: how to calculate DTM Buffer size
                    Ashok Gulagond New Member

                    Hi,

                     

                    I too faced the same issue recently , I tried to change buffer size or create new workflow/session, but it didn't work.

                    In my case issue was with source file definition. The workflow was actually ran successfully first time, then later my teammate had changed the precision in file definition. Which was not propagated into other transformations.

                     

                    Propagating the precision into mapping solved the issue. Hope this helps for someone.

                     

                    Thanks,

                    Ashok Gulagond