-
1. Re: Lookup Data cache size setting(Manual)
Paolo Moretti Nov 8, 2019 6:24 AM (in response to Sourav Saha)Those values are the minimum amounts allocated by the Integration Service for each transformation in a session whose cache is set to "Auto", no matter the values defined for "Maximum Memory Allowed for Auto Memory Attributes" and "Maximum Percentage of Total Memory Allowed for Auto Memory Attributes".
You can set the above "Maximums" to a higher value in order to get more physical memory allocated for each transformation. By reviewing the session logs, you can fine tune those Maximums.
Every additional memory need that goes above that will be overflowed to a cache file on disk.
If you know how much you want to allocate beforehand (e.g. by means of "Cache Calculator" or by reviewing past logs), you can define a specific values for Data/Index cache size at the transformation level. This will ensure such amount is allocated in memory (i.e. "Auto" configuration at the session level won't apply for that specific transformation).
-
2. Re: Lookup Data cache size setting(Manual)
Sourav Saha Nov 11, 2019 12:00 AM (in response to Paolo Moretti)Hi Paolo,
1. Can you tell me as to what might be the impact if I set the cache size manually? As the disk will have a given amount of space, also on the same disk many other loads will be running, so will the manual setting have any impact on other loads running under the same Integration Service?
2. Or any other possible impacts I might see on manually increasing the cache size, that you know of?
3. If we increase these two settings, "Maximum Memory Allowed for Auto Memory Attributes" and "Maximum Percentage of Total Memory Allowed for Auto Memory Attributes" , to a higher value, what is its physical impact on disk and what is its relation with the lookup data cache size?
Please suggest on the above.
-
3. Re: Lookup Data cache size setting(Manual)
Paolo Moretti Nov 11, 2019 3:29 AM (in response to Sourav Saha)The effects on disk are difficult to foresee. Unless you know beforehand the cache is definitely going to disk (e.g. persistent lookup), in most of the cases you get on disk only what cannot be allocated on memory.
From my point of view the best approach is to leave everything set to Auto and configure the two maximums depending on how much memory is available on your system and how many jobs you run at the same time.
Then you start reviewing the session logs to check whether a transformation's cache needs to be tuned. The session logs give you information about the cache's size, how much you need to keep all the cache in memory, and how busy a transformation was.
If you reckon that a specific specific transformation needs to be tuned independently of the other transformations in the same session, then you start thinking about configuring a specific value for its cache (i.e. you can use the cache calculator).
With regards to "sizes", you are the only one who knows how much cache a transformation needs. No matter whether it is set to "Auto" or not, the main point is that what cannot be allocated in memory is overflowed to disk.
If a cache is not persistent, then the cache files are usually deleted from disk once the session terminates.
-
4. Re: Lookup Data cache size setting(Manual)
Paolo Moretti Nov 14, 2019 8:32 AM (in response to Sourav Saha)Hi Sourav. Please, consider marking the provided answers as helpful or solution, depending on whether they shed some light or answered your original query. This will give a feedback not only to those who are taking the time to read/answer questions in the Communities, but also to those who might go through this thread in the future. Thanks