Hi, I am trying to extract data from SuccessFactor's Odata API which uses the basic authentication. I am able to connect to the source system and was able to extract data but I always get 1000 records when I run the job but the actual count of records in the source was 400K records. I believe that the source's pagination defaults to 1000 per page is the reason whey it fetching us only 1000 records all time. I tried with below Advance Settings in Source which is connected to RESTV2 to enable the source pagination and iterate through all the pages.
Paging Type = Page
Start Page = 1
Page Increment Factor = 1
With the above setting, I was able to run the job successfully but that returned 0 records. I am not sure of what missing here. Appreciate any help in getting this resolved.
Just in case increase the Cache size and see how that works