4 Replies Latest reply on Oct 30, 2018 2:28 AM by inuser474494

    How to setup a file monitor to poll the files in a order

    Guru

      Hi All,

       

      Can some one help me, how to setup a file monitor to poll the files in a order i.e either ascending or descending order based on file date or file modified.

       

       

       

      Thanks,

      Seshu

        • 1. Re: How to setup a file monitor to poll the files in a order
          Prakash Jain Guru

          HI Seshu,

           

          Since it is a polling, the sequence of the file to be read is not guaranteed. If you can perform some pre-processing and then place the file into the folder would be the approach to go with.

           

          I would also like to understand your entire use case in-order to provide any alternatives if possible.

           

          Thanks,

          Prakash Jain

          • 2. Re: How to setup a file monitor to poll the files in a order
            Guru

            Thanks prakash for the response.

             

            My complete use case is,

             

            one of my application will generate file(s) in a particular directory.  When ever that file(s) has been generated, my icrt process need to poll that file and need to process to another application.

             

            If i received single file, there is issue with the file monitor, but if i received multiple files in that folder, my process should process those files in the order, i.e first in first poll.

             

             

            thanks,

             

            seshu 

            • 3. Re: How to setup a file monitor to poll the files in a order
              Uma Ashok Guru

              H Seshu,

               

              this sounds similar to the post: ICRT File Connector - multiple files processing order

               

              Another possible option as proposed by one of our in-house experts is:

               

              There is a possibility to set the "Other Attributes” field in the File source . The provided value should look like this: eagerMaxMessagesPerPoll=false&sortBy=file:name

               

              Some more details can be found herehttp://camel.apache.org/file2.html in the “Sorting using sortBy” section.

               

              These 2 settings basically instruct the consumer to:

              1. list all the files in the folder;
              2. then sort them by name;
              3. then pick a subset of “Max Messages Per Poll” files for processing in the current poll

               

              Although this would work for a shorter set of file, but if you have many, this might slow down consumer as it will attempt to sort them during each folder poll.

               

              Also if we have a group of files that are copied to the monitored folder it is possible that we start processing already copied files while others have not been copied yet. In this case, the order will be broken simply because when file consumer polls the target folder there is only a subset of files and it simply sorts files it found. In order to avoid this user should make sure that files appear in the right order, especially if they are large and copying takes a lot of time.

               

              It's best if you try this out in your environment and provide us your feedback.

              • 4. Re: How to setup a file monitor to poll the files in a order
                Guru

                Thanks uma ashok,

                second approch is working fine for me.

                 

                 

                Thanks,

                Seshu