I need to execute a project when files matching a pattern arrive on an FTP site. I have tried using both monitors and schedules, but neither quite solves my problem:
1. Only run when there's something there to act upon (which is exactly what I'm looking for)
2. Will never run again if they failed to run when an event happened. Example: if we restart the service or reboot the server, any files that have arrived on (or were already on) an FTP site will not trigger the monitor. This is a major problem.
1. Always run a project whether there is useful work to do or not.
2. Logs become filled with evaluations and it's like searching for a needle in a haystack to find an entry where meaningful work was done. This is a maintenance headache that I'm looking to avoid.
What would seem ideal for this situation is the ability to add a monitor event type for the existence of any files matching the pattern. This would run only when there is something to do and would be durable in the case of a service outage - it would pickup where it left off and handle any existing files on the site.
The situation you are describing tends to lean more toward utilizing a TRIGGER rather than a Monitor or Scheduled Job.
This activates on the event "Successful Upload". Meaning, as soon as the full file has been received, it can trigger a WorkFlow action for processing/moving, etc.
This option does allow you to pass the file path and name to your project. Thus, making sure you process specifically the file that was just uploaded. Conditions can be set to make sure you're grabbing files that are needed for processing.
There is a system alert for Triggers allowing you to be notified if something doesn't run.
Also, you have the option of utilizing the Parent Directory of the file uploaded to go back and retrieve "all" files from that location (including the one that was just uploaded) .. thus, making sure you retrieve the files that have triggered the movement as well as anything that happened to have been missed.
This would be a much cleaner and smoother way to process the files like you are describing.
Lead Solutions Consultant
If the service is not running when the file arrives, will the trigger ever fire after the service comes back up?
Also, if the trigger fired for a file that arrived, but the project failed (maybe a destination location temporarily unavailable), will the trigger fire again as long as the file is out there or will the file be ignored at that point?
My question here is .. would this be the norm? Or a rare exception? In either case, there are options for you to create a "check" routine to find files that have been missed or needs to be run based on missed triggers or protocol listener being down.
Will the trigger fire again as long as the file is out there or be ignored? No...
Same as above. I would design my workflow to accommodate for these exceptions if that is more than a rare instance. Under most scenarios .. this doesn't happen that often.
Lead Solutions Consultant