Details
Description
I would like to request a feature that allows trigger based download via additional uri parameters.
USE CASE DESCRIPTION
In enterprise env. we often see file downloads ((s)FTP(s)) to be based on trigger files instead of using File locks. Meaning an extra empty file is created once original process is finished filling up the actual data file with data. Only after trigger is created data file is viable for pick up. This is to avoid pick up of partial data.
TRIGGER LOGIC
Triggers are usually named the same way as the actual data files. So if data file being created is dataFile.txt then trigger filename is usually dataFile.txt.trig or dataFile.trig (depending on trigger policy).
We see 2 trigger file policies being applied - APPEND (more common - dataFile.txt.trig) or REPLACE (less common dataFile.trig).
SUGGESTED NEW URI PARAMs
- triggerExtension (default to "trig" but can be anything really)
- triggerPolicy (possible values APPEND or REPLACE - default to APPEND)
SUGGESTED IMPLEMENTATION
Look for trigger files first and then use pollEnrich() and replace trigger file name downloaded appropriatelly for actual dataFile based on trigger extension and trigger policy
If this is not picked up by anybody else our team will pick up and implement and will create a pull request