Details
Description
The current AWS S3Consumer class has a problem if user supply the parameter deleteAfterRead=false. The S3Consumer will always consume the same files over and over again with the size of getMaxMessagesPerPoll().
After some code chasing, i think, the root cause is because the ListObjectsRequest does not has the previous marker value. Hence, i wonder if we could do something like below:
private string marker; // new line to define the marker string
..
..
..
ListObjectsRequest listObjectsRequest = new ListObjectsRequest();
listObjectsRequest.setBucketName(bucketName);
listObjectsRequest.setPrefix(getConfiguration().getPrefix());
listObjectsRequest.setMaxKeys(maxMessagesPerPoll);
listObjectsRequest.setMarker(marker); // new line to set the markerObjectListing listObjects = getAmazonS3Client().listObjects(listObjectsRequest);
marker = listObjects.getMarker(); // where marker is track
Attachments
Issue Links
- is duplicated by
-
CAMEL-8539 S3 marker does not work
- Closed
- is related to
-
CAMEL-9784 Camel polling the files from S3 only once if deleteAfterRead is false
- Resolved