Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.3.6
-
None
-
None
Description
the fs.s3a.multipart.purge option has been in for a long time, to help avoid running up costs from incomplete uploads, especially during testing.
but
- adds overhead on startup (list + delete)
- dangerous if one client has a very short lifespan; all active s3a committer jobs will be broken
- obsoleted by s3 lifecycle rules
- and "hadoop s3guard uploads" cli.
proposed: deprecate for the next release, delete after.