Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-19251

Add Options.Rename.THROW_NON_ATOMIC

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.6
    • None
    • fs
    • None

    Description

      I propose we add an option `Options.Rename.THROW_NON_ATOMIC` to change `rename()` behavior to throw when the underlying filesystem's rename operation is not atomic.

      This would be useful for callers that expect to perform an atomic op but want to fail if when an atomic rename fails.

       

      At first this might seem something that can be done by querying capabilities of the filesystem but that would only work on real filesystems. A motivating example would be a virtual filesystem for which paths can resolve to any concrete filesystem (s3, etc). If `rename()` is called with two virtual paths that resolve to different filesystems (s3 and gcs for example) then obviously the operation can't be atomic since bytes must be copied from one fs to another.

       

      What do you think steve_l ?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              alkis Alkis Evlogimenos
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: