Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17400 Optimize S3A for maximum performance in directory listings
  3. HADOOP-17281

Implement FileSystem.listStatusIterator() in S3AFileSystem

    XMLWordPrintableJSON

Details

    Description

      Currently S3AFileSystem only implements listStatus() api which returns an array. Once we implement the listStatusIterator(), clients can benefit from the async listing done recently 

      https://issues.apache.org/jira/browse/HADOOP-17074  by performing some tasks on files while iterating them.

       

      CC stevel

      Attachments

        Issue Links

          breaks

          Sub-task - The sub-task of the issue HDFS-15620 RBF: Fix test failures after HADOOP-17281

          • Major - Major loss of function.
          • Resolved
          causes

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-17695 adls test suite TestAdlContractGetFileStatusLive failing with no assertJ on the classpath

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HDFS-15626 TestWebHDFS.testLargeDirectory failing

          • Major - Major loss of function.
          • Resolved
          contains

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-17300 FileSystem.DirListingIterator.next() call should return NoSuchElementException

          • Major - Major loss of function.
          • Resolved
          relates to

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-16830 Add Public IOStatistics API

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #2354

          Web Link GitHub Pull Request #2380

          Activity

            People

              mukund-thakur Mukund Thakur
              mukund-thakur Mukund Thakur
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 40m
                  3h 40m