Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13377 AliyunOSS: improvements for stabilization and optimization
  3. HADOOP-15027

AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.1.0, 2.10.0, 2.9.1, 3.0.1
    • fs/oss
    • None
    • Support multi-thread pre-read in AliyunOSSInputStream to improve the sequential read performance from Hadoop to Aliyun OSS.

    Description

      Currently, AliyunOSSInputStream uses single thread to read data from AliyunOSS, so we can do some refactoring by using multi-thread pre-read to improve read performance.

      Attachments

        1. HADOOP-15027.001.patch
          12 kB
          wujinhu
        2. HADOOP-15027.002.patch
          16 kB
          wujinhu
        3. HADOOP-15027.003.patch
          18 kB
          wujinhu
        4. HADOOP-15027.004.patch
          19 kB
          wujinhu
        5. HADOOP-15027.005.patch
          19 kB
          wujinhu
        6. HADOOP-15027.006.patch
          19 kB
          wujinhu
        7. HADOOP-15027.007.patch
          19 kB
          wujinhu
        8. HADOOP-15027.008.patch
          21 kB
          wujinhu
        9. HADOOP-15027.009.patch
          21 kB
          wujinhu
        10. HADOOP-15027.010.patch
          23 kB
          wujinhu
        11. HADOOP-15027.011.patch
          23 kB
          wujinhu
        12. HADOOP-15027.012.patch
          23 kB
          wujinhu
        13. HADOOP-15027.013.patch
          23 kB
          wujinhu
        14. HADOOP-15027.014.patch
          24 kB
          wujinhu

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HADOOP-15868 AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15039 Move SemaphoredDelegatingExecutor to hadoop-common

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              wujinhu wujinhu
              wujinhu wujinhu
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: