Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12756

Incorporate Aliyun OSS file system implementation

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • HADOOP-12756
    • HADOOP-12756, 3.0.0-alpha2, 2.9.1
    • fs, fs/oss
    • None
    • Reviewed
    • Hide
      Aliyun OSS is widely used among China’s cloud users and this work implemented a new Hadoop compatible filesystem AliyunOSSFileSystem with oss scheme, similar to the s3a and azure support.
      Show
      Aliyun OSS is widely used among China’s cloud users and this work implemented a new Hadoop compatible filesystem AliyunOSSFileSystem with oss scheme, similar to the s3a and azure support.

    Description

      Aliyun OSS is widely used among China’s cloud users, but currently it is not easy to access data laid on OSS storage from user’s Hadoop/Spark application, because of no original support for OSS in Hadoop.

      This work aims to integrate Aliyun OSS with Hadoop. By simple configuration, Spark/Hadoop applications can read/write data from OSS without any code change. Narrowing the gap between user’s APP and data storage, like what have been done for S3 in Hadoop

      Attachments

        1. Aliyun-OSS-integration-v2.pdf
          79 kB
          Genmao Yu
        2. HADOOP-12756.010.patch
          147 kB
          Mingfei Shi
        3. Aliyun-OSS-integration.pdf
          144 kB
          Mingfei Shi
        4. HADOOP-12756.009.patch
          103 kB
          Kai Zheng
        5. HADOOP-12756.007.patch
          103 kB
          Mingfei Shi
        6. HADOOP-12756.008.patch
          103 kB
          Kai Zheng
        7. HADOOP-12756.006.patch
          102 kB
          Mingfei Shi
        8. HADOOP-12756.005.patch
          96 kB
          Mingfei Shi
        9. HADOOP-12756.004.patch
          95 kB
          Kai Zheng
        10. HADOOP-12756.003.patch
          95 kB
          Kai Zheng
        11. HADOOP-12756-v02.patch
          92 kB
          Mingfei Shi
        12. HCFS User manual.md
          10 kB
          Ling Zhou
        13. OSS integration.pdf
          225 kB
          Ling Zhou

        Issue Links

          is related to

          Task - A task that needs to be done. HADOOP-16542 Update commons-beanutils version to 1.9.4

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-14649 Update aliyun-sdk-oss version to 2.8.1

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14964 AliyunOSS: backport Aliyun OSS module to branch-2

          • Major - Major loss of function.
          • Resolved
          There are no Sub-Tasks for this issue.

          Activity

            People

              kakagou mingfei.shi
              shimingfei Mingfei Shi
              Votes:
              2 Vote for this issue
              Watchers:
              51 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: