Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15220

Über-jira: S3a phase V: Hadoop 3.2 features

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0
    • None
    • None

    Description

      Über-jira for S3A work for Hadoop 3.2.x

      The items from HADOOP-14831 which didn't get into Hadoop-3.1, and anything else

      Attachments

        Issue Links

          depends upon

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-15541 AWS SDK can mistake stream timeouts for EOF and throw SdkClientExceptions

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15226 Über-JIRA: S3Guard Phase III: Hadoop 3.2 features

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15263 hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-6974 Add standard configuration keys for HTrace values, propagate across to MR committers if set

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is superceded by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15620 Über-jira: S3A phase VI: Hadoop 3.3 features

          • Major - Major loss of function.
          • Resolved
          supercedes

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14831 Über-jira: S3a phase IV: Hadoop 3.1 features

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          1.
          		 add htrace context to HTTP requests as ? parameter Sub-task Resolved Unassigned
          2.
          		 S3ADataBlocks.DiskBlock to lazy create dest file for faster 0-byte puts Sub-task Resolved Unassigned
          3.
          		 use s3 tags/headers to record permissions on objects, so preserving them through distcp round trips Sub-task Resolved Unassigned
          4.
          		 Automatically determine region & hence S3 endpoint of buckets Sub-task Resolved Unassigned
          5.
          		 S3A shell entry point to support commands specific to S3A. Sub-task Resolved Unassigned
          6.
          		 Hive doesn't let s3a patch the credential provider path Sub-task Resolved Unassigned
          7.
          		 s3a encryption tests fail when dest bucket has SSE-KMS enabled Sub-task Resolved Unassigned
          8.
          		 AWSCredentialProviderList to throw custom NoCredentialsException; retry logic to handle Sub-task Resolved Unassigned
          9.
          		 Add Private/Unstable BulkDelete operations to supporting object stores for DistCP Sub-task Resolved Steve Loughran
          10.
          		 S3AInputStream: Handle http stream skip(n) skipping < n bytes in a forward seek Sub-task Resolved Unassigned
          11.
          		 S3ABlockOutputStream.flush() be no-op when stream closed Sub-task Resolved Gabor Bota
          12.
          		 ITestS3AEmptyDirectory.testParallelJobsToAdjacentPaths failure, FileAlreadyExistsException Sub-task Resolved Steve Loughran
          13.
          		 regression: secret stripping from S3x URIs breaks some downstream code Sub-task Resolved Vinayakumar B
          14.
          		 S3A ITestS3GuardWriteBack failing if bucket explicitly set to s3guard+DDB Sub-task Resolved Steve Loughran
          15.
          		 S3 returning 400 on the directory /test/ GET of getFileStatus Sub-task Resolved Steve Loughran
          16.
          		 S3ARetryPolicy for 400/BadArgument to be "fail" Sub-task Resolved Steve Loughran
          17.
          		 Stabilize/tune S3A committers; review correctness & docs Sub-task Resolved Steve Loughran
          18.
          		 distcp numListstatusThreads option doesn't get to -delete scan Sub-task Resolved Steve Loughran
          19.
          		 S3A Metrics in S3AInstrumentation Cause Memory Leaks in HBase Export Sub-task Resolved Unassigned
          20.
          		 S3A directory committer commit job fails if _temporary directory created under dest Sub-task Resolved Steve Loughran
          21.
          		 S3A staging committers to not log FNFEs on job abort listings Sub-task Resolved Unassigned
          22.
          		 S3AFileStatus to add a serialVersionUID; review & test serialization Sub-task Resolved Unassigned
          23.
          		 Expand S3A Assumed Role docs Sub-task Resolved Steve Loughran
          24.
          		 S3A Multipart Uploader to work with S3Guard and encryption Sub-task Resolved Ewan Higgs
          25.
          		 AssumedRoleCredentialProvider to propagate connection settings of S3A FS Sub-task Resolved Steve Loughran
          26.
          		 Stabilize S3A Assumed Role support Sub-task Resolved Steve Loughran
          27.
          		 Update aws-sdk version to 1.11.375 Sub-task Resolved Steve Loughran
          28.
          		 s3guard uploads command to add a way to complete outstanding uploads Sub-task Resolved Unassigned
          29.
          		 FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append. Sub-task Resolved Steve Loughran
          30.
          		 increase default value of fs.s3a.multipart.size to 128M Sub-task Resolved Unassigned
          31.
          		 Remove s3a user:secret authentication Sub-task Resolved Steve Loughran

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: