Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15620 Über-jira: S3A phase VI: Hadoop 3.3 features
  3. HADOOP-15990

S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 3.2.0
    • None
    • fs/s3
    • None

    Description

      in S3AFileSystem.initialize(), we check for the bucket existing with verifyBucketExists(), which calls s3.doesBucketExist(). But that doesn't check for auth issues.

      s3. doesBucketExistV2() does at least validate credentials, and should be switched to. This will help things fail faster

      See SPARK-24000

      (this is a dupe of HADOOP-15409; moving off git PRs so we can get yetus to test everything)

      Attachments

        1. HADOOP-15409-005.patch
          0.9 kB
          Steve Loughran
        2. HADOOP-15990-006.patch
          6 kB
          lqjacklee

        Issue Links

          duplicates

          Sub-task - The sub-task of the issue HADOOP-15409 S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          Is contained by

          Sub-task - The sub-task of the issue HADOOP-16711 S3A bucket existence checks to support v2 API and "no checks at all"

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-16711 S3A bucket existence checks to support v2 API and "no checks at all"

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-24000 S3A: Create Table should fail on invalid AK/SK

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Jack-Lee lqjacklee
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: