Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15220 Über-jira: S3a phase V: Hadoop 3.2 features
  3. HADOOP-15201

Automatically determine region & hence S3 endpoint of buckets

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 3.0.0
    • None
    • fs/s3
    • None

    Description

      Use getBucketLocation to determine location & map to endpoint, if fs.s3a.endpoint is set to "automatic"

      S3guard added the API call String getBucketLocation(), which is used for DDB binding, We can also use this to determine the s3 endpoint, to avoid recurrent issues with auth failures related to it not being valid

      Still need to handle: buckets on third-party servers, and the inevitability that new AWS regions will be added after the Hadoop version & AWS Jar is shipped and frozen

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: