Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15619 Über-JIRA: S3Guard Phase IV: Hadoop 3.3 features
  3. HADOOP-15837

DynamoDB table Update can fail S3A FS init

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 2.10.0, 3.2.0, 3.0.4, 3.1.2
    • fs/s3
    • None

    • s3guard test with small capacity (10) but autoscale enabled & multiple consecutive parallel test runs executed...this seems to have been enough load to trigger the state change

    Description

      When DDB autoscales a table, it goes into an UPDATING state. The waitForTableActive operation in the AWS SDK doesn't seem to wait long enough for this to recover. We need to catch & retry

      Attachments

        1. HADOOP-15837-branch-3.1-004.patch
          1 kB
          Steve Loughran
        2. HADOOP-15837-003.patch
          10 kB
          Steve Loughran
        3. HADOOP-15837-002.patch
          10 kB
          Steve Loughran
        4. HADOOP-15837-001.patch
          10 kB
          Steve Loughran

        Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue HADOOP-14576 s3guard DynamoDB resource not found: tables not ACTIVE state after initial connection

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HADOOP-14576 s3guard DynamoDB resource not found: tables not ACTIVE state after initial connection

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: