Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15226 Über-JIRA: S3Guard Phase III: Hadoop 3.2 features
  3. HADOOP-14918

Remove the Local Dynamo DB test option

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.9.0, 3.0.0
    • 3.2.0, 3.1.4, 2.10.1
    • fs/s3
    • None

    Description

      I'm going to propose cutting out the localdynamo test option for s3guard

      • the local DDB JAR is unmaintained/lags the SDK We work with...eventually there'll be differences in API.
      • as the local dynamo DB is unshaded. it complicates classpath setup for the build. Remove it and there's no need to worry about versions of anything other than the shaded AWS
      • it complicates test runs. Now we need to test for both localdynamo and real dynamo
      • but we can't ignore real dynamo, because that's the one which matters

      While the local option promises to reduce test costs, really, it's just adding complexity. If you are testing with s3guard, you need to have a real table to test against., And with the exception of those people testing s3a against non-AWS, consistent endpoints, everyone should be testing with S3Guard.

      Straightforward to remove.

      Attachments

        1. HADOOP-14918-001.patch
          39 kB
          Steve Loughran
        2. HADOOP-14918-002.patch
          39 kB
          Steve Loughran
        3. HADOOP-14918-003.patch
          34 kB
          Steve Loughran
        4. HADOOP-14918-004.patch
          44 kB
          Gabor Bota
        5. HADOOP-14918.005.patch
          49 kB
          Gabor Bota
        6. HADOOP-14918.006.patch
          49 kB
          Gabor Bota
        7. HADOOP-14918-branch-2.10.001.patch
          39 kB
          Jonathan Hung

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. YARN-8374 Upgrade objenesis to 2.6

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Sub-task - The sub-task of the issue HADOOP-14734 add option to tag DDB table(s) created

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HADOOP-14068 Add integration test version of TestMetadataStore for DynamoDB

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-16175 S3Guard to add DynamoDBLocal Support

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15301 TestDynamoDBMetadataStore test case fails on PPC

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15636 Add ITestDynamoDBMetadataStore

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. YARN-8338 TimelineService V1.5 doesn't come up after HADOOP-15406

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Web Link GitHub Pull Request #1864

          Activity

            People

              gabor.bota Gabor Bota
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: