Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13139

Branch-2: S3a to use thread pool that blocks clients

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0
    • 2.8.0
    • fs/s3
    • None
    • Incompatible change
    • Hide
      The configuration option 'fs.s3a.threads.core' is no longer supported. The string is still defined in org.apache.hadoop.fs.s3a.Constants.CORE_THREADS, however its value is ignored. If it is set, a warning message will be printed when initializing the S3A filesystem
      Show
      The configuration option 'fs.s3a.threads.core' is no longer supported. The string is still defined in org.apache.hadoop.fs.s3a.Constants.CORE_THREADS, however its value is ignored. If it is set, a warning message will be printed when initializing the S3A filesystem

    Description

      HADOOP-11684 is accepted into trunk, but was not applied to branch-2. I will attach a patch applicable to branch-2.

      It should be noted in CHANGES-2.8.0.txt that the config parameter 'fs.s3a.threads.core' has been been removed and the behavior of the ThreadPool for s3a has been changed.

      Attachments

        1. HADOOP-13139-001.patch
          28 kB
          Pieter Reuse
        2. HADOOP-13139-branch-2.001.patch
          28 kB
          Pieter Reuse
        3. HADOOP-13139-branch-2.002.patch
          28 kB
          Pieter Reuse
        4. HADOOP-13139-branch-2-003.patch
          29 kB
          Steve Loughran
        5. HADOOP-13139-branch-2-004.patch
          29 kB
          Steve Loughran
        6. HADOOP-13139-branch-2-005.patch
          29 kB
          Steve Loughran
        7. HADOOP-13139-branch-2-006.patch
          29 kB
          Steve Loughran

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue HADOOP-13262 set multipart delete timeout to 5 * 60s in S3ATestUtils.createTestFileSystem

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-11694 Über-jira: S3a phase II: robustness, scale and performance

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-11684 S3a to use thread pool that blocks clients

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              PieterReuse Pieter Reuse
              PieterReuse Pieter Reuse
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: