Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-896 Roll up for long-lived services in YARN
  3. YARN-2704

Localization and log-aggregation will fail if hdfs delegation token expired after token-max-life-time

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 2.6.0
    • None
    • None
    • Reviewed

    Description

      In secure mode, YARN requires the hdfs-delegation token to do localization and log aggregation on behalf of the user. But the hdfs delegation token will eventually expire after max-token-life-time. So, localization and log aggregation will fail after the token expires.

      Attachments

        1. YARN-2704.4.patch
          93 kB
          Jian He
        2. YARN-2704.3.patch
          92 kB
          Jian He
        3. YARN-2704.2.patch
          92 kB
          Jian He
        4. YARN-2704.2.patch
          88 kB
          Jian He
        5. YARN-2704.1.patch
          85 kB
          Jian He

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. YARN-2964 RM prematurely cancels tokens for jobs that submit jobs (oozie)

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. YARN-3439 RM fails to renew token when Oozie launcher leaves before sub-job finishes

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. YARN-2790 NM can't aggregate logs past HDFS delegation token expiry.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. YARN-2425 When Application submitted by via Yarn RM WS, log aggregation does not happens

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. YARN-2794 Fix log msgs about distributing system-credentials

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. YARN-3472 Possible leak in DelegationTokenRenewer#allTokens

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-7295 Support arbitrary max expiration times for delegation token

          • Major - Major loss of function.
          • Open

          Activity

            People

              jianhe Jian He
              jianhe Jian He
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: