Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-128

[Umbrella] RM Restart Phase 1: State storage and non-work-preserving recovery

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0-alpha
    • None
    • resourcemanager
    • None

    Description

      This umbrella jira tracks the work needed to preserve critical state information and reload them upon RM restart.

      Attachments

        1. YARN-128.full-code.5.patch
          255 kB
          Bikas Saha
        2. restart-fs-store-11-17.patch
          17 kB
          Bikas Saha
        3. restart-zk-store-11-17.patch
          61 kB
          Bikas Saha
        4. restart-12-11-zkstore.patch
          21 kB
          Bikas Saha
        5. YARN-128.full-code-4.patch
          179 kB
          Bikas Saha
        6. YARN-128.new-code-added-4.patch
          78 kB
          Bikas Saha
        7. YARN-128.old-code-removed.4.patch
          123 kB
          Bikas Saha
        8. YARN-128.full-code.3.patch
          176 kB
          Bikas Saha
        9. YARN-128.new-code-added.3.patch
          74 kB
          Bikas Saha
        10. YARN-128.old-code-removed.3.patch
          123 kB
          Bikas Saha
        11. RMRestartPhase1.pdf
          59 kB
          Bikas Saha
        12. YARN-128.patch
          92 kB
          Devaraj Kavali
        13. RM-recovery-initial-thoughts.txt
          3 kB
          Bikas Saha
        14. MR-4343.1.patch
          17 kB
          Tsuyoshi Ozawa

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue MAPREDUCE-5086 MR app master deletes staging dir when sent a reboot command from the RM

          • Major - Major loss of function.
          • Closed
          is blocked by

          Sub-task - The sub-task of the issue MAPREDUCE-5505 Clients should be notified job finished only after job successfully unregistered

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. YARN-1082 Secure RM with recovery enabled and rm state store on hdfs fails with gss exception

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. YARN-495 Change NM behavior of reboot to resync

          • Major - Major loss of function.
          • Closed
          is related to

          Sub-task - The sub-task of the issue MAPREDUCE-5476 Job can fail when RM restarts after staging dir is cleaned but before MR successfully unregister with RM

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Sub-task - The sub-task of the issue MAPREDUCE-5471 Succeed job tries to restart after RMrestart

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue MAPREDUCE-5127 MR job succeeds and exits even when unregister with RM fails

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue MAPREDUCE-5466 Historyserver does not refresh the result of restarted jobs after RM restart

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Sub-task - The sub-task of the issue YARN-1305 RMHAProtocolService#serviceInit should handle HAUtil's IllegalArgumentException

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5472 reducer of sort job restarts from scratch in between after RM restart

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5567 [Umbrella] Stabilize MR framework w.r.t ResourceManager restart

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-209 Capacity scheduler doesn't trigger app-activation after adding nodes

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. YARN-479 NM retry behavior for connection to RM should be similar for lost heartbeats

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. YARN-149 [Umbrella] ResourceManager (RM) Fail-over

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. YARN-218 Distiguish between "failed" and "killed" app attempts

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. YARN-556 [Umbrella] RM Restart phase 2 - Work preserving restart

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-1139 [Umbrella] Convert all RM components to Services

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              acmurthy Arun Murthy
              Votes:
              1 Vote for this issue
              Watchers:
              69 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: