Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5489

MR jobs hangs as it does not use the node-blacklisting feature in RM requests

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 2.2.0
    • None
    • None
    • Reviewed

    Description

      When RM restarted, if during restart one NM went bad (bad disk), NM got blacklisted by AM and RM keeps giving the containers on the same node even though AM doesn't want it there.

      Need to change AM to specifically blacklist node in the RM requests.

      Attachments

        1. MAPREDUCE-5489.1.patch
          17 kB
          Zhijie Shen
        2. MAPREDUCE-5489.2.patch
          17 kB
          Zhijie Shen

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. YARN-1141 Updating resource requests should be decoupled with updating blacklist

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. YARN-1133 AMRMClient should be able to blacklist nodes, and specify them in AllocateRequest

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5559 Reconsidering the policy of ignoring the blacklist after reaching the threshold

          • Major - Major loss of function.
          • Open

          Activity

            People

              zjshen Zhijie Shen
              yeshavora Yesha Vora
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: