Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2492 (Clone of YARN-796) Allow for (admin) labels on nodes and resource-requests
  3. YARN-3361

CapacityScheduler side changes to support non-exclusive node labels

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • capacityscheduler
    • None
    • Reviewed

    Description

      According to design doc attached in YARN-3214, we need implement following logic in CapacityScheduler:
      1) When allocate a resource request with no node-label specified, it should get preferentially allocated to node without labels.

      2) When there're some available resource in a node with label, they can be used by applications with following order:

      • Applications under queues which can access the label and ask for same labeled resource.
      • Applications under queues which can access the label and ask for non-labeled resource.
      • Applications under queues cannot access the label and ask for non-labeled resource.

      3) Expose necessary information that can be used by preemption policy to make preemption decisions.

      Attachments

        1. YARN-3361.10.patch
          247 kB
          Wangda Tan
        2. YARN-3361.9.patch
          247 kB
          Wangda Tan
        3. YARN-3361.8.patch
          247 kB
          Wangda Tan
        4. YARN-3361.7.patch
          244 kB
          Wangda Tan
        5. YARN-3361.6.patch
          243 kB
          Wangda Tan
        6. YARN-3361.5.patch
          246 kB
          Wangda Tan
        7. YARN-3361.4.patch
          243 kB
          Wangda Tan
        8. YARN-3361.3.patch
          242 kB
          Wangda Tan
        9. YARN-3361.2.patch
          181 kB
          Wangda Tan
        10. YARN-3361.1.patch
          181 kB
          Wangda Tan

        Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue YARN-3213 Respect labels in Capacity Scheduler when computing user-limit

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-4839 ResourceManager deadlock between RMAppAttemptImpl and SchedulerApplicationAttempt

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-2176 CapacityScheduler loops over all running applications rather than actively requesting apps

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue YARN-3486 Duplicate check for partitioned request on partitioned node

          • Major - Major loss of function.
          • Open

          Activity

            People

              leftnoteasy Wangda Tan
              leftnoteasy Wangda Tan
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: