Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5774

MR Job stuck in ACCEPTED status without any progress in Fair Scheduler if set yarn.scheduler.minimum-allocation-mb to 0.

    XMLWordPrintableJSON

Details

    Description

      MR Job stuck in ACCEPTED status without any progress in Fair Scheduler because there is no resource request for the AM. This happened when you configure yarn.scheduler.minimum-allocation-mb to zero.

      The problem is in the code used by both Capacity Scheduler and Fair Scheduler. scheduler.increment-allocation-mb is a concept in FS, but not CS. So the common code in class RMAppManager passes the yarn.scheduler.minimum-allocation-mb as incremental one because there is no incremental one for CS when it tried to normalize the resource requests.

           SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
                scheduler.getClusterResource(),
                scheduler.getMinimumResourceCapability(),
                scheduler.getMaximumResourceCapability(),
                scheduler.getMinimumResourceCapability());  --> incrementResource should be passed here.
      

      Attachments

        1. YARN-5774.001.patch
          16 kB
          Yufei Gu
        2. YARN-5774.002.patch
          17 kB
          Yufei Gu
        3. YARN-5774.003.patch
          17 kB
          Yufei Gu
        4. YARN-5774.004.patch
          23 kB
          Yufei Gu
        5. YARN-5774.005.patch
          34 kB
          Yufei Gu
        6. YARN-5774.006.patch
          36 kB
          Yufei Gu
        7. YARN-5774.007.patch
          38 kB
          Yufei Gu

        Activity

          People

            yufeigu Yufei Gu
            yufeigu Yufei Gu
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: