Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-556 [Umbrella] RM Restart phase 2 - Work preserving restart
  3. YARN-2229

ContainerId can overflow with RM restart

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.6.0
    • resourcemanager
    • None
    • Reviewed

    Description

      On YARN-2052, we changed containerId format: upper 10 bits are for epoch, lower 22 bits are for sequence number of Ids. This is for preserving semantics of ContainerId#getId(), ContainerId#toString(), ContainerId#compareTo(), ContainerId#equals, and ConverterUtils#toContainerId. One concern is epoch can overflow after RM restarts 1024 times.
      To avoid the problem, its better to make containerId long. We need to define the new format of container Id with preserving backward compatibility on this JIRA.

      Attachments

        1. YARN-2229.1.patch
          17 kB
          Tsuyoshi Ozawa
        2. YARN-2229.10.patch
          30 kB
          Tsuyoshi Ozawa
        3. YARN-2229.10.patch
          30 kB
          Tsuyoshi Ozawa
        4. YARN-2229.11.patch
          30 kB
          Tsuyoshi Ozawa
        5. YARN-2229.12.patch
          28 kB
          Tsuyoshi Ozawa
        6. YARN-2229.13.patch
          29 kB
          Tsuyoshi Ozawa
        7. YARN-2229.14.patch
          31 kB
          Tsuyoshi Ozawa
        8. YARN-2229.15.patch
          34 kB
          Tsuyoshi Ozawa
        9. YARN-2229.16.patch
          34 kB
          Tsuyoshi Ozawa
        10. YARN-2229.2.patch
          19 kB
          Tsuyoshi Ozawa
        11. YARN-2229.2.patch
          19 kB
          Tsuyoshi Ozawa
        12. YARN-2229.3.patch
          19 kB
          Tsuyoshi Ozawa
        13. YARN-2229.4.patch
          19 kB
          Tsuyoshi Ozawa
        14. YARN-2229.5.patch
          20 kB
          Tsuyoshi Ozawa
        15. YARN-2229.6.patch
          12 kB
          Tsuyoshi Ozawa
        16. YARN-2229.7.patch
          13 kB
          Tsuyoshi Ozawa
        17. YARN-2229.8.patch
          30 kB
          Tsuyoshi Ozawa
        18. YARN-2229.9.patch
          31 kB
          Tsuyoshi Ozawa

        Issue Links

          breaks

          Task - A task that needs to be done. SLIDER-425 Build broken against 2.6.0-SNAPSHOT due to new abstract methods added by YARN-2229

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue YARN-2052 ContainerId creation after work preserving restart is broken

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue YARN-2182 Update ContainerId#toString() to avoid conflicts before and after RM restart

          • Major - Major loss of function.
          • Closed

          Activity

            People

              ozawa Tsuyoshi Ozawa
              ozawa Tsuyoshi Ozawa
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: