Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-11669 [Umbrella] cgroup v2 support
  3. YARN-11675

Update MemoryResourceHandler implementation for cgroup v2 support

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.5.0
    • None
    • Reviewed

    Description

      cgroup v2 has some changes in various controllers (some changed their functionality, some were removed). This task is about updating MemoryResourceHandler's implementation.

      Differences in the controls comparing to cgroup v1:

      Hard limit on memory

      memory.limit_in_bytes control is replaced with memory.max

      Soft limit on memory

      memory.soft_limit_in_bytes control is replaced with memory.low

      Detailed descriptions about the memory controls can be found in the official cgroup v2 documentation.

      Swappiness

      memory.swappiness has been removed from the available cgroup v2 controls.

      Quoting redhat documentation:

      Swappiness is a property for the Linux kernel that changes the balance between swapping out runtime memory, as opposed to dropping pages from the system page cache. Swappiness can be set to values between 0 and 100, inclusive. A low value means the kernel will try to avoid swapping as much as possible where a higher value instead will make the kernel aggressively try to use swap space.

      Referring this case study we found that most of the time swappiness didn't work as expected as it mostly depends on the I/O balance of the system, so it is no longer available in cgroup v2.

      Attachments

        Issue Links

          Activity

            People

              pszucs Peter Szucs
              bteke Benjamin Teke
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: