Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-543 [Umbrella] NodeManager localization related issues
  3. YARN-2902

Killing a container that is localizing can orphan resources in the DOWNLOADING state

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      If a container is in the process of localizing when it is stopped/killed then resources are left in the DOWNLOADING state. If no other container comes along and requests these resources they linger around with no reference counts but aren't cleaned up during normal cache cleanup scans since it will never delete resources in the DOWNLOADING state even if their reference count is zero.

      Attachments

        1. YARN-2902.patch
          8 kB
          Varun Saxena
        2. YARN-2902.002.patch
          9 kB
          Varun Saxena
        3. YARN-2902.03.patch
          29 kB
          Varun Saxena
        4. YARN-2902.04.patch
          26 kB
          Varun Saxena
        5. YARN-2902.05.patch
          108 kB
          Varun Saxena
        6. YARN-2902.06.patch
          113 kB
          Varun Saxena
        7. YARN-2902.07.patch
          79 kB
          Varun Saxena
        8. YARN-2902.08.patch
          33 kB
          Varun Saxena
        9. YARN-2902.09.patch
          22 kB
          Varun Saxena
        10. YARN-2902.10.patch
          25 kB
          Varun Saxena
        11. YARN-2902.11.patch
          25 kB
          Varun Saxena
        12. YARN-2902-branch-2.6.01.patch
          23 kB
          Varun Saxena

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. YARN-4354 Public resource localization fails with NPE

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. YARN-4380 TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. YARN-6056 Yarn NM using LCE shows a failure when trying to delete a non-existing dir

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              varun_saxena Varun Saxena
              jlowe Jason Darrell Lowe
              Votes:
              0 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: