Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5177

Make Node-Manager Download-Resource Component extensible.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 2.7.0
    • None
    • nodemanager

    Description

      Problem:

      • Downloading files to a local machine/node is called "resource-localization".
      • There are two components that perform resource-location (PublicLocalizer and ComponetLocalizers)
      • Both components utilizes FSDownload.class to perform their downloads.
      • We need a custom implementation of FSDownload.

      Solution:

      • With this change, we make FSDownload.class extensible by wrapping it in a new ResourceDownloader.interface
      • We also update the PublicLocalizer and ComponetLocalizers to load ResourceDownloader rather than FSDownload.
      • NOTE: We use reflection to load the right implementation during runtime.

      Attachments

        1. YARN-5177-V0.patch
          16 kB
          Emeka-NMC

        Activity

          People

            nnaemeka Emeka-NMC
            nnaemeka Emeka-NMC
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: