Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6355

[Umbrella] Preprocessor framework for AM and Client interactions with the RM

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Currently on the NM, we have the AMRMProxy framework to intercept the AM <-> RM communication and enforce policies. This is used both by YARN federation (YARN-2915) as well as Distributed Scheduling (YARN-2877).

      This JIRA proposes to introduce a similar framework on the the RM side, so that pluggable policies can be enforced on ApplicationMasterService centrally as well.

      This would be similar in spirit to a Java Servlet Filter Chain. Where the order of the interceptors can declared externally.

      Once possible usecase would be:
      the OpportunisticContainerAllocatorAMService is implemented as a wrapper over the ApplicationMasterService. It would probably be better to implement it as an Interceptor.

      Attachments

        1. YARN-6355.007.patch
          90 kB
          Arun Suresh
        2. YARN-6355.006.patch
          90 kB
          Arun Suresh
        3. YARN-6355.005.patch
          85 kB
          Arun Suresh
        4. YARN-6355.004.patch
          91 kB
          Arun Suresh
        5. YARN-6355.003.patch
          92 kB
          Arun Suresh
        6. YARN-6355-one-pager.pdf
          235 kB
          Arun Suresh
        7. YARN-6355.002.patch
          91 kB
          Arun Suresh
        8. YARN-6355.001.patch
          67 kB
          Arun Suresh

        Issue Links

          Activity

            People

              asuresh Arun Suresh
              asuresh Arun Suresh
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated: