Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-913

Umbrella: Add a way to register long-lived services in a YARN cluster

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.5.0, 2.4.1
    • None
    • api, resourcemanager
    • None

    Description

      In a YARN cluster you can't predict where services will come up -or on what ports. The services need to work those things out as they come up and then publish them somewhere.

      Applications need to be able to find the service instance they are to bond to -and not any others in the cluster.

      Some kind of service registry -in the RM, in ZK, could do this. If the RM held the write access to the ZK nodes, it would be more secure than having apps register with ZK themselves.

      Attachments

        1. 2014-09-03_Proposed_YARN_Service_Registry.pdf
          337 kB
          Steve Loughran
        2. 2014-09-08_YARN_Service_Registry.pdf
          327 kB
          Steve Loughran
        3. RegistrationServiceDetails.txt
          2 kB
          Robert Joseph Evans
        4. YARN-913-001.patch
          215 kB
          Steve Loughran
        5. YARN-913-002.patch
          261 kB
          Steve Loughran
        6. YARN-913-003.patch
          374 kB
          Steve Loughran
        7. YARN-913-003.patch
          343 kB
          Steve Loughran
        8. YARN-913-004.patch
          375 kB
          Steve Loughran
        9. YARN-913-006.patch
          389 kB
          Steve Loughran
        10. YARN-913-007.patch
          394 kB
          Steve Loughran
        11. YARN-913-008.patch
          424 kB
          Steve Loughran
        12. YARN-913-009.patch
          492 kB
          Steve Loughran
        13. YARN-913-010.patch
          497 kB
          Steve Loughran
        14. YARN-913-011.patch
          502 kB
          Steve Loughran
        15. YARN-913-012.patch
          523 kB
          Steve Loughran
        16. YARN-913-013.patch
          526 kB
          Steve Loughran
        17. YARN-913-014.patch
          532 kB
          Steve Loughran
        18. YARN-913-015.patch
          529 kB
          Steve Loughran
        19. YARN-913-016.patch
          544 kB
          Steve Loughran
        20. YARN-913-017.patch
          544 kB
          Steve Loughran
        21. YARN-913-018.patch
          545 kB
          Steve Loughran
        22. YARN-913-019.patch
          550 kB
          Steve Loughran
        23. YARN-913-020.patch
          550 kB
          Steve Loughran
        24. YARN-913-021.patch
          549 kB
          Steve Loughran
        25. yarnregistry.pdf
          168 kB
          Steve Loughran
        26. yarnregistry.pdf
          176 kB
          Steve Loughran
        27. yarnregistry.pdf
          176 kB
          Steve Loughran
        28. yarnregistry.tla
          14 kB
          Steve Loughran

        Issue Links

          is depended upon by

          Sub-task - The sub-task of the issue SLIDER-365 slider "resolve" command to retrieve service record

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. YARN-896 Roll up for long-lived services in YARN

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. SLIDER-149 Support a YARN service registry

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-11117 UGI HadoopLoginModule doesn't catch & wrap all kerberos-related exceptions

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-11111 MiniKDC to use locale EN_US for case conversions

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. YARN-4758 Enable discovery of AMs by containers

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-6413 FileSystem based Yarn Registry implementation

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-1143 Restrict the names that apps and types can have

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              4 Vote for this issue
              Watchers:
              68 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 1.5h Original Estimate - 1.5h
                  1.5h
                  Remaining:
                  Time Spent - 1h Remaining Estimate - 1h
                  1h
                  Logged:
                  Time Spent - 1h Remaining Estimate - 1h
                  1h