Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4236

DAGClientServer is not really needed to be started/used in local mode

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.10.1
    • None
    • None

    Description

      For a customer, we found that they try to run hive on tez local mode tests, and due to strict firewall rules and security policy, the DagClientServer struggles to start or respond to connections (think about corporate laptops here). Instead of investigating the network-related workarounds, let's find another way and decide if DAGAppMaster really needs this server.
      LocalClient has an embedded DAGAppMaster, so at the first sight, it doesn't make sense to rely on the network environment in terms of AM usage. Hopefully, AM can become more lightweight in the long term if we try to eliminate pointless network usage.

      Attachments

        1. TEZ-4236.06.patch
          40 kB
          László Bodor
        2. TEZ-4236.05.patch
          42 kB
          László Bodor
        3. TEZ-4236.04.patch
          42 kB
          László Bodor
        4. TEZ-4236.03.patch
          42 kB
          László Bodor
        5. TEZ-4236.02.patch
          42 kB
          László Bodor
        6. TEZ-4236.01.patch
          33 kB
          László Bodor

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. TEZ-4282 Possible NPE in LocalClient after TEZ-4236

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue TEZ-1336 Avoid interacting with the DAGAppMaster via RPC

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. TEZ-4447 Collect VertexStatus properly when DAGClientServer is not used (local mode without network)

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-26444 Use tez.local.mode.without.network for unit tests

          • Major - Major loss of function.
          • Closed
          Testing discovered

          Bug - A problem which impairs or prevents the functions of the product. TEZ-4277 AsyncDispatcher can hang on serviceStop if the eventhandling thread is in BLOCKED state

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              abstractdog László Bodor
              abstractdog László Bodor
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: