Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5486 Port control-plane parts of ImpalaInternalService to KRPC
  3. IMPALA-5119

Don't make RPCs from Coordinator::UpdateBackendExecStatus()

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • Impala 2.9.0
    • None
    • Distributed Exec
    • None

    Description

      If it reports a bad status, UpdateFragmentExecStatus() will call UpdateStatus(), which takes Coordinator::lock_ and then calls Cancel(). That method issues one RPC per fragment instance.

      In KRPC, doing so much work from UpdateFragmentExecStatus() - which is an RPC handler - is a bad idea, even if the RPCs are issued asynchronously. There's still some serialization cost.

      It's also a bad idea to do all this work while holding lock_. We should address both of these to ensure scalability of the cancellation path.

      Attachments

        Issue Links

          depends upon

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-2990 Coordinator should timeout and cancel queries with unresponsive / stuck executors

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-5384 Simplify coordinator locking protocol

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Sub-task - The sub-task of the issue IMPALA-6984 Coordinator should cancel backends when returning EOS

          • Major - Major loss of function.
          • Reopened
          is related to

          Sub-task - The sub-task of the issue IMPALA-6984 Coordinator should cancel backends when returning EOS

          • Major - Major loss of function.
          • Reopened

          Activity

            People

              wzhou Wenzhe Zhou
              henryr Henry Robinson
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: