Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6437

increase frequency of admission control topic updates

    XMLWordPrintableJSON

Details

    Description

      Currently admission controls don't have a globally consistent view of how many queries have admitted so different coordinators can over-admit if queries arrive at multiple coordinators in-between topic updates. Topic updates happen every 2s by default. Worse, admission control topic updates can get stuck behind large catalog updates.

      IMPALA-3649 proposes having a globally consistent view of admission control, which will require architectural changes.

      In the meantime we could reduce the problem by increasing the frequency of topic updates for admission control and separating them from the larger catalog updates.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-6583 Various tests fail with missing database or table from catalog

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-6785 Starting an Impalad on an already running cluster may result in inconsistent cluster subscription

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-4140 Increase default threadpool size for statestore heartbeat and topic update

          • Major - Major loss of function.
          • Resolved
          relates to

          New Feature - A new feature of the product, which has yet to be developed. IMPALA-3649 Globally consistent admission control

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            People

              tarmstrong Tim Armstrong
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: