Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7349

Automatically choose mem_limit based on estimate, clamped to range

    XMLWordPrintableJSON

Details

    Description

      We should add admission control support for intelligently choosing how much memory to give a query, based on the memory estimate. Currently we require that you set a single mem_limit per pool. Instead it would be better if we allowed configuring min/max guardrails within which admission control chose an amount of memory that will allow good performance.

      Initially, I think mem_limit will be the same for all backends. Eventually it could make sense to have different limits per backend depending on the fragments.

      Attachments

        Issue Links

          duplicates

          New Feature - A new feature of the product, which has yet to be developed. IMPALA-8679 Make the query options set in the dynamic resource pool/admission control un-overridable in the user session

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-7389 Admission control should set aside less memory on dedicated coordinator if coordinator fragment is lightweight

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-8418 Restrict user from setting resource limits to high values at session level

          • Major - Major loss of function.
          • Open

          Activity

            People

              bikramjeet.vig Bikramjeet Vig
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: