Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6460

More flexible memory-based admission control policies

    XMLWordPrintableJSON

Details

    • Memory-based admission control policies
    • ghx-label-8

    Description

      Currently there are only two ways to decided how much memory to reserve for each query in memory-based admission control:

      • Using the memory estimates, which often makes bad decisions (e.g. huge overestimates) and doesn't have any enforcement on the backend
      • Using a static pool or user-set mem_limit, which is very difficult to set to a reasonable value for all queries.

      The memory reservation work will allow us to come up with more powerful and flexible polices

      Attachments

        Issue Links

          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-4835 HDFS scans should operate with a constrained number of I/O buffers

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-2945 Pre-aggregation cardinality estimates do not take into account data distribution

          • Major - Major loss of function.
          • Open
          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-6856 High memory estimates in Impala 2.11

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-6847 Consider adding workaround for high memory estimates in admission control

          • Major - Major loss of function.
          • Resolved
          relates to

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. IMPALA-4834 All operators should operate within a memory constraint

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: