Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-13251

Planner uses session MEM_LIMIT with Max MEM_LIMIT set on Resource Pool

    XMLWordPrintableJSON

Details

    • ghx-label-14

    Description

      impalad version 4.3.0

      Query is INSERT SELECT FROM WHERE
      with no joins, functions, orders etc
      fails with OOM

      Query MEM_LIMIT is set to 35Gb
      Resource Pool max query MEM_LIMIT range is 20Gb

      Cluster has 14 hosts and Admission Control is enabled
      so we see in query profile:
      Query Options (set by configuration): MEM_LIMIT=37580963840, MT_DOP=8, ..
      Query Options (set by configuration and planner): MEM_LIMIT=37580963840, MT_DOP=8, ..
      Cluster Memory Admitted: 280.00 GB
      Max Per-Host Resource Reservation: Memory=128.00MB Threads=8
      Per-Host Resource Estimates: Memory=123.54GB
      Query(x:y): memory limit exceeded. Limit=20.00 GB Reservation=192.25 MB ReservationLimit=16.00 GB OtherMemory=19.81 GB Total=20.00 GB Peak=20.00 GB

      The same query with MEM_LIMIT=0 and MT_DOP=0 is successful with:
      Cluster Memory Admitted 280.00 GB
      Max Per-Host Resource Reservation: Memory=16.00MB Threads=2
      Per-Host Resource Estimates: Memory=35.02GB

      The question is why planner uses 35G but not 20G
      And perhaps, it has impact on memory utilization

      Attachments

        Activity

          People

            Unassigned Unassigned
            yander Andrey Popkov
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: