Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
Impala 4.3.0
-
None
-
ghx-label-14
Description
impalad version 4.3.0
Query is INSERT SELECT FROM WHERE
with no joins, functions, orders etc
fails with OOM
Query MEM_LIMIT is set to 35Gb
Resource Pool max query MEM_LIMIT range is 20Gb
Cluster has 14 hosts and Admission Control is enabled
so we see in query profile:
Query Options (set by configuration): MEM_LIMIT=37580963840, MT_DOP=8, ..
Query Options (set by configuration and planner): MEM_LIMIT=37580963840, MT_DOP=8, ..
Cluster Memory Admitted: 280.00 GB
Max Per-Host Resource Reservation: Memory=128.00MB Threads=8
Per-Host Resource Estimates: Memory=123.54GB
Query(x:y): memory limit exceeded. Limit=20.00 GB Reservation=192.25 MB ReservationLimit=16.00 GB OtherMemory=19.81 GB Total=20.00 GB Peak=20.00 GB
The same query with MEM_LIMIT=0 and MT_DOP=0 is successful with:
Cluster Memory Admitted 280.00 GB
Max Per-Host Resource Reservation: Memory=16.00MB Threads=2
Per-Host Resource Estimates: Memory=35.02GB
The question is why planner uses 35G but not 20G
And perhaps, it has impact on memory utilization